diff --git a/en_US.ISO8859-1/articles/dialup-firewall/article.sgml b/en_US.ISO8859-1/articles/dialup-firewall/article.sgml index a1dd71ce69..8ec0013ac1 100644 --- a/en_US.ISO8859-1/articles/dialup-firewall/article.sgml +++ b/en_US.ISO8859-1/articles/dialup-firewall/article.sgml @@ -1,362 +1,358 @@ %man; ]>
Dialup firewalling with FreeBSD Marc Silver
marcs@draenor.org
 - $Date: 2001-07-06 12:50:08 $ + $Date: 2001-07-06 13:02:48 $ This article documents how to setup a firewall using a PPP dialup with FreeBSD and IPFW, and specifically with firewalling over a dialup with a dynamically assigned IP address. This document does not cover setting up your PPP connection in the first place. Preface Dialup Firewalling with FreeBSD This document aims to cover the process that is required in order to setup firewalling with FreeBSD when are dynamically assigned an IP address by your ISP. While every effort has been made to make this document as informative and correct as possible, you are welcome to mail your comments/suggestions to the marcs@draenor.org. Kernel Options The first thing you'll need to do is recompile your kernel in FreeBSD. If you need more information on how to recompile the kernel, then the best place to start is the kernel configuration section in the Handbook. You need to compile the following options into the kernel: options IPFIREWALL Enables the kernel's firewall code. options IPFIREWALL_VERBOSE Sends logged packets to the system logger. options IPFIREWALL_VERBOSE_LIMIT=100 Limits the number of times a matching entry is logged. This stops your log files filling up with lots of repetitive entries. 100 is a reasonable number to use, but you can adjust it based on your requirements. options IPDIVERT Enables divert sockets, which will be shown later. There are also some other OPTIONAL items that you can compile into the kernel for some added security. These are not required in order to get firewalling to work, but some more paranoid users may want to use them. options TCP_RESTRICT_RST This option blocks all TCP RST packets. This is best used for systems that might be exposed to SYN flooding (IRC Servers are a good example) or for those who do not want to be easily portscannable. options TCP_DROP_SYNFIN This option ignores TCP packets with SYN and FIN. This prevents tools such as nmap etc from identifying the TCP/IP stack of the machine, but breaks support for RFC1644 extensions. This is NOT recommended if the machine will be running a web server. Don't reboot once you have recompiled the kernel. Hopefully, we will need to reboot just once in order to complete the installing of the firewall. Changing <filename>/etc/rc.conf</filename> to load the firewall We now need to make some changes to /etc/rc.conf in order to tell it about the firewall. Simply add the following lines: firewall_enable="YES" firewall_script="/etc/firewall/fwrules" natd_enable="YES" natd_interface="tun0" natd_flags="-dynamic" For more information on what the above do take a look at /etc/defaults/rc.conf and read &man.rc.conf.5; Disable PPP's network address translation You may already be using PPP's built in network address translation (NAT). If that is the case you will have to disable it, as these examples use &man.natd.8; to do the same. If you already have a block of entries to automatically start PPP it probably looks like this: ppp_enable="YES" ppp_mode="auto" ppp_nat="YES" ppp_profile="profile" If so, remove the ppp_nat="YES" line. You will also need to remove any nat enable yes or alias enable yes in /etc/ppp/ppp.conf. The ruleset for the firewall We're nearly done now. All that remains now is to define the firewall rules and then we can reboot and the firewall should be up and running. I realize that everyone will want something slightly different when it comes to their rulebase. What I've tried to do is write a rulebase that suits most dialup users. You can obviously modify it to your needs by simply using the following rules as the foundation for your own rulebase. First, let's start with the basics of closed firewalling. What you want to do is deny everything by default and then only open up for the things you really need. Rules should be in the order of allow first and then deny. The premise is that you add the rules for your allows, and then everything else is denied. :) Now, let's make the dir /etc/firewall. Change into the directory and edit the file fwrules as we specified in rc.conf. Please note that you can change this filename to be anything you wish. This guide just gives an example of a filename. Now, let's look at a sample firewall file, and we'll detail everything in it. # Firewall rules # Written by Marc Silver (marcs@draenor.org) # http://draenor.org/ipfw # Freely distributable # Define the firewall command (as in /etc/rc.firewall) for easy # reference. Helps to make it easier to read. fwcmd="/sbin/ipfw" # Force a flushing of the current rules before we reload. $fwcmd -f flush # Divert all packets through the tunnel interface. $fwcmd add divert natd all from any to any via tun0 # Allow all data from my network card and localhost. Make sure you # change your network card (mine was fxp0) before you reboot. :) $fwcmd add allow ip from any to any via lo0 $fwcmd add allow ip from any to any via fxp0 # Allow all connections that I initiate. $fwcmd add allow tcp from any to any out xmit tun0 setup # Once connections are made, allow them to stay open. $fwcmd add allow tcp from any to any via tun0 established # Everyone on the internet is allowed to connect to the following # services on the machine. This example shows that people may connect # to ssh and apache. $fwcmd add allow tcp from any to any 80 setup $fwcmd add allow tcp from any to any 22 setup # This sends a RESET to all ident packets. $fwcmd add reset log tcp from any to any 113 in recv tun0 # Allow outgoing DNS queries ONLY to the specified servers. $fwcmd add allow udp from any to x.x.x.x 53 out xmit tun0 # Allow them back in with the answers... :) $fwcmd add allow udp from x.x.x.x 53 to any in recv tun0 # Allow ICMP (for ping and traceroute to work). You may wish to # disallow this, but I feel it suits my needs to keep them in. $fwcmd add 65435 allow icmp from any to any # Deny all the rest. $fwcmd add 65435 deny log ip from any to any You now have a fully functional firewall that will allow on connections to ports 80 and 22 and will log any other connection attempts. Now, you should be able to safely reboot and your firewall should come up fine. If you find this incorrect in anyway or experience any problems, or have any suggestions to improve this page, please email me. Questions Why are you using natd and ipfw when you could be using the built in ppp-filters? I'll have to be honest and say there's no definitive reason why I use ipfw and natd instead of the built in ppp filters. From the discussions I've had with people the consensus seems to be that while ipfw is certainly more powerful and more configurable than the ppp filters, what it makes up for in functionality it loses in being easy to customise. One of the reasons I use it is because I prefer firewalling to be done at a kernel level rather than by a userland program. If I'm using private addresses internally, such as in the 192.168.0.0 range, could I add a command like $fwcmd add deny all from any to 192.168.0.0:255.255.0.0 via tun0 to the firewall rules to prevent outside attempts to connect to internal machines? The simple answer is no. The reason for this is that natd is doing address translation for anything being diverted through the tun0 device. As far as it's concerned incoming packets will speak only to the dynamically assigned IP address and NOT to the internal network. Note though that you can add a rule like $fwcmd add deny all from 192.168.0.4:255.255.0.0 to any via tun0 which would limit a host on your internal network from going out via the firewall. There must be something wrong. I followed your instructions to the letter and now I am locked out. This tutorial assumes that you are running userland-ppp, therefore the supplied ruleset operates on the tun0 interface, which corresponds to the first connection made with &man.ppp.8; (a.k.a. user-ppp). Additional connections would use tun1, tun2 and so on. You should also note that &man.pppd.8; uses the ppp0 interface instead, so if you start the connection with &man.pppd.8; you must substitute tun0 for ppp0. A quick way to edit the firewall rules to reflect this change is shown below. The original ruleset is backed up as fwrules_tun0. - - &prompt.user; cd /etc/firewall + &prompt.user; cd /etc/firewall /etc/firewall&prompt.user; su Password: /etc/firewall&prompt.root; mv fwrules fwrules_tun0 /etc/firewall&prompt.root; cat fwrules_tun0 | sed s/tun0/ppp0/g > fwrules To know whether you are currently using &man.ppp.8; or &man.pppd.8; you can examine the output of &man.ifconfig.8; once the connection is up. E.g., for a connection made with &man.pppd.8; you would see something like this (showing only the relevant lines): - - &prompt.user; ifconfig + &prompt.user; ifconfig (skipped...) ppp0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1524 inet xxx.xxx.xxx.xxx --> xxx.xxx.xxx.xxx netmask 0xff000000 (skipped...) On the other hand, for a connection made with &man.ppp.8; (user-ppp) you should see something similar to this: - - &prompt.user; ifconfig + &prompt.user; ifconfig (skipped...) ppp0: flags=8010<POINTOPOINT,MULTICAST> mtu 1500 (skipped...) tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1524 (IPv6 stuff skipped...) inet xxx.xxx.xxx.xxx --> xxx.xxx.xxx.xxx netmask 0xffffff00 Opened by PID xxxxx - (skipped...) - + (skipped...)
diff --git a/en_US.ISO8859-1/articles/fonts/article.sgml b/en_US.ISO8859-1/articles/fonts/article.sgml index 8d8291fdae..a2f2b5a1a5 100644 --- a/en_US.ISO8859-1/articles/fonts/article.sgml +++ b/en_US.ISO8859-1/articles/fonts/article.sgml @@ -1,988 +1,976 @@ - + %man; ]>
Fonts and FreeBSD A Tutorial Dave Bodenstab
imdave@synet.net
Wed Aug 7, 1996 This document contains a description of the various font files that may be used with FreeBSD and the syscons driver, X11, Ghostscript and Groff. Cookbook examples are provided for switching the syscons display to 80x60 mode, and for using type 1 fonts with the above application programs. Introduction There are many sources of fonts available, and one might ask how they might be used with FreeBSD. The answer can be found by carefully searching the documentation for the component that one would like to use. This is very time consuming, so this tutorial is an attempt to provide a shortcut for others who might be interested. Basic terminology There are many different font formats and associated font file suffixes. A few that will be addressed here are: .pfa, .pfb Postscript type 1 fonts. The .pfa is the Ascii form and .pfb the Binary form. .afm The font metrics associated with a type 1 font. .pfm The printer font metrics associated with a type 1 font. .ttf A TrueType font .fot An indirect reference to a TrueType font (not an actual font) .fon, .fnt Bitmapped screen fonts The .fot file is used by Windows as sort of a symbolic link to the actual TrueType font (.ttf) file. The .fon font files are also used by Windows. I know of no way to use this font format with FreeBSD. What font formats can I use? Which font file format is useful depends on the application being used. FreeBSD by itself uses no fonts. Application programs and/or drivers may make use of the font files. Here is a small cross reference of application/driver to the font type suffixes: Driver syscons .fnt Application Ghostscript .pfa, .pfb, .ttf X11 .pfa, .pfb Groff .pfa, .afm Povray .ttf The .fnt suffix is used quite frequently. I suspect that whenever someone wanted to create a specialized font file for their application, more often than not they chose this suffix. Therefore, it is likely that files with this suffix are not all the same format; specifically, the .fnt files used by syscons under FreeBSD may not be the same format as a .fnt file one encounters in the MSDOS/Windows environment. I have not made any attempt at using other .fnt files other than those provided with FreeBSD. Setting a virtual console to 80x60 line mode First, an 8x8 font must be loaded. To do this, /etc/rc.conf should contain the line (change the font name to an appropriate one for your locale): font8x8="iso-8x8" # font 8x8 from /usr/share/syscons/fonts/* (or NO). The command to actually switch the mode is &man.vidcontrol.1;: - &prompt.user; vidcontrol VGA_80x60 - + &prompt.user; vidcontrol VGA_80x60 Various screen orientated programs, such as &man.vi.1;, must be able to determine the current screen dimensions. As this is achieved this through ioctl calls to the console driver (such as &man.syscons.4;) they will correctly determine the new screen dimensions. To make this more seamless, one can embed these commands in the startup scripts so it takes place when the system boots. To do this is add this line to /etc/rc.conf allscreens_flags="VGA_80x60" # Set this vidcontrol mode for all virtual screens References: &man.rc.conf.5;, &man.vidcontrol.1;. Using type 1 fonts with X11 X11 can use either the .pfa or the .pfb format fonts. The X11 fonts are located in various subdirectories under /usr/X11R6/lib/X11/fonts. Each font file is cross referenced to its X11 name by the contents of the fonts.dir file in each directory. There is already a directory named Type1. The most straight forward way to add a new font is to put it into this directory. A better way is to keep all new fonts in a separate directory and use a symbolic link to the additional font. This allows one to more easily keep track of ones fonts without confusing them with the fonts that were originally provided. For example: Create a directory to contain the font files &prompt.user; mkdir -p /usr/local/share/fonts/type1 &prompt.user; cd /usr/local/share/fonts/type1 Place the .pfa, .pfb and .afm files here One might want to keep readme files, and other documentation for the fonts here also &prompt.user; cp /cdrom/fonts/atm/showboat/showboat.pfb . &prompt.user; cp /cdrom/fonts/atm/showboat/showboat.afm . Maintain an index to cross reference the fonts -&prompt.user; echo showboat - InfoMagic CICA, Dec 1994, /fonts/atm/showboat >>INDEX - +&prompt.user; echo showboat - InfoMagic CICA, Dec 1994, /fonts/atm/showboat >>INDEX Now, to use a new font with X11, one must make the font file available and update the font name files. The X11 font names look like: -bitstream-charter-medium-r-normal-xxx-0-0-0-0-p-0-iso8859-1 | | | | | | | | | | | | \ \ | | | | | \ \ \ \ \ \ \ +----+- character set | | | | \ \ \ \ \ \ \ +- average width | | | | \ \ \ \ \ \ +- spacing | | | \ \ \ \ \ \ +- vertical res. | | | \ \ \ \ \ +- horizontal res. | | | \ \ \ \ +- points | | | \ \ \ +- pixels | | | \ \ \ - foundry family weight slant width additional style - + foundry family weight slant width additional style A new name needs to be created for each new font. If you have some information from the documentation that accompanied the font, then it could serve as the basis for creating the name. If there is no information, then you can get some idea by using &man.strings.1; on the font file. For example: &prompt.user; strings showboat.pfb | more %!FontType1-1.0: Showboat 001.001 %%CreationDate: 1/15/91 5:16:03 PM %%VMusage: 1024 45747 % Generated by Fontographer 3.1 % Showboat 1991 by David Rakowski. Alle Rechte Vorbehalten. FontDirectory/Showboat known{/Showboat findfont dup/UniqueID known{dup /UniqueID get 4962377 eq exch/FontType get 1 eq and}{pop false}ifelse {save true}{false}ifelse}{false}ifelse 12 dict begin /FontInfo 9 dict dup begin /version (001.001) readonly def /FullName (Showboat) readonly def /FamilyName (Showboat) readonly def /Weight (Medium) readonly def /ItalicAngle 0 def /isFixedPitch false def /UnderlinePosition -106 def /UnderlineThickness 16 def /Notice (Showboat 1991 by David Rakowski. Alle Rechte Vorbehalten.) readonly def end readonly def /FontName /Showboat def ---stdin-- - +--stdin-- Using this information, a possible name might be: - -type1-Showboat-medium-r-normal-decorative-0-0-0-0-p-0-iso8859-1 - + -type1-Showboat-medium-r-normal-decorative-0-0-0-0-p-0-iso8859-1 The components of our name are: Foundry Lets just name all the new fonts type1. Family The name of the font. Weight Normal, bold, medium, semibold, etc. From the &man.strings.1; output above, it appears that this font has a weight of medium. Slant roman, italic, oblique, etc. Since the ItalicAngle is zero, roman will be used. Width Normal, wide, condensed, extended, etc. Until it can be examined, the assumption will be normal. Additional style Usually omitted, but this will indicate that the font contains decorative capital letters. Spacing proportional or monospaced. Proportional is used since isFixedPitch is false. All of these names are arbitrary, but one should strive to be compatible with the existing conventions. A font is referenced by name with possible wild cards by an X11 program, so the name chosen should make some sense. One might begin by simply using …-normal-r-normal-…-p-… as the name, and then use &man.xfontsel.1; to examine it and adjust the name based on the appearance of the font. So, to complete our example: Make the font accessible to X11 &prompt.user; cd /usr/X11R6/lib/X11/fonts/Type1 &prompt.user; ln -s /usr/local/share/fonts/type1/showboat.pfb . Edit fonts.dir and fonts.scale, adding the line describing the font and incrementing the number of fonts which is found on the first line. &prompt.user; ex fonts.dir :1p 25 :1c 26 . :$a showboat.pfb -type1-showboat-medium-r-normal-decorative-0-0-0-0-p-0-iso8859-1 . :wq fonts.scale seems to be identical to fonts.dir… &prompt.user; cp fonts.dir fonts.scale Tell X11 that things have changed &prompt.user; xset fp rehash Examine the new font -&prompt.user; xfontsel -pattern -type1-* - +&prompt.user; xfontsel -pattern -type1-* References: &man.xfontsel.1;, &man.xset.1;, The X Windows System in a Nutshell, O'Reilly & Associates. Using type 1 fonts with Ghostscript Ghostscript references a font via its Fontmap file. This must be modified in a similar way to the X11 fonts.dir file. Ghostscript can use either the .pfa or the .pfb format fonts. Using the font from the previous example, here is how to use it with Ghostscript: Put the font in Ghostscript's font directory &prompt.user; cd /usr/local/share/ghostscript/fonts &prompt.user; ln -s /usr/local/share/fonts/type1/showboat.pfb . Edit Fontmap so Ghostscript knows about the font &prompt.user; cd /usr/local/share/ghostscript/4.01 &prompt.user; ex Fontmap :$a /Showboat (showboat.pfb) ; % From CICA /fonts/atm/showboat . :wq Use Ghostscript to examine the font &prompt.user; gs prfont.ps Aladdin Ghostscript 4.01 (1996-7-10) Copyright (C) 1996 Aladdin Enterprises, Menlo Park, CA. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. Loading Times-Roman font from /usr/local/share/ghostscript/fonts/tir_____.pfb... /1899520 581354 1300084 13826 0 done. GS>Showboat DoFont Loading Showboat font from /usr/local/share/ghostscript/fonts/showboat.pfb... 1939688 565415 1300084 16901 0 done. >>showpage, press <return> to continue<< >>showpage, press <return> to continue<< >>showpage, press <return> to continue<< -GS>quit - +GS>quit References: fonts.txt in the Ghostscript 4.01 distribution Using type 1 fonts with Groff Now that the new font can be used by both X11 and Ghostscript, how can one use the new font with groff? First of all, since we are dealing with type 1 postscript fonts, the groff device that is applicable is the ps device. A font file must be created for each font that groff can use. A groff font name is just a file in /usr/share/groff_font/devps. With our example, the font file could be /usr/share/groff_font/devps/SHOWBOAT. The file must be created using tools provided by groff. The first tool is afmtodit. This is not normally installed, so it must be retrieved from the source distribution. I found I had to change the first line of the file, so I did: &prompt.user; cp /usr/src/gnu/usr.bin/groff/afmtodit/afmtodit.pl /tmp &prompt.user; ex /tmp/afmtodit.pl :1c #!/usr/bin/perl -P- . -:wq - +:wq This tool will create the groff font file from the metrics file (.afm suffix.) Continuing with our example: Many .afm files are in Mac format… ^M delimited lines We need to convert them to unix style ^J delimited lines &prompt.user; cd /tmp &prompt.user; cat /usr/local/share/fonts/type1/showboat.afm | tr '\015' '\012' >showboat.afm Now create the groff font file &prompt.user; cd /usr/share/groff_font/devps -&prompt.user; /tmp/afmtodit.pl -d DESC -e text.enc /tmp/showboat.afm generate/textmap SHOWBOAT - +&prompt.user; /tmp/afmtodit.pl -d DESC -e text.enc /tmp/showboat.afm generate/textmap SHOWBOAT The font can now be referenced with the name SHOWBOAT. If ghostscript is used to drive the printers on the system, then nothing more needs to be done. However, if true postscript printers are used, then the font must be down loaded to the printer in order for the font to be used (unless the printer happens to have the showboat font built in or on an accessible font disk.) The final step is to create a down loadable font. The pfbtops tool is used to create the .pfa format of the font, and the download file is modified to reference the new font. The download file must reference the internal name of the font. This can easily be determined from the groff font file as illustrated: Create the .pfa font file -&prompt.user; pfbtops /usr/local/share/fonts/type1/showboat.pfb >showboat.pfa - +&prompt.user; pfbtops /usr/local/share/fonts/type1/showboat.pfb >showboat.pfa Of course, if the .pfa file is already available, just use a symbolic link to reference it. Get the internal font name &prompt.user; fgrep internalname SHOWBOAT internalname Showboat Tell groff that the font must be down loaded &prompt.user; ex download :$a Showboat showboat.pfa . -:wq - +:wq To test the font: &prompt.user; cd /tmp &prompt.user; cat >example.t <<EOF .sp 5 .ps 16 This is an example of the Showboat font: .br .ps 48 .vs (\n(.s+2)p .sp .ft SHOWBOAT ABCDEFGHI .br JKLMNOPQR .br STUVWXYZ .sp .ps 16 .vs (\n(.s+2)p .fp 5 SHOWBOAT .ft R To use it for the first letter of a paragraph, it will look like: .sp 50p \s(48\f5H\s0\fRere is the first sentence of a paragraph that uses the showboat font as its first letter. Additional vertical space must be used to allow room for the larger letter. EOF &prompt.user; groff -Tps example.t >example.ps To use ghostscript/ghostview &prompt.user; ghostview example.ps To print it -&prompt.user; lpr -Ppostscript example.ps - +&prompt.user; lpr -Ppostscript example.ps References: /usr/src/gnu/usr.bin/groff/afmtodit/afmtodit.man, &man.groff.font.5;, &man.groff.char.7;, &man.pfbtops.1;. Converting TrueType fonts to a groff/postscript format for groff This potentially requires a bit of work, simply because it depends on some utilities that are not installed as part of the base system. They are: ttf2pf TrueType to postscript convertsion utilities. This allows conversion of a TrueType font to an ascii font metric (.afm) file. Currently available at http://sunsite.icm.edu.pl/pub/GUST/contrib/BachoTeX98/ttf2pf. Note: These files are postscript programs and must be downloaded to disk by holding down the Shift key when clicking on the link. Otherwise, your browser may try to launch ghostview to view them. The files of interest are: GS_TTF.PS PF2AFM.PS ttf2pf.ps The funny upper/lower case is due to their being intended also for DOS shells. ttf2pf.ps makes use of the others as upper case, so any renaming must be consistent with this. (Actually, GS_TTF.PS and PFS2AFM.PS are supposedly part of the ghostscript distribution, but it's just as easy to use these as an isolated utility. FreeBSD doesn't seem to include the latter.) You also may want to have these installed to /usr/local/share/groff_font/devps(?). afmtodit Creates font files for use with groff from ascii font metrics file. This usually resides in the directory, /usr/src/contrib/groff/afmtodit, and requires some work to get going. If you're paranoid about working in the /usr/src tree, simply copy the contents of the above directory to a work location. In the work area, you'll need to make the utility. Just type: # make -f Makefile.sub afmtodit You may also need to copy /usr/contrib/groff/devps/generate/textmap to /usr/share/groff_font/devps/generate if it doesn't already exist. Once all these utilities are in place, you're ready to commence: Create the .afm file by typing: % gs -dNODISPLAY -q -- ttf2pf.ps TTF_name PS_font_name AFM_name Where, TTF_name is your TrueType font file, PS_font_name is the file name for the .pfa file, AFM_name is the name you wish for the .afm file. If you do not specify output file names for the .pfa or .afm files, then default names will be generated from the TrueType font file name. This also produces a .pfa file, the ascii postscript font metrics file (.pfb is for the binrary form). This won't be needed, but could (I think) be useful for a fontserver. For example, to convert the 30f9 Barcode font using the default file names, use the following command: % gs -dNODISPLAY -- ttf2pf.ps 3of9.ttf Aladdin Ghostscript 5.10 (1997-11-23) Copyright (C) 1997 Aladdin Enterprises, Menlo Park, CA. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. Converting 3of9.ttf to 3of9.pfa and 3of9.afm. If you want the converted fonts to be stored in A.pfa and B.afm, then use this command: % gs -dNODISPLAY -- ttf2pf.ps 3of9.ttf A B Aladdin Ghostscript 5.10 (1997-11-23) Copyright (C) 1997 Aladdin Enterprises, Menlo Park, CA. All rights reserved. This software comes with NO WARRANTY: see the file PUBLIC for details. Converting 3of9.ttf to A.pfa and B.afm. Create the groff postscript file: Change directories to /usr/share/groff_font/devps so as to make the following command easier to execute. You'll probably need root priviledges for this. (Or, if you're paranoid about working there, make sure you reference the files DESC, text.enc and generate/textmap as being in this directory.) % afmtodit -d DESC -e text.enc file.afm \ generate/textmap PS_font_name Where, file.afm is the AFM_name created by ttf2pf.ps above, and PS_font_name is the font name used from that command, as well as the name that &man.groff.1; will use for references to this font. For example, assuming you used the first tiff2pf.ps command above, then the 3of9 Barcode font can be created using the command: % afmtodit -d DESC -e text.enc 3of9.afm \ generate/textmap 3of9 Ensure that the resulting PS_font_name file (e.g., 3of9 in the example above) is located in the directory /usr/share/groff_font/devps by copying or moving it there. Note that if ttf2pf.ps assigns a font name using the one it finds in the TrueType font file and you want to use a different name, you must edit the .afm file prior to running afmtodit. This name must also match the one used in the Fontmap file if you wish to pipe &man.groff.1; into &man.gs.1;. Can TrueType fonts be used with other programs? The TrueType font format is used by Windows, Windows 95, and Mac's. It is quite popular and there are a great number of fonts available in this format. Unfortunately, there are few applications that I am aware of that can use this format: Ghostscript and Povray come to mind. Ghostscript's support, according to the documentation, is rudimentary and the results are likely to be inferior to type 1 fonts. Povray version 3 also has the ability to use TrueType fonts, but I rather doubt many people will be creating documents as a series of raytraced pages :-). This rather dismal situation may soon change. The FreeType Project is currently developing a useful set of FreeType tools: The freetype module is included with XFree86 4.x. For more information please see the FreeBSD Handbook or the XFree86 4.0.2 Fonts page. The xfsft font server for X11 can serve TrueType fonts in addition to regular fonts. Though currently in beta, it is said to be quite useable. See Juliusz Chroboczek's page for further information. Porting instructions for FreeBSD can be found at Stephen Montgomery's software page. xfstt is another font server for X11, available under ftp://sunsite.unc.edu/pub/Linux/X11/fonts. A program called ttf2bdf can produce BDF files suitable for use in an X environment from TrueType files. Linux binaries are said to be available from ftp://crl.nmsu.edu/CLR/multiling/General/. For people requiring the use of Asian TrueType fonts, the XTT font server may be worth a look. Information about XTT can be found at URL: http://hawk.ise.chuo-u.ac.jp/student/person/tshiozak/study/freebsd-at-random/x-tt/index-en.html. and others … The FreeType Projects page is a good starting point for information on these and other free TrueType projects. Where can additional fonts be obtained? Many fonts are available on the Internet. They are either entirely free, or are share-ware. In addition, there are many inexpensive CDROMs available that contain many fonts. Some Internet locations (as of August 1996) are: ftp://ftp.winsite.com (Formerly CICA) http://www.simtel.net/ ftp://ftp.coast.net/ http://af-pc-plloyd.ecel.uwa.edu.au/fonts/index.html http://www.esselte.com/letraset/index.html http://www.inil.com/users/elfring/esf.htm Additional questions What use are the .pfm files? Can one generate the .afm file from a .pfa or .pfb? How to generate the groff character mapping files for postscript fonts with non-standard character names? Can xditview and devX?? devices be setup to access all the new fonts? It would be good to have examples of using TrueType fonts with povray and ghostscript.
diff --git a/en_US.ISO8859-1/articles/multi-os/article.sgml b/en_US.ISO8859-1/articles/multi-os/article.sgml index 63394e6f5f..1a93cfe0a7 100644 --- a/en_US.ISO8859-1/articles/multi-os/article.sgml +++ b/en_US.ISO8859-1/articles/multi-os/article.sgml @@ -1,743 +1,742 @@ - +
Installing and Using FreeBSD With Other Operating Systems Jay Richmond
jayrich@sysc.com
6 August 1996 This document discusses how to make FreeBSD coexist nicely with other popular operating systems such as Linux, MS-DOS, OS/2, and Windows 95. Special thanks to: Annelise Anderson andrsn@stanford.edu, Randall Hopper rhh@ct.picker.com, and Jordan K. Hubbard jkh@time.cdrom.com Overview Most people can't fit these operating systems together comfortably without having a larger hard disk, so special information on large EIDE drives is included. Because there are so many combinations of possible operating systems and hard disk configurations, the section may be of the most use to you. It contains descriptions of specific working computer setups that use multiple operating systems. This document assumes that you have already made room on your hard disk for an additional operating system. Any time you repartition your hard drive, you run the risk of destroying the data on the original partitions. However, if your hard drive is completely occupied by DOS, you might find the FIPS utility (included on the FreeBSD CD-ROM in the \TOOLS directory or via ftp) useful. It lets you repartition your hard disk without destroying the data already on it. There is also a commercial program available called Partition Magic, which lets you size and delete partitions without consequence. Overview of Boot Managers These are just brief descriptions of some of the different boot managers you may encounter. Depending on your computer setup, you may find it useful to use more than one of them on the same system. Boot Easy This is the default boot manager used with FreeBSD. It has the ability to boot most anything, including BSD, OS/2 (HPFS), Windows 95 (FAT and FAT32), and Linux. Partitions are selected with the function keys. OS/2 Boot Manager This will boot FAT, HPFS, FFS (FreeBSD), and EXT2 (Linux). It will also boot FAT32 partitions. Partitions are selected using arrow keys. The OS/2 Boot Manager is the only one to use its own separate partition, unlike the others which use the master boot record (MBR). Therefore, it must be installed below the 1024th cylinder to avoid booting problems. It can boot Linux using LILO when it is part of the boot sector, not the MBR. Go to Linux HOWTOs on the World Wide Web for more information on booting Linux with OS/2's boot manager. OS-BS This is an alternative to Boot Easy. It gives you more control over the booting process, with the ability to set the default partition to boot and the booting timeout. The beta version of this programs allows you to boot by selecting the OS with your arrow keys. It is included on the FreeBSD CD in the \TOOLS directory, and via ftp. LILO, or LInux LOader This is a limited boot manager. It will boot FreeBSD, though some customization work is required in the LILO configuration file. About FAT32 FAT32 is the replacement to the FAT filesystem included in Microsoft's OEM SR2 Beta release, which is expected to be utilitized on computers pre-loaded with Windows 95 towards the end of 1996. It converts the normal FAT file system and allows you to use smaller cluster sizes for larger hard drives. FAT32 also modifies the traditional FAT boot sector and allocation table, making it incompatible with some boot managers. A Typical Installation Let's say I have two large EIDE hard drives, and I want to install FreeBSD, Linux, and Windows 95 on them. Here's how I might do it using these hard disks: /dev/wd0 (first physical hard disk) /dev/wd1 (second hard disk) Both disks have 1416 cylinders. I boot from a MS-DOS or Windows 95 boot disk that contains the FDISK.EXE utility and make a small 50 meg primary partition (35-40 for Windows 95, plus a little breathing room) on the first disk. Also create a larger partition on the second hard disk for my Windows applications and data. I reboot and install Windows 95 (easier said than done) on the C: partition. The next thing I do is install Linux. I'm not sure about all the distributions of Linux, but slackware includes LILO (see ). When I am partitioning out my hard disk with Linux fdisk, I would put all of Linux on the first drive (maybe 300 megs for a nice root partition and some swap space). After I install Linux, and are prompted about installing LILO, make SURE that I install it on the boot sector of my root Linux partition, not in the MBR (master boot record). The remaining hard disk space can go to FreeBSD. I also make sure that my FreeBSD root slice does not go beyond the 1024th cylinder. (The 1024th cylinder is 528 megs into the disk with our hypothetical 720MB disks). I will use the rest of the hard drive (about 270 megs) for the /usr and / slices if I wish. The rest of the second hard disk (size depends on the amount of my Windows application/data partition that I created in step 1 can go to the /usr/src slice and swap space. When viewed with the Windows 95 fdisk utility, my hard drives should now look something like this: - ---------------------------------------------------------------------- + --------------------------------------------------------------------- Display Partition Information Current fixed disk drive: 1 Partition Status Type Volume_Label Mbytes System Usage C: 1 A PRI DOS 50 FAT** 7% 2 A Non-DOS (Linux) 300 43% Total disk space is 696 Mbytes (1 Mbyte = 1048576 bytes) Press Esc to continue --------------------------------------------------------------------- Display Partition Information Current fixed disk drive: 2 Partition Status Type Volume_Label Mbytes System Usage D: 1 A PRI DOS 420 FAT** 60% Total disk space is 696 Mbytes (1 Mbyte = 1048576 bytes) Press Esc to continue --------------------------------------------------------------------- ** May say FAT16 or FAT32 if you are using the OEM SR2 update. See ). Install FreeBSD. I make sure to boot with my first hard disk set at NORMAL in the BIOS. If it is not, I'll have the enter my true disk geometry at boot time (to get this, boot Windows 95 and consult Microsoft Diagnostics (MSD.EXE), or check your BIOS) with the parameter hd0=1416,16,63 where 1416 is the number of cylinders on my hard disk, 16 is the number of heads per track, and 63 is the number of sectors per track on the drive. When partitioning out the hard disk, I make sure to install Boot Easy on the first disk. I don't worry about the second disk, nothing is booting off of it. When I reboot, Boot Easy should recognize my three bootable partitions as DOS (Windows 95), Linux, and BSD (FreeBSD). Special Considerations Most operating systems are very picky about where and how they are placed on the hard disk. Windows 95 and DOS need to be on the first primary partitiin on the first hard disk. OS/2 is the exception. It can be installed on the first or second disk in a primary or extended partition. If you are not sure, keep the beginning of the bootable partitions below the 1024th cylinder. If you install Windows 95 on an existing BSD system, it will destroy the MBR, and you will have to reinstall your previous boot manager. Boot Easy can be reinstalled by using the BOOTINST.EXE utility included in the \TOOLS directory on the CD-ROM, and via ftp. You can also re-start the installation process and go to the partition editor. From there, mark the FreeBSD partition as bootable, select Boot Manager, and then type W to (W)rite out the information to the MBR. You can now reboot, and Boot Easy should then recognize Windows 95 as DOS. Please keep in mind that OS/2 can read FAT and HPFS partitions, but not FFS (FreeBSD) or EXT2 (Linux) partitions. Likewise, Windows 95 can only read and write to FAT and FAT32 (see ) partitions. FreeBSD can read most file systems, but currently cannot read HPFS partitions. Linux can read HPFS partitions, but can't write to them. Recent versions of the Linux kernel (2.x) can read and write to Windows 95 VFAT partitions (VFAT is what gives Windows 95 long file names - it's pretty much the same as FAT). Linux can read and write to most file systems. Got that? I hope so. Examples (section needs work, please send your example to jayrich@sysc.com). FreeBSD+Win95: If you installed FreeBSD after Windows 95, you should see DOS on the Boot Easy menu. This is Windows 95. If you installed Windows 95 after FreeBSD, read above. As long as your hard disk does not have 1024 cylinders you should not have a problem booting. If one of your partitions goes beyond the 1024th cylinder however, and you get messages like invalid system disk under DOS (Windows 95) and FreeBSD will not boot, try looking for a setting in your BIOS called > 1024 cylinder support or NORMAL/LBA mode. DOS may need LBA (Logical Block Addressing) in order to boot correctly. If the idea of switching BIOS settings every time you boot up doesn't appeal to you, you can boot FreeBSD through DOS via the FBSDBOOT.EXE utility on the CD (It should find your FreeBSD partition and boot it.) FreeBSD+OS/2+Win95: Nothing new here. OS/2's boot manger can boot all of these operating systems, so that shouldn't be a problem. FreeBSD+Linux: You can also use Boot Easy to boot both operating systems. FreeBSD+Linux+Win95: (see ) Other Sources of Help There are many Linux HOW-TOs that deal with multiple operating systems on the same hard disk. The Linux+DOS+Win95+OS2 mini-HOWTO offers help on configuring the OS/2 boot manager, and the Linux+FreeBSD mini-HOWTO might be interesting as well. The Linux-HOWTO is also helpful. The NT Loader Hacking Guide provides good information on multibooting Windows NT, '95, and DOS with other operating systems. And Hale Landis's "How It Works" document pack contains some good info on all sorts of disk geometry and booting related topics. You can find it at ftp://fission.dt.wdc.com/pub/otherdocs/pc_systems/how_it_works/allhiw.zip. Finally, don't overlook FreeBSD's kernel documentation on the booting procedure, available in the kernel source distribution (it unpacks to file:/usr/src/sys/i386/boot/biosboot/README.386BSD. Technical Details (Contributed by Randall Hopper, rhh@ct.picker.com) This section attempts to give you enough basic information about your hard disks and the disk booting process so that you can troubleshoot most problems you might encounter when getting set up to boot several operating systems. It starts in pretty basic terms, so you may want to skim down in this section until it begins to look unfamiliar and then start reading. Disk Primer Three fundamental terms are used to describe the location of data on your hard disk: Cylinders, Heads, and Sectors. It's not particularly important to know what these terms relate to except to know that, together, they identify where data is physically on your disk. Your disk has a particular number of cylinders, number of heads, and number of sectors per cylinder-head (a cylinder-head also known nown as a track). Collectively this information defines the "physical disk geometry" for your hard disk. There are typically 512 bytes per sector, and 63 sectors per track, with the number of cylinders and heads varying widely from disk to disk. Thus you can figure the number of bytes of data that'll fit on your own disk by calculating: (# of cylinders) × (# heads) × (63 sectors/track) × (512 bytes/sect) For example, on my 1.6 Gig Western Digital AC31600 EIDE hard disk,that's: (3148 cyl) × (16 heads) × (63 sectors/track) × (512 bytes/sect) which is 1,624,670,208 bytes, or around 1.6 Gig. You can find out the physical disk geometry (number of cylinders, heads, and sectors/track counts) for your hard disks using ATAID or other programs off the net. Your hard disk probably came with this information as well. Be careful though: if you're using BIOS LBA (see ), you can't use just any program to get the physical geometry. This is because many programs (e.g. MSD.EXE or FreeBSD fdisk) don't identify the physical disk geometry; they instead report the translated geometry (virtual numbers from using LBA). Stay tuned for what that means. One other useful thing about these terms. Given 3 numbers—a cylinder number, a head number, and a sector-within-track number—you identify a specific absolute sector (a 512 byte block of data) on your disk. Cylinders and Heads are numbered up from 0, and Sectors are numbered up from 1. For those that are interested in more technical details, information on disk geometry, boot sectors, BIOSes, etc. can be found all over the net. Query Lycos, Yahoo, etc. for boot sector or master boot record. Among the useful info you'll find are Hale Landis's How It Works document pack. See the section for a few pointers to this pack. Ok, enough terminology. We're talking about booting here. The Booting Process On the first sector of your disk (Cyl 0, Head 0, Sector 1) lives the Master Boot Record (MBR). It contains a map of your disk. It identifies up to 4 partitions, each of which is a contiguous chunk of that disk. FreeBSD calls partitions slices to avoid confusion with it's own partitions, but we won't do that here. Each partition can contain its own operating system. Each partition entry in the MBR has a Partition ID, a Start Cylinder/Head/Sector, and an End Cylinder/Head/Sector. The Partition ID tells what type of partition it is (what OS) and the Start/End tells where it is. lists a smattering of some common Partition IDs. Partition IDs ID (hex) Description 01 Primary DOS12 (12-bit FAT) 04 Primary DOS16 (16-bit FAT) 05 Extended DOS 06 Primary big DOS (> 32MB) 0A OS/2 83 Linux (EXT2FS) A5 FreeBSD, NetBSD, 386BSD (UFS)
Note that not all partitions are bootable (e.g. Extended DOS). Some are—some aren't. What makes a partition bootable is the configuration of the Partition Boot Sector that exists at the beginning of each partition. When you configure your favorite boot manager, it looks up the entries in the MBR partition tables of all your hard disks and lets you name the entries in that list. Then when you boot, the boot manager is invoked by special code in the Master Boot Sector of the first probed hard disk on your system. It looks at the MBR partition table entry corresponding to the partition choice you made, uses the Start Cylinder/Head/Sector information for that partition, loads up the Partition Boot Sector for that partition, and gives it control. That Boot Sector for the partition itself contains enough information to start loading the operating system on that partition. One thing we just brushed past that's important to know. All of your hard disks have MBRs. However, the one that's important is the one on the disk that's first probed by the BIOS. If you have only IDE hard disks, its the first IDE disk (e.g. primary disk on first controller). Similarly for SCSI only systems. If you have both IDE and SCSI hard disks though, the IDE disk is typically probed first by the BIOS, so the first IDE disk is the first probed disk. The boot manager you will install will be hooked into the MBR on this first probed hard disk that we've just described. Booting Limitations and Warnings Now the interesting stuff that you need to watch out for. The dreaded 1024 cylinder limit and how BIOS LBA helps The first part of the booting process is all done through the BIOS, (if that's a new term to you, the BIOS is a software chip on your system motherboard which provides startup code for your computer). As such, this first part of the process is subject to the limitations of the BIOS interface. The BIOS interface used to read the hard disk during this period (INT 13H, Subfunction 2) allocates 10 bits to the Cylinder Number, 8 bits to the Head Number, and 6 bits to the Sector Number. This restricts users of this interface (i.e. boot managers hooked into your disk's MBR as well as OS loaders hooked into the Boot Sectors) to the following limits: 1024 cylinders, max 256 heads, max 64 sectors/track, max (actually 63, 0 isn't available) Now big hard disks have lots of cylinders but not a lot of heads, so invariably with big hard disks the number of cylinders is greater than 1024. Given this and the BIOS interface as is, you can't boot off just anywhere on your hard disk. The boot code (the boot manager and the OS loader hooked into all bootable partitions' Boot Sectors) has to reside below cylinder 1024. In fact, if your hard disk is typical and has 16 heads, this equates to: 1024 cyl/disk × 16 heads/disk × 63 sect/(cyl-head) × 512 bytes/sector which is around the often-mentioned 528MB limit. This is where BIOS LBA (Logical Block Addressing) comes in. BIOS LBA gives the user of the BIOS API calls access to physical cylinders above 1024 though the BIOS interfaces by redefining a cylinder. That is, it remaps your cylinders and heads, making it appear through the BIOS as though the disk has fewer cylinders and more heads than it actually does. In other words, it takes advantage of the fact that hard disks have relatively few heads and lots of cylinders by shifting the balance between number of cylinders and number of heads so that both numbers lie below the above-mentioned limits (1024 cylinders, 256 heads). With BIOS LBA, the hard disk size limitation is virtually removed (well, pushed up to 8 Gigabytes anyway). If you have an LBA BIOS, you can put FreeBSD or any OS anywhere you want and not hit the 1024 cylinder limit. To use my 1.6 Gig Western Digital as an example again, it's physical geometry is: (3148 cyl, 16 heads, 63 sectors/track, 512 bytes/sector) However, my BIOS LBA remaps this to: (787 cyl, 64 heads, 63 sectors/track, 512 bytes/sector) giving the same effective size disk, but with cylinder and head counts within the BIOS API's range (Incidentally, I have both Linux and FreeBSD existing on one of my hard disks above the 1024th physical cylinder, and both operating systems boot fine, thanks to BIOS LBA). Boot Managers and Disk Allocation Another gotcha to watch out when installing boot managers is allocating space for your boot manager. It's best to be aware of this issue up front to save yourself from having to reinstall one or more of your OSs. If you followed the discussion in about the Master Boot Sector (where the MBR is), Partition Boot Sectors, and the booting process, you may have been wondering just exactly where on your hard disk that nifty boot manager is going to live. Well, some boot managers are small enough to fit entirely within the Master Boot Sector (Cylinder 0, Head 0, Sector 0) along with the partition table. Others need a bit more room and actually extend a few sectors past the Master Boot Sector in the Cylinder 0 Head 0 track, since that's typically free…typically. That's the catch. Some operating systems (FreeBSD included) let you start their partitions right after the Master Boot Sector at Cylinder 0, Head 0, Sector 2 if you want. In fact, if you give FreeBSD's sysinstall a disk with an empty chunk up front or the whole disk empty, that's where it'll start the FreeBSD partition by default (at least it did when I fell into this trap). Then when you go to install your boot manager, if it's one that occupies a few extra sectors after the MBR, it'll overwrite the front of the first partition's data. In the case of FreeBSD, this overwrites the disk label, and renders your FreeBSD partition unbootable. The easy way to avoid this problem (and leave yourself the flexibility to try different boot managers later) is just to always leave the first full track on your disk unallocated when you partition your disk. That is, leave the space from Cylinder 0, Head 0, Sector 2 through Cylinder 0, Head 0, Sector 63 unallocated, and start your first partition at Cylinder 0, Head 1, Sector 1. For what it's worth, when you create a DOS partition at the front of your disk, DOS leaves this space open by default (this is why some boot managers assume it's free). So creating a DOS partition up at the front of your disk avoids this problem altogether. I like to do this myself, creating 1 Meg DOS partition up front, because it also avoids my primary DOS drive letters shifting later when I repartition. For reference, the following boot managers use the Master Boot Sector to store their code and data: OS-BS 1.35 Boot Easy LILO These boot managers use a few additional sectors after the Master Boot Sector: OS-BS 2.0 Beta 8 (sectors 2-5) OS/2's boot manager What if your machine won't boot? At some point when installing boot managers, you might leave the MBR in a state such that your machine won't boot. This is unlikely, but possible when re-FDISKing underneath an already-installed boot manager. If you have a bootable DOS partition on your disk, you can boot off a DOS floppy, and run: A:\> FDISK /MBR to put the original, simple DOS boot code back into the system. You can then boot DOS (and DOS only) off the hard drive. Alternatively, just re-run your boot manager installation program off a bootable floppy.
diff --git a/en_US.ISO8859-1/articles/new-users/article.sgml b/en_US.ISO8859-1/articles/new-users/article.sgml index 99f641c894..90bccba82d 100644 --- a/en_US.ISO8859-1/articles/new-users/article.sgml +++ b/en_US.ISO8859-1/articles/new-users/article.sgml @@ -1,1052 +1,1034 @@ - +
For People New to Both FreeBSD and Unix Annelise Anderson
andrsn@andrsn.stanford.edu
 August 15, 1997 Congratulations on installing FreeBSD! This introduction is for people new to both FreeBSD and Un*x—so it starts with basics. It assumes you're using version 2.0.5 or later of FreeBSD as distributed by BSDi or FreeBSD.org, your system (for now) has a single user (you)—and you're probably pretty good with DOS/Windows or OS/2. Logging in and Getting Out Log in (when you see login:) as a user you created during installation or as root. (Your FreeBSD installation will already have an account for root; root can go anywhere and do anything, including deleting essential files, so be careful!) The symbols &prompt.user; and &prompt.root; in the following stand for the prompt (yours may be different), with &prompt.user; indicating an ordinary user and &prompt.root; indicating root. To log out (and get a new login: prompt) type - &prompt.root; exit - + &prompt.root; exit as often as necessary. Yes, press enter after commands, and remember that Unix is case-sensitive—exit, not EXIT. To shut down the machine type - &prompt.root; /sbin/shutdown -h now - + &prompt.root; /sbin/shutdown -h now Or to reboot type - &prompt.root; /sbin/shutdown -r now - + &prompt.root; /sbin/shutdown -r now or - &prompt.root; /sbin/reboot - + &prompt.root; /sbin/reboot You can also reboot with CtrlAltDelete. Give it a little time to do its work. This is equivalent to /sbin/reboot in recent releases of FreeBSD and is much, much better than hitting the reset button. You don't want to have to reinstall this thing, do you? Adding A User with Root Privileges If you didn't create any users when you installed the system and are thus logged in as root, you should probably create a user now with - &prompt.root; adduser - + &prompt.root; adduser The first time you use adduser, it might ask for some defaults to save. You might want to make the default shell csh instead of sh, if it suggests sh as the default. Otherwise just press enter to accept each default. These defaults are saved in /etc/adduser.conf, an editable file. Suppose you create a user jack with full name Jack Benimble. Give jack a password if security (even kids around who might pound on the keyboard) is an issue. When it asks you if you want to invite jack into other groups, type wheel - Login group is ``jack''. Invite jack into other groups: wheel - + Login group is ``jack''. Invite jack into other groups: wheel This will make it possible to log in as jack and use the su command to become root. Then you won't get scolded any more for logging in as root. You can quit adduser any time by typing CtrlC, and at the end you'll have a chance to approve your new user or simply type n for no. You might want to create a second new user (jill?) so that when you edit jack's login files, you'll have a hot spare in case something goes wrong. Once you've done this, use exit to get back to a login prompt and log in as jack. In general, it's a good idea to do as much work as possible as an ordinary user who doesn't have the power—and risk—of root. If you already created a user and you want the user to be able to su to root, you can log in as root and edit the file /etc/group, adding jack to the first line (the group wheel). But first you need to practice vi, the text editor--or use the simpler text editor, ee, installed on recent version of FreeBSD. To delete a user, use the rmuser command. Looking Around Logged in as an ordinary user, look around and try out some commands that will access the sources of help and information within FreeBSD. Here are some commands and what they do: id Tells you who you are! pwd Shows you where you are—the current working directory. ls Lists the files in the current directory. ls Lists the files in the current directory with a * after executables, a / after directories, and an @ after symbolic links. ls Lists the files in long format—size, date, permissions. ls Lists hidden dot files with the others. If you're root, the dot files show up without the switch. cd Changes directories. cd .. backs up one level; note the space after cd. cd /usr/local goes there. cd ~ goes to the home directory of the person logged in—e.g., /usr/home/jack. Try cd /cdrom, and then ls, to find out if your CDROM is mounted and working. view filename Lets you look at a file (named filename) without changing it. Try view /etc/fstab. :q to quit. cat filename Displays filename on screen. If it's too long and you can see only the end of it, press ScrollLock and use the up-arrow to move backward; you can use ScrollLock with man pages too. Press ScrollLock again to quit scrolling. You might want to try cat on some of the dot files in your home directory—cat .cshrc, cat .login, cat .profile. You'll notice aliases in .cshrc for some of the ls commands (they're very convenient). You can create other aliases by editing .cshrc. You can make these aliases available to all users on the system by putting them in the system-wide csh configuration file, /etc/csh.cshrc. Getting Help and Information Here are some useful sources of help. Text stands for something of your choice that you type in—usually a command or filename. apropos text Everything containing string text in the whatis database. man text The man page for text. The major source of documentation for Un*x systems. man ls will tell you all the ways to use the ls command. Press Enter to move through text, Ctrlb to go back a page, Ctrlf to go forward, q or Ctrlc to quit. which text Tells you where in the user's path the command text is found. locate text All the paths where the string text is found. whatis text Tells you what the command text does and its man page. Typing whatis * will tell you about all the binaries in the current directory. whereis text Finds the file text, giving its full path. You might want to try using whatis on some common useful commands like cat, more, grep, mv, find, tar, chmod, chown, date, and script. more lets you read a page at a time as it does in DOS, e.g., ls -l | more or more filename. The * works as a wildcard—e.g., ls w* will show you files beginning with w. Are some of these not working very well? Both locate and whatis depend on a database that's rebuilt weekly. If your machine isn't going to be left on over the weekend (and running FreeBSD), you might want to run the commands for daily, weekly, and monthly maintenance now and then. Run them as root and give each one time to finish before you start the next one, for now. &prompt.root; periodic daily output omitted &prompt.root; periodic weekly output omitted &prompt.root; periodic monthly -output omitted - +output omitted If you get tired of waiting, press AltF2 to get another virtual console, and log in again. After all, it's a multi-user, multi-tasking system. Nevertheless these commands will probably flash messages on your screen while they're running; you can type clear at the prompt to clear the screen. Once they've run, you might want to look at /var/mail/root and /var/log/messages. Running such commands is part of system administration—and as a single user of a Unix system, you're your own system administrator. Virtually everything you need to be root to do is system administration. Such responsibilities aren't covered very well even in those big fat books on Unix, which seem to devote a lot of space to pulling down menus in windows managers. You might want to get one of the two leading books on systems administration, either Evi Nemeth et.al.'s UNIX System Administration Handbook (Prentice-Hall, 1995, ISBN 0-13-15051-7)—the second edition with the red cover; or Æleen Frisch's Essential System Administration (O'Reilly & Associates, 1993, ISBN 0-937175-80-3). I used Nemeth. Editing Text To configure your system, you need to edit text files. Most of them will be in the /etc directory; and you'll need to su to root to be able to change them. You can use the easy ee, but in the long run the text editor vi is worth learning. There's an excellent tutorial on vi in /usr/src/contrib/nvi/docs/tutorial if you have that installed; otherwise you can get it by ftp to ftp.cdrom.com in the directory FreeBSD/FreeBSD-current/src/contrib/nvi/docs/tutorial. Before you edit a file, you should probably back it up. Suppose you want to edit /etc/rc.conf. You could just use cd /etc to get to the /etc directory and do: - &prompt.root; cp rc.conf rc.conf.orig - + &prompt.root; cp rc.conf rc.conf.orig This would copy rc.conf to rc.conf.orig, and you could later copy rc.conf.orig to rc.conf to recover the original. But even better would be moving (renaming) and then copying back: &prompt.root; mv rc.conf rc.conf.orig -&prompt.root; cp rc.conf.orig rc.conf - +&prompt.root; cp rc.conf.orig rc.conf because the mv command preserves the original date and owner of the file. You can now edit rc.conf. If you want the original back, you'd then mv rc.conf rc.conf.myedit (assuming you want to preserve your edited version) and then &prompt.root; mv rc.conf.orig rc.conf to put things back the way they were. To edit a file, type - &prompt.root; vi filename - + &prompt.root; vi filename Move through the text with the arrow keys. Esc (the escape key) puts vi in command mode. Here are some commands: x delete letter the cursor is on dd delete the entire line (even if it wraps on the screen) i insert text at the cursor a insert text after the cursor Once you type i or a, you can enter text. Esc puts you back in command mode where you can type :w to write your changes to disk and continue editing :wq to write and quit :q! to quit without saving changes /text to move the cursor to text; /Enter (the enter key) to find the next instance of text. G to go to the end of the file nG to go to line n in the file, where n is a number CtrlL to redraw the screen Ctrlb and Ctrlf go back and forward a screen, as they do with more and view. Practice with vi in your home directory by creating a new file with vi filename and adding and deleting text, saving the file, and calling it up again. vi delivers some surprises because it's really quite complex, and sometimes you'll inadvertently issue a command that will do something you don't expect. (Some people actually like vi—it's more powerful than DOS EDIT—find out about the :r command.) Use Esc one or more times to be sure you're in command mode and proceed from there when it gives you trouble, save often with :w, and use :q! to get out and start over (from your last :w) when you need to. Now you can cd to /etc, su to root, use vi to edit the file /etc/group, and add a user to wheel so the user has root privileges. Just add a comma and the user's login name to the end of the first line in the file, press Esc, and use :wq to write the file to disk and quit. Instantly effective. (You didn't put a space after the comma, did you?) Printing Files from DOS At this point you probably don't have the printer working, so here's a way to create a file from a man page, move it to a floppy, and then print it from DOS. Suppose you want to read carefully about changing permissions on files (pretty important). You can use the command man chmod to read about it. The command - &prompt.user; man chmod | col -b > chmod.txt - + &prompt.user; man chmod | col -b > chmod.txt will remove formatting codes and send the man page to the chmod.txt file instead of showing it on your screen. Now put a dos-formatted diskette in your floppy drive a, su to root, and type - &prompt.root; /sbin/mount -t msdos /dev/fd0 /mnt - + &prompt.root; /sbin/mount -t msdos /dev/fd0 /mnt to mount the floppy drive on /mnt. Now (you no longer need to be root, and you can type exit to get back to being user jack) you can go to the directory where you created chmod.txt and copy the file to the floppy with: - &prompt.user; cp chmod.txt /mnt - + &prompt.user; cp chmod.txt /mnt and use ls /mnt to get a directory listing of /mnt, which should show the file chmod.txt. You might especially want to make a file from /sbin/dmesg by typing - &prompt.user; /sbin/dmesg > dmesg.txt - + &prompt.user; /sbin/dmesg > dmesg.txt and copying dmesg.txt to the floppy. /sbin/dmesg is the boot log record, and it's useful to understand it because it shows what FreeBSD found when it booted up. If you ask questions on freebsd-questions@FreeBSD.org or on a USENET group—like FreeBSD isn't finding my tape drive, what do I do?—people will want to know what dmesg has to say. You can now dismount the floppy drive (as root) to get the disk out with - &prompt.root; /sbin/umount /mnt - + &prompt.root; /sbin/umount /mnt and reboot to go to DOS. Copy these files to a DOS directory, call them up with DOS EDIT, Windows Notepad or Wordpad, or a word processor, make a minor change so the file has to be saved, and print as you normally would from DOS or Windows. Hope it works! man pages come out best if printed with the dos print command. (Copying files from FreeBSD to a mounted dos partition is in some cases still a little risky.) Getting the printer printing from FreeBSD involves creating an appropriate entry in /etc/printcap and creating a matching spool directory in /var/spool/output. If your printer is on lpt0 (what dos calls LPT1), you may only need to go to /var/spool/output and (as root) create the directory lpd by typing: mkdir lpd, if it doesn't already exist. Then the printer should respond if it's turned on when the system is booted, and lp or lpr should send a file to the printer. Whether or not the file actually prints depends on configuring it, which is covered in the FreeBSD handbook. Other Useful Commands df shows file space and mounted systems. ps aux shows processes running. ps ax is a narrower form. rm filename remove filename. rm -R dir removes a directory dir and all subdirectories—careful! ls -R lists files in the current directory and all subdirectories; I used a variant, ls -AFR > where.txt, to get a list of all the files in / and (separately) /usr before I found better ways to find files. passwd to change user's password (or root's password) man hier man page on the Unix file system Use find to locate filename in /usr or any of its subdirectories with - &prompt.user; find /usr -name "filename" - + &prompt.user; find /usr -name "filename" You can use * as a wildcard in "filename" (which should be in quotes). If you tell find to search in / instead of /usr it will look for the file(s) on all mounted file systems, including the CDROM and the dos partition. An excellent book that explains Unix commands and utilities is Abrahams & Larson, Unix for the Impatient (2nd ed., Addison-Wesley, 1996). There's also a lot of Unix information on the Internet. Try the Unix Reference Desk. Next Steps You should now have the tools you need to get around and edit files, so you can get everything up and running. There is a great deal of information in the FreeBSD handbook (which is probably on your hard drive) and FreeBSD's web site. A wide variety of packages and ports are on the CDROM as well as the web site. The handbook tells you more about how to use them (get the package if it exists, with pkg_add /cdrom/packages/All/packagename, where packagename is the filename of the package). The cdrom has lists of the packages and ports with brief descriptions in cdrom/packages/index, cdrom/packages/index.txt, and cdrom/ports/index, with fuller descriptions in /cdrom/ports/*/*/pkg/DESCR, where the *s represent subdirectories of kinds of programs and program names respectively. If you find the handbook too sophisticated (what with lndir and all) on installing ports from the cdrom, here's what usually works: Find the port you want, say kermit. There will be a directory for it on the cdrom. Copy the subdirectory to /usr/local (a good place for software you add that should be available to all users) with: - &prompt.root; cp -R /cdrom/ports/comm/kermit /usr/local - + &prompt.root; cp -R /cdrom/ports/comm/kermit /usr/local This should result in a /usr/local/kermit subdirectory that has all the files that the kermit subdirectory on the CDROM has. Next, create the directory /usr/ports/distfiles if it doesn't already exist using mkdir. Now check check /cdrom/ports/distfiles for a file with a name that indicates it's the port you want. Copy that file to /usr/ports/distfiles; in recent versions you can skip this step, as FreeBSD will do it for you. In the case of kermit, there is no distfile. Then cd to the subdirectory of /usr/local/kermit that has the file Makefile. Type - &prompt.root; make all install - + &prompt.root; make all install During this process the port will ftp to get any compressed files it needs that it didn't find on the cdrom or in /usr/ports/distfiles. If you don't have your network running yet and there was no file for the port in /cdrom/ports/distfiles, you will have to get the distfile using another machine and copy it to /usr/ports/distfiles from a floppy or your dos partition. Read Makefile (with cat or more or view) to find out where to go (the master distribution site) to get the file and what its name is. Its name will be truncated when downloaded to DOS, and after you get it into /usr/ports/distfiles you'll have to rename it (with the mv command) to its original name so it can be found. (Use binary file transfers!) Then go back to /usr/local/kermit, find the directory with Makefile, and type make all install. The other thing that happens when installing ports or packages is that some other program is needed. If the installation stops with a message can't find unzip or whatever, you might need to install the package or port for unzip before you continue. Once it's installed type rehash to make FreeBSD reread the files in the path so it knows what's there. (If you get a lot of path not found messages when you use whereis or which, you might want to make additions to the list of directories in the path statement in .cshrc in your home directory. The path statement in Unix does the same kind of work it does in DOS, except the current directory is not (by default) in the path for security reasons; if the command you want is in the directory you're in, you need to type ./ before the command to make it work; no space after the slash.) You might want to get the most recent version of Netscape from their ftp site. (Netscape requires the X Window System.) There's now a FreeBSD version, so look around carefully. Just use gunzip filename and tar xvf filename on it, move the binary to /usr/local/bin or some other place binaries are kept, rehash, and then put the following lines in .cshrc in each user's home directory or (easier) in /etc/csh.cshrc, the system-wide csh start-up file: setenv XKEYSYMDB /usr/X11R6/lib/X11/XKeysymDB setenv XNLSPATH /usr/X11R6/lib/X11/nls This assumes that the file XKeysymDB and the directory nls are in /usr/X11R6/lib/X11; if they're not, find them and put them there. If you originally got Netscape as a port using the CDROM (or ftp), don't replace /usr/local/bin/netscape with the new netscape binary; this is just a shell script that sets up the environment variables for you. Instead rename the new binary to netscape.bin and replace the old binary, which is /usr/local/netscape/netscape. Your Working Environment Your shell is the most important part of your working environment. In DOS, the usual shell is command.com. The shell is what interprets the commands you type on the command line, and thus communicates with the rest of the operating system. You can also write shell scripts, which are like DOS batch files: a series of commands to be run without your intervention. Two shells come installed with FreeBSD: csh and sh. csh is good for command-line work, but scripts should be written with sh (or bash). You can find out what shell you have by typing echo $SHELL. The csh shell is okay, but tcsh does everything csh does and more. It allows you to recall commands with the arrow keys and edit them. It has tab-key completion of filenames (csh uses the escape key), and it lets you switch to the directory you were last in with cd -. It's also much easier to alter your prompt with tcsh. It makes life a lot easier. Here are the three steps for installing a new shell: Install the shell as a port or a package, just as you would any other port or package. Use rehash and which tcsh (assuming you're installing tcsh) to make sure it got installed. As root, edit /etc/shells, adding a line in the file for the new shell, in this case /usr/local/bin/tcsh, and save the file. (Some ports may do this for you.) Use the chsh command to change your shell to tcsh permanently, or type tcsh at the prompt to change your shell without logging in again. It can be dangerous to change root's shell to something other than sh or csh on early versions of FreeBSD and many other versions of Unix; you may not have a working shell when the system puts you into single user mode. The solution is to use su -m to become root, which will give you the tcsh as root, because the shell is part of the environment. You can make this permanent by adding it to your .tcshrc file as an alias with alias su su -m. When tcsh starts up, it will read the /etc/csh.cshrc and /etc/csh.login files, as does csh. It will also read the .login file in your home directory and the .cshrc file as well, unless you provide a .tcshrc file. This you can do by simply copying .cshrc to .tcshrc. Now that you've installed tcsh, you can adjust your prompt. You can find the details in the manual page for tcsh, but here is a line to put in your .tcshrc that will tell you how many commands you have typed, what time it is, and what directory you are in. It also produces a > if you're an ordinary user and a # if you're root, but tsch will do that in any case: set prompt = "%h %t %~ %# " This should go in the same place as the existing set prompt line if there is one, or under "if($?prompt) then" if not. Comment out the old line; you can always switch back to it if you prefer it. Don't forget the spaces and quotes. You can get the .tcshrc reread by typing source .tcshrc. You can get a listing of other environmental variables that have been set by typing env at the prompt. The result will show you your default editor, pager, and terminal type, among possibly many others. A useful command if you log in from a remote location and can't run a program because the terminal isn't capable is setenv TERM vt100. Other As root, you can dismount the CDROM with /sbin/umount /cdrom, take it out of the drive, insert another one, and mount it with /sbin/mount_cd9660 /dev/cd0a /cdrom assuming cd0a is the device name for your CDROM drive. The most recent versions of FreeBSD let you mount the cdrom with just /sbin/mount /cdrom. Using the live file system—the second of FreeBSD's CDROM disks—is useful if you've got limited space. What is on the live file system varies from release to release. You might try playing games from the cdrom. This involves using lndir, which gets installed with the X Window System, to tell the program(s) where to find the necessary files, because they're in the /cdrom file system instead of in /usr and its subdirectories, which is where they're expected to be. Read man lndir. Comments Welcome If you use this guide I'd be interested in knowing where it was unclear and what was left out that you think should be included, and if it was helpful. My thanks to Eugene W. Stark, professor of computer science at SUNY-Stony Brook, and John Fieber for helpful comments. Annelise Anderson, andrsn@andrsn.stanford.edu
diff --git a/en_US.ISO8859-1/articles/programming-tools/article.sgml b/en_US.ISO8859-1/articles/programming-tools/article.sgml index eabcea2da8..12db95ba14 100644 --- a/en_US.ISO8859-1/articles/programming-tools/article.sgml +++ b/en_US.ISO8859-1/articles/programming-tools/article.sgml @@ -1,2256 +1,2234 @@ - +
A User's Guide to FreeBSD Programming Tools James Raynard
jraynard@FreeBSD.org
August 17, 1997 1997 James Raynard This document is an introduction to using some of the programming tools supplied with FreeBSD, although much of it will be applicable to many other versions of Unix. It does not attempt to describe coding in any detail. Most of the document assumes little or no previous programming knowledge, although it is hoped that most programmers will find something of value in it Introduction<anchor id=foo> FreeBSD offers an excellent development environment. Compilers for C, C++, and Fortran and an assembler come with the basic system, not to mention a Perl interpreter and classic Unix tools such as sed and awk. If that is not enough, there are many more compilers and interpreters in the Ports collection. FreeBSD is very compatible with standards such as POSIX and ANSI C, as well with its own BSD heritage, so it is possible to write applications that will compile and run with little or no modification on a wide range of platforms. However, all this power can be rather overwhelming at first if you've never written programs on a Unix platform before. This document aims to help you get up and running, without getting too deeply into more advanced topics. The intention is that this document should give you enough of the basics to be able to make some sense of the documentation. Most of the document requires little or no knowledge of programming, although it does assume a basic competence with using Unix and a willingness to learn! Introduction to Programming A program is a set of instructions that tell the computer to do various things; sometimes the instruction it has to perform depends on what happened when it performed a previous instruction. This section gives an overview of the two main ways in which you can give these instructions, or commands as they are usually called. One way uses an interpreter, the other a compiler. As human languages are too difficult for a computer to understand in an unambiguous way, commands are usually written in one or other languages specially designed for the purpose. Interpreters With an interpreter, the language comes as an environment, where you type in commands at a prompt and the environment executes them for you. For more complicated programs, you can type the commands into a file and get the interpreter to load the file and execute the commands in it. If anything goes wrong, many interpreters will drop you into a debugger to help you track down the problem. The advantage of this is that you can see the results of your commands immediately, and mistakes can be corrected readily. The biggest disadvantage comes when you want to share your programs with someone. They must have the same interpreter, or you must have some way of giving it to them, and they need to understand how to use it. Also users may not appreciate being thrown into a debugger if they press the wrong key! From a performance point of view, interpreters can use up a lot of memory, and generally do not generate code as efficiently as compilers. In my opinion, interpreted languages are the best way to start if you have not done any programming before. This kind of environment is typically found with languages like Lisp, Smalltalk, Perl and Basic. It could also be argued that the Unix shell (sh, csh) is itself an interpreter, and many people do in fact write shell scripts to help with various housekeeping tasks on their machine. Indeed, part of the original Unix philosophy was to provide lots of small utility programs that could be linked together in shell scripts to perform useful tasks. Interpreters available with FreeBSD Here is a list of interpreters that are available as FreeBSD packages, with a brief discussion of some of the more popular interpreted languages. To get one of these packages, all you need to do is to click on the hotlink for the package, then run - &prompt.root; pkg_add package name - + &prompt.root; pkg_add package name as root. Obviously, you will need to have a fully functional FreeBSD 2.1.0 or later system for the package to work! BASIC Short for Beginner's All-purpose Symbolic Instruction Code. Developed in the 1950s for teaching University students to program and provided with every self-respecting personal computer in the 1980s, BASIC has been the first programming language for many programmers. It's also the foundation for Visual Basic. The Bywater Basic Interpreter and the Phil Cockroft's Basic Interpreter (formerly Rabbit Basic) are available as FreeBSD FreeBSD packages Lisp A language that was developed in the late 1950s as an alternative to the number-crunching languages that were popular at the time. Instead of being based on numbers, Lisp is based on lists; in fact the name is short for List Processing. Very popular in AI (Artificial Intelligence) circles. Lisp is an extremely powerful and sophisticated language, but can be rather large and unwieldy. FreeBSD has GNU Common Lisp available as a package. Perl Very popular with system administrators for writing scripts; also often used on World Wide Web servers for writing CGI scripts. The latest version (version 5) comes with FreeBSD. Scheme A dialect of Lisp that is rather more compact and cleaner than Common Lisp. Popular in Universities as it is simple enough to teach to undergraduates as a first language, while it has a high enough level of abstraction to be used in research work. FreeBSD has packages of the Elk Scheme Interpreter, the MIT Scheme Interpreter and the SCM Scheme Interpreter. Icon The Icon Programming Language. Logo Brian Harvey's LOGO Interpreter. Python The Python Object-Oriented Programming Language Compilers Compilers are rather different. First of all, you write your code in a file (or files) using an editor. You then run the compiler and see if it accepts your program. If it did not compile, grit your teeth and go back to the editor; if it did compile and gave you a program, you can run it either at a shell command prompt or in a debugger to see if it works properly. If you run it in the shell, you may get a core dump. Obviously, this is not quite as direct as using an interpreter. However it allows you to do a lot of things which are very difficult or even impossible with an interpreter, such as writing code which interacts closely with the operating system—or even writing your own operating system! It's also useful if you need to write very efficient code, as the compiler can take its time and optimise the code, which would not be acceptable in an interpreter. And distributing a program written for a compiler is usually more straightforward than one written for an interpreter—you can just give them a copy of the executable, assuming they have the same operating system as you. Compiled languages include Pascal, C and C++. C and C++ are rather unforgiving languages, and best suited to more experienced programmers; Pascal, on the other hand, was designed as an educational language, and is quite a good language to start with. Unfortunately, FreeBSD doesn't have any Pascal support, except for a Pascal-to-C converter in the ports. As the edit-compile-run-debug cycle is rather tedious when using separate programs, many commercial compiler makers have produced Integrated Development Environments (IDEs for short). FreeBSD does not have an IDE as such; however it is possible to use Emacs for this purpose. This is discussed in . Compiling with <command>cc</command> This section deals only with the GNU compiler for C and C++, since that comes with the base FreeBSD system. It can be invoked by either cc or gcc. The details of producing a program with an interpreter vary considerably between interpreters, and are usually well covered in the documentation and on-line help for the interpreter. Once you've written your masterpiece, the next step is to convert it into something that will (hopefully!) run on FreeBSD. This usually involves several steps, each of which is done by a separate program. Pre-process your source code to remove comments and do other tricks like expanding macros in C. Check the syntax of your code to see if you have obeyed the rules of the language. If you have not, it will complain! Convert the source code into assembly language—this is very close to machine code, but still understandable by humans. Allegedly. To be strictly accurate, cc converts the source code into its own, machine-independent p-code instead of assembly language at this stage. Convert the assembly language into machine code—yep, we are talking bits and bytes, ones and zeros here. Check that you have used things like functions and global variables in a consistent way. For example, if you have called a non-existent function, it will complain. If you are trying to produce an executable from several source code files, work out how to fit them all together. Work out how to produce something that the system's run-time loader will be able to load into memory and run. Finally, write the executable on the file system. The word compiling is often used to refer to just steps 1 to 4—the others are referred to as linking. Sometimes step 1 is referred to as pre-processing and steps 3-4 as assembling. Fortunately, almost all this detail is hidden from you, as cc is a front end that manages calling all these programs with the right arguments for you; simply typing - &prompt.user; cc foobar.c - + &prompt.user; cc foobar.c will cause foobar.c to be compiled by all the steps above. If you have more than one file to compile, just do something like - &prompt.user; cc foo.c bar.c - + &prompt.user; cc foo.c bar.c Note that the syntax checking is just that—checking the syntax. It will not check for any logical mistakes you may have made, like putting the program into an infinite loop, or using a bubble sort when you meant to use a binary sort. In case you didn't know, a binary sort is an efficient way of sorting things into order and a bubble sort isn't. There are lots and lots of options for cc, which are all in the man page. Here are a few of the most important ones, with examples of how to use them. The output name of the file. If you do not use this option, cc will produce an executable called a.out. The reasons for this are buried in the mists of history. &prompt.user; cc foobar.c executable is a.out &prompt.user; cc -o foobar foobar.c executable is foobar Just compile the file, do not link it. Useful for toy programs where you just want to check the syntax, or if you are using a Makefile. &prompt.user; cc -c foobar.c This will produce an object file (not an executable) called foobar.o. This can be linked together with other object files into an executable. Create a debug version of the executable. This makes the compiler put information into the executable about which line of which source file corresponds to which function call. A debugger can use this information to show the source code as you step through the program, which is very useful; the disadvantage is that all this extra information makes the program much bigger. Normally, you compile with while you are developing a program and then compile a release version without when you're satisfied it works properly. &prompt.user; cc -g foobar.c This will produce a debug version of the program. Note, we didn't use the flag to specify the executable name, so we will get an executable called a.out. Producing a debug version called foobar is left as an exercise for the reader! Create an optimised version of the executable. The compiler performs various clever tricks to try and produce an executable that runs faster than normal. You can add a number after the to specify a higher level of optimisation, but this often exposes bugs in the compiler's optimiser. For instance, the version of cc that comes with the 2.1.0 release of FreeBSD is known to produce bad code with the option in some circumstances. Optimisation is usually only turned on when compiling a release version. &prompt.user; cc -O -o foobar foobar.c This will produce an optimised version of foobar. The following three flags will force cc to check that your code complies to the relevant international standard, often referred to as the ANSI standard, though strictly speaking it is an ISO standard. Enable all the warnings which the authors of cc believe are worthwhile. Despite the name, it will not enable all the warnings cc is capable of. Turn off most, but not all, of the non-ANSI C features provided by cc. Despite the name, it does not guarantee strictly that your code will comply to the standard. Turn off all cc's non-ANSI C features. Without these flags, cc will allow you to use some of its non-standard extensions to the standard. Some of these are very useful, but will not work with other compilers—in fact, one of the main aims of the standard is to allow people to write code that will work with any compiler on any system. This is known as portable code. Generally, you should try to make your code as portable as possible, as otherwise you may have to completely re-write the program later to get it to work somewhere else—and who knows what you may be using in a few years time? - &prompt.user; cc -Wall -ansi -pedantic -o foobar foobar.c - + &prompt.user; cc -Wall -ansi -pedantic -o foobar foobar.c This will produce an executable foobar after checking foobar.c for standard compliance. Specify a function library to be used during when linking. The most common example of this is when compiling a program that uses some of the mathematical functions in C. Unlike most other platforms, these are in a separate library from the standard C one and you have to tell the compiler to add it. The rule is that if the library is called libsomething.a, you give cc the argument . For example, the math library is libm.a, so you give cc the argument . A common gotcha with the math library is that it has to be the last library on the command line. &prompt.user; cc -o foobar foobar.c -lm This will link the math library functions into foobar. If you are compiling C++ code, you need to add , or if you are using FreeBSD 2.2 or later, to the command line argument to link the C++ library functions. Alternatively, you can run c++ instead of cc, which does this for you. c++ can also be invoked as g++ on FreeBSD. &prompt.user; cc -o foobar foobar.cc -lg++ For FreeBSD 2.1.6 and earlier &prompt.user; cc -o foobar foobar.cc -lstdc++ For FreeBSD 2.2 and later &prompt.user; c++ -o foobar foobar.cc Each of these will both produce an executable foobar from the C++ source file foobar.cc. Note that, on Unix systems, C++ source files traditionally end in .C, .cxx or .cc, rather than the MS-DOS style .cpp (which was already used for something else). gcc used to rely on this to work out what kind of compiler to use on the source file; however, this restriction no longer applies, so you may now call your C++ files .cpp with impunity! Common <command>cc</command> Queries and Problems I am trying to write a program which uses the sin() function and I get an error like this. What does it mean? /var/tmp/cc0143941.o: Undefined symbol `_sin' referenced from text segment When using mathematical functions like sin(), you have to tell cc to link in the math library, like so: &prompt.user; cc -o foobar foobar.c -lm All right, I wrote this simple program to practice using . All it does is raise 2.1 to the power of 6. #include <stdio.h> int main() { float f; f = pow(2.1, 6); printf("2.1 ^ 6 = %f\n", f); return 0; } and I compiled it as: &prompt.user; cc temp.c -lm like you said I should, but I get this when I run it: &prompt.user; ./a.out 2.1 ^ 6 = 1023.000000 This is not the right answer! What is going on? When the compiler sees you call a function, it checks if it has already seen a prototype for it. If it has not, it assumes the function returns an int, which is definitely not what you want here. So how do I fix this? The prototypes for the mathematical functions are in math.h. If you include this file, the compiler will be able to find the prototype and it will stop doing strange things to your calculation! #include <math.h> #include <stdio.h> int main() { ... After recompiling it as you did before, run it: &prompt.user; ./a.out 2.1 ^ 6 = 85.766121 If you are using any of the mathematical functions, always include math.h and remember to link in the math library. I compiled a file called foobar.c and I cannot find an executable called foobar. Where's it gone? Remember, cc will call the executable a.out unless you tell it differently. Use the option: &prompt.user; cc -o foobar foobar.c OK, I have an executable called foobar, I can see it when I run ls, but when I type in foobar at the command prompt it tells me there is no such file. Why can it not find it? Unlike MS-DOS, Unix does not look in the current directory when it is trying to find out which executable you want it to run, unless you tell it to. Either type ./foobar, which means run the file called foobar in the current directory, or change your PATH environment variable so that it looks something like bin:/usr/bin:/usr/local/bin:. The dot at the end means look in the current directory if it is not in any of the others. I called my executable test, but nothing happens when I run it. What is going on? Most Unix systems have a program called test in /usr/bin and the shell is picking that one up before it gets to checking the current directory. Either type: &prompt.user; ./test or choose a better name for your program! I compiled my program and it seemed to run all right at first, then there was an error and it said something about core dumped. What does that mean? The name core dump dates back to the very early days of Unix, when the machines used core memory for storing data. Basically, if the program failed under certain conditions, the system would write the contents of core memory to disk in a file called core, which the programmer could then pore over to find out what went wrong. Fascinating stuff, but what I am supposed to do now? Use gdb to analyse the core (see ). When my program dumped core, it said something about a segmentation fault. What's that? This basically means that your program tried to perform some sort of illegal operation on memory; Unix is designed to protect the operating system and other programs from rogue programs. Common causes for this are: Trying to write to a NULL pointer, eg char *foo = NULL; strcpy(foo, "bang!"); Using a pointer that hasn't been initialised, eg char *foo; strcpy(foo, "bang!"); The pointer will have some random value that, with luck, will point into an area of memory that isn't available to your program and the kernel will kill your program before it can do any damage. If you're unlucky, it'll point somewhere inside your own program and corrupt one of your data structures, causing the program to fail mysteriously. Trying to access past the end of an array, eg int bar[20]; bar[27] = 6; Trying to store something in read-only memory, eg char *foo = "My string"; strcpy(foo, "bang!"); Unix compilers often put string literals like "My string" into read-only areas of memory. Doing naughty things with malloc() and free(), eg char bar[80]; free(bar); or char *foo = malloc(27); free(foo); free(foo); Making one of these mistakes will not always lead to an error, but they are always bad practice. Some systems and compilers are more tolerant than others, which is why programs that ran well on one system can crash when you try them on an another. Sometimes when I get a core dump it says bus error. It says in my Unix book that this means a hardware problem, but the computer still seems to be working. Is this true? No, fortunately not (unless of course you really do have a hardware problem…). This is usually another way of saying that you accessed memory in a way you shouldn't have. This dumping core business sounds as though it could be quite useful, if I can make it happen when I want to. Can I do this, or do I have to wait until there's an error? Yes, just go to another console or xterm, do &prompt.user; ps to find out the process ID of your program, and do &prompt.user; kill -ABRT pid where pid is the process ID you looked up. This is useful if your program has got stuck in an infinite loop, for instance. If your program happens to trap SIGABRT, there are several other signals which have a similar effect. Make What is <command>make</command>? When you're working on a simple program with only one or two source files, typing in - &prompt.user; cc file1.c file2.c - + &prompt.user; cc file1.c file2.c is not too bad, but it quickly becomes very tedious when there are several files—and it can take a while to compile, too. One way to get around this is to use object files and only recompile the source file if the source code has changed. So we could have something like: - &prompt.user; cc file1.o file2.o … file37.c &hellip - + &prompt.user; cc file1.o file2.o … file37.c &hellip if we'd changed file37.c, but not any of the others, since the last time we compiled. This may speed up the compilation quite a bit, but doesn't solve the typing problem. Or we could write a shell script to solve the typing problem, but it would have to re-compile everything, making it very inefficient on a large project. What happens if we have hundreds of source files lying about? What if we're working in a team with other people who forget to tell us when they've changed one of their source files that we use? Perhaps we could put the two solutions together and write something like a shell script that would contain some kind of magic rule saying when a source file needs compiling. Now all we need now is a program that can understand these rules, as it's a bit too complicated for the shell. This program is called make. It reads in a file, called a makefile, that tells it how different files depend on each other, and works out which files need to be re-compiled and which ones don't. For example, a rule could say something like if fromboz.o is older than fromboz.c, that means someone must have changed fromboz.c, so it needs to be re-compiled. The makefile also has rules telling make how to re-compile the source file, making it a much more powerful tool. Makefiles are typically kept in the same directory as the source they apply to, and can be called makefile, Makefile or MAKEFILE. Most programmers use the name Makefile, as this puts it near the top of a directory listing, where it can easily be seen. They don't use the MAKEFILE form as block capitals are often used for documentation files like README. Example of using <command>make</command> Here's a very simple make file: foo: foo.c cc -o foo foo.c It consists of two lines, a dependency line and a creation line. The dependency line here consists of the name of the program (known as the target), followed by a colon, then whitespace, then the name of the source file. When make reads this line, it looks to see if foo exists; if it exists, it compares the time foo was last modified to the time foo.c was last modified. If foo does not exist, or is older than foo.c, it then looks at the creation line to find out what to do. In other words, this is the rule for working out when foo.c needs to be re-compiled. The creation line starts with a tab (press the tab key) and then the command you would type to create foo if you were doing it at a command prompt. If foo is out of date, or does not exist, make then executes this command to create it. In other words, this is the rule which tells make how to re-compile foo.c. So, when you type make, it will make sure that foo is up to date with respect to your latest changes to foo.c. This principle can be extended to Makefiles with hundreds of targets—in fact, on FreeBSD, it is possible to compile the entire operating system just by typing make world in the appropriate directory! Another useful property of makefiles is that the targets don't have to be programs. For instance, we could have a make file that looks like this: foo: foo.c cc -o foo foo.c install: cp foo /home/me We can tell make which target we want to make by typing: - &prompt.user; make target - + &prompt.user; make target make will then only look at that target and ignore any others. For example, if we type make foo with the makefile above, make will ignore the install target. If we just type make on its own, make will always look at the first target and then stop without looking at any others. So if we typed make here, it will just go to the foo target, re-compile foo if necessary, and then stop without going on to the install target. Notice that the install target doesn't actually depend on anything! This means that the command on the following line is always executed when we try to make that target by typing make install. In this case, it will copy foo into the user's home directory. This is often used by application makefiles, so that the application can be installed in the correct directory when it has been correctly compiled. This is a slightly confusing subject to try and explain. If you don't quite understand how make works, the best thing to do is to write a simple program like hello world and a make file like the one above and experiment. Then progress to using more than one source file, or having the source file include a header file. The touch command is very useful here—it changes the date on a file without you having to edit it. FreeBSD Makefiles Makefiles can be rather complicated to write. Fortunately, BSD-based systems like FreeBSD come with some very powerful ones as part of the system. One very good example of this is the FreeBSD ports system. Here's the essential part of a typical ports Makefile: MASTER_SITES= ftp://freefall.cdrom.com/pub/FreeBSD/LOCAL_PORTS/ DISTFILES= scheme-microcode+dist-7.3-freebsd.tgz .include <bsd.port.mk> Now, if we go to the directory for this port and type make, the following happens: A check is made to see if the source code for this port is already on the system. If it isn't, an FTP connection to the URL in MASTER_SITES is set up to download the source. The checksum for the source is calculated and compared it with one for a known, good, copy of the source. This is to make sure that the source was not corrupted while in transit. Any changes required to make the source work on FreeBSD are applied—this is known as patching. Any special configuration needed for the source is done. (Many Unix program distributions try to work out which version of Unix they are being compiled on and which optional Unix features are present—this is where they are given the information in the FreeBSD ports scenario). The source code for the program is compiled. In effect, we change to the directory where the source was unpacked and do make—the program's own make file has the necessary information to build the program. We now have a compiled version of the program. If we wish, we can test it now; when we feel confident about the program, we can type make install. This will cause the program and any supporting files it needs to be copied into the correct location; an entry is also made into a package database, so that the port can easily be uninstalled later if we change our mind about it. Now I think you'll agree that's rather impressive for a four line script! The secret lies in the last line, which tells make to look in the system makefile called bsd.port.mk. It's easy to overlook this line, but this is where all the clever stuff comes from—someone has written a makefile that tells make to do all the things above (plus a couple of other things I didn't mention, including handling any errors that may occur) and anyone can get access to that just by putting a single line in their own make file! If you want to have a look at these system makefiles, they're in /usr/share/mk, but it's probably best to wait until you've had a bit of practice with makefiles, as they are very complicated (and if you do look at them, make sure you have a flask of strong coffee handy!) More advanced uses of <command>make</command> Make is a very powerful tool, and can do much more than the simple example above shows. Unfortunately, there are several different versions of make, and they all differ considerably. The best way to learn what they can do is probably to read the documentation—hopefully this introduction will have given you a base from which you can do this. The version of make that comes with FreeBSD is the Berkeley make; there is a tutorial for it in /usr/share/doc/psd/12.make. To view it, do - &prompt.user; zmore paper.ascii.gz - + &prompt.user; zmore paper.ascii.gz in that directory. Many applications in the ports use GNU make, which has a very good set of info pages. If you have installed any of these ports, GNU make will automatically have been installed as gmake. It's also available as a port and package in its own right. To view the info pages for GNU make, you will have to edit the dir file in the /usr/local/info directory to add an entry for it. This involves adding a line like * Make: (make). The GNU Make utility. to the file. Once you have done this, you can type info and then select make from the menu (or in Emacs, do C-h i). Debugging The Debugger The debugger that comes with FreeBSD is called gdb (GNU debugger). You start it up by typing - &prompt.user; gdb progname - + &prompt.user; gdb progname although most people prefer to run it inside Emacs. You can do this by: - M-x gdb RET progname RET - + M-x gdb RET progname RET Using a debugger allows you to run the program under more controlled circumstances. Typically, you can step through the program a line at a time, inspect the value of variables, change them, tell the debugger to run up to a certain point and then stop, and so on. You can even attach to a program that's already running, or load a core file to investigate why the program crashed. It's even possible to debug the kernel, though that's a little trickier than the user applications we'll be discussing in this section. gdb has quite good on-line help, as well as a set of info pages, so this section will concentrate on a few of the basic commands. Finally, if you find its text-based command-prompt style off-putting, there's a graphical front-end for it xxgdb in the ports collection. This section is intended to be an introduction to using gdb and does not cover specialised topics such as debugging the kernel. Running a program in the debugger You'll need to have compiled the program with the option to get the most out of using gdb. It will work without, but you'll only see the name of the function you're in, instead of the source code. If you see a line like: - … (no debugging symbols found) … - + … (no debugging symbols found) … when gdb starts up, you'll know that the program wasn't compiled with the option. At the gdb prompt, type break main. This will tell the debugger to skip over the preliminary set-up code in the program and start at the beginning of your code. Now type run to start the program—it will start at the beginning of the set-up code and then get stopped by the debugger when it calls main(). (If you've ever wondered where main() gets called from, now you know!). You can now step through the program, a line at a time, by pressing n. If you get to a function call, you can step into it by pressing s. Once you're in a function call, you can return from stepping into a function call by pressing f. You can also use up and down to take a quick look at the caller. Here's a simple example of how to spot a mistake in a program with gdb. This is our program (with a deliberate mistake): #include <stdio.h> int bazz(int anint); main() { int i; printf("This is my program\n"); bazz(i); return 0; } int bazz(int anint) { printf("You gave me %d\n", anint); return anint; } This program sets i to be 5 and passes it to a function bazz() which prints out the number we gave it. When we compile and run the program we get &prompt.user; cc -g -o temp temp.c &prompt.user; ./temp This is my program -anint = 4231 - +anint = 4231 That wasn't what we expected! Time to see what's going on! &prompt.user; gdb temp GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.13 (i386-unknown-freebsd), Copyright 1994 Free Software Foundation, Inc. (gdb) break main Skip the set-up code Breakpoint 1 at 0x160f: file temp.c, line 9. gdb puts breakpoint at main() (gdb) run Run as far as main() Starting program: /home/james/tmp/temp Program starts running Breakpoint 1, main () at temp.c:9 gdb stops at main() (gdb) n Go to next line This is my program Program prints out (gdb) s step into bazz() bazz (anint=4231) at temp.c:17 gdb displays stack frame -(gdb) - +(gdb) Hang on a minute! How did anint get to be 4231? Didn't we set it to be 5 in main()? Let's move up to main() and have a look. (gdb) up Move up call stack #1 0x1625 in main () at temp.c:11 gdb displays stack frame (gdb) p i Show us the value of i -$1 = 4231 gdb displays 4231 - +$1 = 4231 gdb displays 4231 Oh dear! Looking at the code, we forgot to initialise i. We meant to put … main() { int i; i = 5; printf("This is my program\n"); &hellip but we left the i=5; line out. As we didn't initialise i, it had whatever number happened to be in that area of memory when the program ran, which in this case happened to be 4231. gdb displays the stack frame every time we go into or out of a function, even if we're using up and down to move around the call stack. This shows the name of the function and the values of its arguments, which helps us keep track of where we are and what's going on. (The stack is a storage area where the program stores information about the arguments passed to functions and where to go when it returns from a function call). Examining a core file A core file is basically a file which contains the complete state of the process when it crashed. In the good old days, programmers had to print out hex listings of core files and sweat over machine code manuals, but now life is a bit easier. Incidentally, under FreeBSD and other 4.4BSD systems, a core file is called progname.core instead of just core, to make it clearer which program a core file belongs to. To examine a core file, start up gdb in the usual way. Instead of typing break or run, type - (gdb) core progname.core - + (gdb) core progname.core If you're not in the same directory as the core file, you'll have to do dir /path/to/core/file first. You should see something like this: &prompt.user; gdb a.out GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.13 (i386-unknown-freebsd), Copyright 1994 Free Software Foundation, Inc. (gdb) core a.out.core Core was generated by `a.out'. Program terminated with signal 11, Segmentation fault. Cannot access memory at address 0x7020796d. #0 0x164a in bazz (anint=0x5) at temp.c:17 -(gdb) - +(gdb) In this case, the program was called a.out, so the core file is called a.out.core. We can see that the program crashed due to trying to access an area in memory that was not available to it in a function called bazz. Sometimes it's useful to be able to see how a function was called, as the problem could have occurred a long way up the call stack in a complex program. The bt command causes gdb to print out a back-trace of the call stack: (gdb) bt #0 0x164a in bazz (anint=0x5) at temp.c:17 #1 0xefbfd888 in end () #2 0x162c in main () at temp.c:11 -(gdb) - +(gdb) The end() function is called when a program crashes; in this case, the bazz() function was called from main(). Attaching to a running program One of the neatest features about gdb is that it can attach to a program that's already running. Of course, that assumes you have sufficient permissions to do so. A common problem is when you are stepping through a program that forks, and you want to trace the child, but the debugger will only let you trace the parent. What you do is start up another gdb, use ps to find the process ID for the child, and do - (gdb) attach pid - + (gdb) attach pid in gdb, and then debug as usual. That's all very well, you're probably thinking, but by the time I've done that, the child process will be over the hill and far away. Fear not, gentle reader, here's how to do it (courtesy of the gdb info pages): &hellip if ((pid = fork()) < 0) /* _Always_ check this */ error(); else if (pid == 0) { /* child */ int PauseMode = 1; while (PauseMode) sleep(10); /* Wait until someone attaches to us */ &hellip } else { /* parent */ - &hellip - + &hellip Now all you have to do is attach to the child, set PauseMode to 0, and wait for the sleep() call to return! Using Emacs as a Development Environment Emacs Unfortunately, Unix systems don't come with the kind of everything-you-ever-wanted-and-lots-more-you-didn't-in-one-gigantic-package integrated development environments that other systems have. At least, not unless you pay out very large sums of money. However, it is possible to set up your own environment. It may not be as pretty, and it may not be quite as integrated, but you can set it up the way you want it. And it's free. And you have the source to it. The key to it all is Emacs. Now there are some people who loathe it, but many who love it. If you're one of the former, I'm afraid this section will hold little of interest to you. Also, you'll need a fair amount of memory to run it—I'd recommend 8MB in text mode and 16MB in X as the bare minimum to get reasonable performance. Emacs is basically a highly customisable editor—indeed, it has been customised to the point where it's more like an operating system than an editor! Many developers and sysadmins do in fact spend practically all their time working inside Emacs, leaving it only to log out. It's impossible even to summarise everything Emacs can do here, but here are some of the features of interest to developers: Very powerful editor, allowing search-and-replace on both strings and regular expressions (patterns), jumping to start/end of block expression, etc, etc. Pull-down menus and online help. Language-dependent syntax highlighting and indentation. Completely customisable. You can compile and debug programs within Emacs. On a compilation error, you can jump to the offending line of source code. Friendly-ish front-end to the info program used for reading GNU hypertext documentation, including the documentation on Emacs itself. Friendly front-end to gdb, allowing you to look at the source code as you step through your program. You can read Usenet news and mail while your program is compiling. And doubtless many more that I've overlooked. Emacs can be installed on FreeBSD using the Emacs port. Once it's installed, start it up and do C-h t to read an Emacs tutorial—that means hold down the control key, press h, let go of the control key, and then press t. (Alternatively, you can you use the mouse to select Emacs Tutorial from the Help menu). Although Emacs does have menus, it's well worth learning the key bindings, as it's much quicker when you're editing something to press a couple of keys than to try and find the mouse and then click on the right place. And, when you're talking to seasoned Emacs users, you'll find they often casually throw around expressions like M-x replace-s RET foo RET bar RET so it's useful to know what they mean. And in any case, Emacs has far too many useful functions for them to all fit on the menu bars. Fortunately, it's quite easy to pick up the key-bindings, as they're displayed next to the menu item. My advice is to use the menu item for, say, opening a file until you understand how it works and feel confident with it, then try doing C-x C-f. When you're happy with that, move on to another menu command. If you can't remember what a particular combination of keys does, select Describe Key from the Help menu and type it in—Emacs will tell you what it does. You can also use the Command Apropos menu item to find out all the commands which contain a particular word in them, with the key binding next to it. By the way, the expression above means hold down the Meta key, press x, release the Meta key, type replace-s (short for replace-string—another feature of Emacs is that you can abbreviate commands), press the return key, type foo (the string you want replaced), press the return key, type bar (the string you want to replace foo with) and press return again. Emacs will then do the search-and-replace operation you've just requested. If you're wondering what on earth the Meta key is, it's a special key that many Unix workstations have. Unfortunately, PC's don't have one, so it's usually the alt key (or if you're unlucky, the escape key). Oh, and to get out of Emacs, do C-x C-c (that means hold down the control key, press x, press c and release the control key). If you have any unsaved files open, Emacs will ask you if you want to save them. (Ignore the bit in the documentation where it says C-z is the usual way to leave Emacs—that leaves Emacs hanging around in the background, and is only really useful if you're on a system which doesn't have virtual terminals). Configuring Emacs Emacs does many wonderful things; some of them are built in, some of them need to be configured. Instead of using a proprietary macro language for configuration, Emacs uses a version of Lisp specially adapted for editors, known as Emacs Lisp. This can be quite useful if you want to go on and learn something like Common Lisp, as it's considerably smaller than Common Lisp (although still quite big!). The best way to learn Emacs Lisp is to download the Emacs Tutorial However, there's no need to actually know any Lisp to get started with configuring Emacs, as I've included a sample .emacs file, which should be enough to get you started. Just copy it into your home directory and restart Emacs if it's already running; it will read the commands from the file and (hopefully) give you a useful basic setup. A sample <filename>.emacs</filename> file Unfortunately, there's far too much here to explain it in detail; however there are one or two points worth mentioning. Everything beginning with a ; is a comment and is ignored by Emacs. In the first line, the -*- Emacs-Lisp -*- is so that we can edit the .emacs file itself within Emacs and get all the fancy features for editing Emacs Lisp. Emacs usually tries to guess this based on the filename, and may not get it right for .emacs. The tab key is bound to an indentation function in some modes, so when you press the tab key, it will indent the current line of code. If you want to put a tab character in whatever you're writing, hold the control key down while you're pressing the tab key. This file supports syntax highlighting for C, C++, Perl, Lisp and Scheme, by guessing the language from the filename. Emacs already has a pre-defined function called next-error. In a compilation output window, this allows you to move from one compilation error to the next by doing M-n; we define a complementary function, previous-error, that allows you to go to a previous error by doing M-p. The nicest feature of all is that C-c C-c will open up the source file in which the error occurred and jump to the appropriate line. We enable Emacs's ability to act as a server, so that if you're doing something outside Emacs and you want to edit a file, you can just type in &prompt.user; emacsclient filename and then you can edit the file in your Emacs! Many Emacs users set their EDITOR environment to emacsclient so this happens every time they need to edit a file. A sample <filename>.emacs</filename> file ;; -*-Emacs-Lisp-*- ;; This file is designed to be re-evaled; use the variable first-time ;; to avoid any problems with this. (defvar first-time t "Flag signifying this is the first time that .emacs has been evaled") ;; Meta (global-set-key "\M- " 'set-mark-command) (global-set-key "\M-\C-h" 'backward-kill-word) (global-set-key "\M-\C-r" 'query-replace) (global-set-key "\M-r" 'replace-string) (global-set-key "\M-g" 'goto-line) (global-set-key "\M-h" 'help-command) ;; Function keys (global-set-key [f1] 'manual-entry) (global-set-key [f2] 'info) (global-set-key [f3] 'repeat-complex-command) (global-set-key [f4] 'advertised-undo) (global-set-key [f5] 'eval-current-buffer) (global-set-key [f6] 'buffer-menu) (global-set-key [f7] 'other-window) (global-set-key [f8] 'find-file) (global-set-key [f9] 'save-buffer) (global-set-key [f10] 'next-error) (global-set-key [f11] 'compile) (global-set-key [f12] 'grep) (global-set-key [C-f1] 'compile) (global-set-key [C-f2] 'grep) (global-set-key [C-f3] 'next-error) (global-set-key [C-f4] 'previous-error) (global-set-key [C-f5] 'display-faces) (global-set-key [C-f8] 'dired) (global-set-key [C-f10] 'kill-compilation) ;; Keypad bindings (global-set-key [up] "\C-p") (global-set-key [down] "\C-n") (global-set-key [left] "\C-b") (global-set-key [right] "\C-f") (global-set-key [home] "\C-a") (global-set-key [end] "\C-e") (global-set-key [prior] "\M-v") (global-set-key [next] "\C-v") (global-set-key [C-up] "\M-\C-b") (global-set-key [C-down] "\M-\C-f") (global-set-key [C-left] "\M-b") (global-set-key [C-right] "\M-f") (global-set-key [C-home] "\M-<") (global-set-key [C-end] "\M->") (global-set-key [C-prior] "\M-<") (global-set-key [C-next] "\M->") ;; Mouse (global-set-key [mouse-3] 'imenu) ;; Misc (global-set-key [C-tab] "\C-q\t") ; Control tab quotes a tab. (setq backup-by-copying-when-mismatch t) ;; Treat 'y' or <CR> as yes, 'n' as no. (fset 'yes-or-no-p 'y-or-n-p) (define-key query-replace-map [return] 'act) (define-key query-replace-map [?\C-m] 'act) ;; Load packages (require 'desktop) (require 'tar-mode) ;; Pretty diff mode (autoload 'ediff-buffers "ediff" "Intelligent Emacs interface to diff" t) (autoload 'ediff-files "ediff" "Intelligent Emacs interface to diff" t) (autoload 'ediff-files-remote "ediff" "Intelligent Emacs interface to diff") (if first-time (setq auto-mode-alist (append '(("\\.cpp$" . c++-mode) ("\\.hpp$" . c++-mode) ("\\.lsp$" . lisp-mode) ("\\.scm$" . scheme-mode) ("\\.pl$" . perl-mode) ) auto-mode-alist))) ;; Auto font lock mode (defvar font-lock-auto-mode-list (list 'c-mode 'c++-mode 'c++-c-mode 'emacs-lisp-mode 'lisp-mode 'perl-mode 'scheme-mode) "List of modes to always start in font-lock-mode") (defvar font-lock-mode-keyword-alist '((c++-c-mode . c-font-lock-keywords) (perl-mode . perl-font-lock-keywords)) "Associations between modes and keywords") (defun font-lock-auto-mode-select () "Automatically select font-lock-mode if the current major mode is in font-lock-auto-mode-list" (if (memq major-mode font-lock-auto-mode-list) (progn (font-lock-mode t)) ) ) (global-set-key [M-f1] 'font-lock-fontify-buffer) ;; New dabbrev stuff ;(require 'new-dabbrev) (setq dabbrev-always-check-other-buffers t) (setq dabbrev-abbrev-char-regexp "\\sw\\|\\s_") (add-hook 'emacs-lisp-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) nil) (set (make-local-variable 'dabbrev-case-replace) nil))) (add-hook 'c-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) nil) (set (make-local-variable 'dabbrev-case-replace) nil))) (add-hook 'text-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) t) (set (make-local-variable 'dabbrev-case-replace) t))) ;; C++ and C mode... (defun my-c++-mode-hook () (setq tab-width 4) (define-key c++-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key c++-mode-map "\C-ce" 'c-comment-edit) (setq c++-auto-hungry-initial-state 'none) (setq c++-delete-function 'backward-delete-char) (setq c++-tab-always-indent t) (setq c-indent-level 4) (setq c-continued-statement-offset 4) (setq c++-empty-arglist-indent 4)) (defun my-c-mode-hook () (setq tab-width 4) (define-key c-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key c-mode-map "\C-ce" 'c-comment-edit) (setq c-auto-hungry-initial-state 'none) (setq c-delete-function 'backward-delete-char) (setq c-tab-always-indent t) ;; BSD-ish indentation style (setq c-indent-level 4) (setq c-continued-statement-offset 4) (setq c-brace-offset -4) (setq c-argdecl-indent 0) (setq c-label-offset -4)) ;; Perl mode (defun my-perl-mode-hook () (setq tab-width 4) (define-key c++-mode-map "\C-m" 'reindent-then-newline-and-indent) (setq perl-indent-level 4) (setq perl-continued-statement-offset 4)) ;; Scheme mode... (defun my-scheme-mode-hook () (define-key scheme-mode-map "\C-m" 'reindent-then-newline-and-indent)) ;; Emacs-Lisp mode... (defun my-lisp-mode-hook () (define-key lisp-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key lisp-mode-map "\C-i" 'lisp-indent-line) (define-key lisp-mode-map "\C-j" 'eval-print-last-sexp)) ;; Add all of the hooks... (add-hook 'c++-mode-hook 'my-c++-mode-hook) (add-hook 'c-mode-hook 'my-c-mode-hook) (add-hook 'scheme-mode-hook 'my-scheme-mode-hook) (add-hook 'emacs-lisp-mode-hook 'my-lisp-mode-hook) (add-hook 'lisp-mode-hook 'my-lisp-mode-hook) (add-hook 'perl-mode-hook 'my-perl-mode-hook) ;; Complement to next-error (defun previous-error (n) "Visit previous compilation error message and corresponding source code." (interactive "p") (next-error (- n))) ;; Misc... (transient-mark-mode 1) (setq mark-even-if-inactive t) (setq visible-bell nil) (setq next-line-add-newlines nil) (setq compile-command "make") (setq suggest-key-bindings nil) (put 'eval-expression 'disabled nil) (put 'narrow-to-region 'disabled nil) (put 'set-goal-column 'disabled nil) ;; Elisp archive searching (autoload 'format-lisp-code-directory "lispdir" nil t) (autoload 'lisp-dir-apropos "lispdir" nil t) (autoload 'lisp-dir-retrieve "lispdir" nil t) (autoload 'lisp-dir-verify "lispdir" nil t) ;; Font lock mode (defun my-make-face (face colour &optional bold) "Create a face from a colour and optionally make it bold" (make-face face) (copy-face 'default face) (set-face-foreground face colour) (if bold (make-face-bold face)) ) (if (eq window-system 'x) (progn (my-make-face 'blue "blue") (my-make-face 'red "red") (my-make-face 'green "dark green") (setq font-lock-comment-face 'blue) (setq font-lock-string-face 'bold) (setq font-lock-type-face 'bold) (setq font-lock-keyword-face 'bold) (setq font-lock-function-name-face 'red) (setq font-lock-doc-string-face 'green) (add-hook 'find-file-hooks 'font-lock-auto-mode-select) (setq baud-rate 1000000) (global-set-key "\C-cmm" 'menu-bar-mode) (global-set-key "\C-cms" 'scroll-bar-mode) (global-set-key [backspace] 'backward-delete-char) ; (global-set-key [delete] 'delete-char) (standard-display-european t) (load-library "iso-transl"))) ;; X11 or PC using direct screen writes (if window-system (progn ;; (global-set-key [M-f1] 'hilit-repaint-command) ;; (global-set-key [M-f2] [?\C-u M-f1]) (setq hilit-mode-enable-list '(not text-mode c-mode c++-mode emacs-lisp-mode lisp-mode scheme-mode) hilit-auto-highlight nil hilit-auto-rehighlight 'visible hilit-inhibit-hooks nil hilit-inhibit-rebinding t) (require 'hilit19) (require 'paren)) (setq baud-rate 2400) ; For slow serial connections ) ;; TTY type terminal (if (and (not window-system) (not (equal system-type 'ms-dos))) (progn (if first-time (progn (keyboard-translate ?\C-h ?\C-?) (keyboard-translate ?\C-? ?\C-h))))) ;; Under UNIX (if (not (equal system-type 'ms-dos)) (progn (if first-time (server-start)))) ;; Add any face changes here (add-hook 'term-setup-hook 'my-term-setup-hook) (defun my-term-setup-hook () (if (eq window-system 'pc) (progn ;; (set-face-background 'default "red") ))) ;; Restore the "desktop" - do this as late as possible (if first-time (progn (desktop-load-default) (desktop-read))) ;; Indicate that this file has been read at least once (setq first-time nil) ;; No need to debug anything now (setq debug-on-error nil) ;; All done (message "All done, %s%s" (user-login-name) ".") Extending the Range of Languages Emacs Understands Now, this is all very well if you only want to program in the languages already catered for in the .emacs file (C, C++, Perl, Lisp and Scheme), but what happens if a new language called whizbang comes out, full of exciting features? The first thing to do is find out if whizbang comes with any files that tell Emacs about the language. These usually end in .el, short for Emacs Lisp. For example, if whizbang is a FreeBSD port, we can locate these files by doing - &prompt.user; find /usr/ports/lang/whizbang -name "*.el" -print - + &prompt.user; find /usr/ports/lang/whizbang -name "*.el" -print and install them by copying them into the Emacs site Lisp directory. On FreeBSD 2.1.0-RELEASE, this is /usr/local/share/emacs/site-lisp. So for example, if the output from the find command was - /usr/ports/lang/whizbang/work/misc/whizbang.el - + /usr/ports/lang/whizbang/work/misc/whizbang.el we would do - &prompt.root; cp /usr/ports/lang/whizbang/work/misc/whizbang.el /usr/local/share/emacs/site-lisp - + &prompt.root; cp /usr/ports/lang/whizbang/work/misc/whizbang.el /usr/local/share/emacs/site-lisp Next, we need to decide what extension whizbang source files have. Let's say for the sake of argument that they all end in .wiz. We need to add an entry to our .emacs file to make sure Emacs will be able to use the information in whizbang.el. Find the auto-mode-alist entry in .emacs and add a line for whizbang, such as: … ("\\.lsp$" . lisp-mode) ("\\.wiz$" . whizbang-mode) ("\\.scm$" . scheme-mode) … This means that Emacs will automatically go into whizbang-mode when you edit a file ending in .wiz. Just below this, you'll find the font-lock-auto-mode-list entry. Add whizbang-mode to it like so: ;; Auto font lock mode (defvar font-lock-auto-mode-list (list 'c-mode 'c++-mode 'c++-c-mode 'emacs-lisp-mode 'whizbang-mode 'lisp-mode 'perl-mode 'scheme-mode) "List of modes to always start in font-lock-mode") This means that Emacs will always enable font-lock-mode (ie syntax highlighting) when editing a .wiz file. And that's all that's needed. If there's anything else you want done automatically when you open up a .wiz file, you can add a whizbang-mode hook (see my-scheme-mode-hook for a simple example that adds auto-indent). Further Reading Brian Harvey and Matthew Wright Simply Scheme MIT 1994. ISBN 0-262-08226-8 Randall Schwartz Learning Perl O'Reilly 1993 ISBN 1-56592-042-2 Patrick Henry Winston and Berthold Klaus Paul Horn Lisp (3rd Edition) Addison-Wesley 1989 ISBN 0-201-08319-1 Brian W. Kernighan and Rob Pike The Unix Programming Environment Prentice-Hall 1984 ISBN 0-13-937681-X Brian W. Kernighan and Dennis M. Ritchie The C Programming Language (2nd Edition) Prentice-Hall 1988 ISBN 0-13-110362-8 Bjarne Stroustrup The C++ Programming Language Addison-Wesley 1991 ISBN 0-201-53992-6 W. Richard Stevens Advanced Programming in the Unix Environment Addison-Wesley 1992 ISBN 0-201-56317-7 W. Richard Stevens Unix Network Programming Prentice-Hall 1990 ISBN 0-13-949876-1
diff --git a/en_US.ISO8859-1/books/arch-handbook/driverbasics/chapter.sgml b/en_US.ISO8859-1/books/arch-handbook/driverbasics/chapter.sgml index bab255c26d..cd0a35b2ce 100644 --- a/en_US.ISO8859-1/books/arch-handbook/driverbasics/chapter.sgml +++ b/en_US.ISO8859-1/books/arch-handbook/driverbasics/chapter.sgml @@ -1,379 +1,374 @@ Writing FreeBSD Device Drivers This chapter was written by &a.murray; with selections from a variety of sources including the intro(4) man page by &a.joerg;. Introduction This chapter provides a brief introduction to writing device drivers for FreeBSD. A device in this context is a term used mostly for hardware-related stuff that belongs to the system, like disks, printers, or a graphics display with its keyboard. A device driver is the software component of the operating system that controls a specific device. There are also so-called pseudo-devices where a device driver emulates the behaviour of a device in software without any particular underlying hardware. Device drivers can be compiled into the system statically or loaded on demand through the dynamic kernel linker facility `kld'. Most devices in a Unix-like operating system are accessed through device-nodes, sometimes also called special files. These files are usually located under the directory /dev in the file system hierarchy. Until devfs is fully integrated into FreeBSD, each device node must be created statically and independent of the existence of the associated device driver. Most device nodes on the system are created by running MAKEDEV. Device drivers can roughly be broken down into two categories; character and network device drivers. Dynamic Kernel Linker Facility - KLD The kld interface allows system administrators to dynamically add and remove functionality from a running system. This allows device driver writers to load their new changes into a running kernel without constantly rebooting to test changes. The kld interface is used through the following administrator commands : kldload - loads a new kernel module kldunload - unloads a kernel module kldstat - lists the currently loadded modules Skeleton Layout of a kernel module /* * KLD Skeleton * Inspired by Andrew Reiter's Daemonnews article */ #include <sys/types.h> #include <sys/module.h> #include <sys/systm.h> /* uprintf */ #include <sys/errno.h> #include <sys/param.h> /* defines used in kernel.h */ #include <sys/kernel.h> /* types used in module initialization */ /* * Load handler that deals with the loading and unloading of a KLD. */ static int skel_loader(struct module *m, int what, void *arg) { int err = 0; switch (what) { case MOD_LOAD: /* kldload */ uprintf("Skeleton KLD loaded.\n"); break; case MOD_UNLOAD: uprintf("Skeleton KLD unloaded.\n"); break; default: err = EINVAL; break; } return(err); } /* Declare this module to the rest of the kernel */ DECLARE_MODULE(skeleton, skel_loader, SI_SUB_KLD, SI_ORDER_ANY); Makefile FreeBSD provides a makefile include that you can use to quickly compile your kernel addition. SRCS=skeleton.c KMOD=skeleton .include <bsd.kmod.mk> Simply running make with this makefile will create a file skeleton.ko that can be loaded into your system by typing : &prompt.root - kldload -v ./skeleton.ko - + kldload -v ./skeleton.ko Accessing a device driver Unix provides a common set of system calls for user applications to use. The upper layers of the kernel dispatch these calls to the corresponding device driver when a user accesses a device node. The /dev/MAKEDEV script makes most of the device nodes for your system but if you are doing your own driver development it may be necessary to create your own device nodes with mknod Creating static device nodes The mknod command requires four arguments to create a device node. You must specify the name of this device node, the type of device, the major number of the device, and the minor number of the device. Dynamic device nodes The device filesystem, or devfs, provides access to the kernel's device namespace in the global filesystem namespace. This eliminates the problems of potentially having a device driver without a static device node, or a device node without an installed device driver. Devfs is still a work in progress, but it is already working quite nice. Character Devices A character device driver is one that transfers data directly to and from a user process. This is the most common type of device driver and there are plenty of simple examples in the source tree. This simple example pseudo-device remembers whatever values you write to it and can then supply them back to you when you read from it. /* * Simple `echo' pseudo-device KLD * * Murray Stokely */ #define MIN(a,b) (((a) < (b)) ? (a) : (b)) #include <sys/types.h> #include <sys/module.h> #include <sys/systm.h> /* uprintf */ #include <sys/errno.h> #include <sys/param.h> /* defines used in kernel.h */ #include <sys/kernel.h> /* types used in module initialization */ #include <sys/conf.h> /* cdevsw struct */ #include <sys/uio.h> /* uio struct */ #include <sys/malloc.h> #define BUFFERSIZE 256 /* Function prototypes */ d_open_t echo_open; d_close_t echo_close; d_read_t echo_read; d_write_t echo_write; /* Character device entry points */ static struct cdevsw echo_cdevsw = { echo_open, echo_close, echo_read, echo_write, noioctl, nopoll, nommap, nostrategy, "echo", 33, /* reserved for lkms - /usr/src/sys/conf/majors */ nodump, nopsize, D_TTY, -1 }; typedef struct s_echo { char msg[BUFFERSIZE]; int len; } t_echo; /* vars */ static dev_t sdev; static int len; static int count; static t_echo *echomsg; MALLOC_DECLARE(M_ECHOBUF); MALLOC_DEFINE(M_ECHOBUF, "echobuffer", "buffer for echo module"); /* * This function acts is called by the kld[un]load(2) system calls to * determine what actions to take when a module is loaded or unloaded. */ static int echo_loader(struct module *m, int what, void *arg) { int err = 0; switch (what) { case MOD_LOAD: /* kldload */ sdev = make_dev(&echo_cdevsw, 0, UID_ROOT, GID_WHEEL, 0600, "echo"); /* kmalloc memory for use by this driver */ /* malloc(256,M_ECHOBUF,M_WAITOK); */ MALLOC(echomsg, t_echo *, sizeof(t_echo), M_ECHOBUF, M_WAITOK); printf("Echo device loaded.\n"); break; case MOD_UNLOAD: destroy_dev(sdev); FREE(echomsg,M_ECHOBUF); printf("Echo device unloaded.\n"); break; default: err = EINVAL; break; } return(err); } int echo_open(dev_t dev, int oflags, int devtype, struct proc *p) { int err = 0; uprintf("Opened device \"echo\" successfully.\n"); return(err); } int echo_close(dev_t dev, int fflag, int devtype, struct proc *p) { uprintf("Closing device \"echo.\"\n"); return(0); } /* * The read function just takes the buf that was saved via * echo_write() and returns it to userland for accessing. * uio(9) */ int echo_read(dev_t dev, struct uio *uio, int ioflag) { int err = 0; int amt; /* How big is this read operation? Either as big as the user wants, or as big as the remaining data */ amt = MIN(uio->uio_resid, (echomsg->len - uio->uio_offset > 0) ? echomsg->len - uio->uio_offset : 0); if ((err = uiomove(echomsg->msg + uio->uio_offset,amt,uio)) != 0) { uprintf("uiomove failed!\n"); } return err; } /* * echo_write takes in a character string and saves it * to buf for later accessing. */ int echo_write(dev_t dev, struct uio *uio, int ioflag) { int err = 0; /* Copy the string in from user memory to kernel memory */ err = copyin(uio->uio_iov->iov_base, echomsg->msg, MIN(uio->uio_iov->iov_len,BUFFERSIZE)); /* Now we need to null terminate */ *(echomsg->msg + MIN(uio->uio_iov->iov_len,BUFFERSIZE)) = 0; /* Record the length */ echomsg->len = MIN(uio->uio_iov->iov_len,BUFFERSIZE); if (err != 0) { uprintf("Write failed: bad address!\n"); } count++; return(err); } DEV_MODULE(echo,echo_loader,NULL); To install this driver you will first need to make a node on your filesystem with a command such as : - - &prompt.root mknod /dev/echo c 33 0 - + &prompt.root mknod /dev/echo c 33 0 With this driver loaded you should now be able to type something like : - - &prompt.root echo -n "Test Data" > /dev/echo +&prompt.root echo -n "Test Data" > /dev/echo &prompt.root cat /dev/echo - Test Data - + Test Data Real hardware devices in the next chapter.. Additional Resources Dynamic Kernel Linker (KLD) Facility Programming Tutorial - Daemonnews October 2000 How to Write Kernel Drivers with NEWBUS - Daemonnews July 2000 Network Drivers Drivers for network devices do not use device nodes in order to be accessed. Their selection is based on other decisions made inside the kernel and instead of calling open(), use of a network device is generally introduced by using the system call socket(2). man ifnet(), loopback device, Bill Paul's drivers, etc.. diff --git a/en_US.ISO8859-1/books/developers-handbook/driverbasics/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/driverbasics/chapter.sgml index bab255c26d..cd0a35b2ce 100644 --- a/en_US.ISO8859-1/books/developers-handbook/driverbasics/chapter.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/driverbasics/chapter.sgml @@ -1,379 +1,374 @@ Writing FreeBSD Device Drivers This chapter was written by &a.murray; with selections from a variety of sources including the intro(4) man page by &a.joerg;. Introduction This chapter provides a brief introduction to writing device drivers for FreeBSD. A device in this context is a term used mostly for hardware-related stuff that belongs to the system, like disks, printers, or a graphics display with its keyboard. A device driver is the software component of the operating system that controls a specific device. There are also so-called pseudo-devices where a device driver emulates the behaviour of a device in software without any particular underlying hardware. Device drivers can be compiled into the system statically or loaded on demand through the dynamic kernel linker facility `kld'. Most devices in a Unix-like operating system are accessed through device-nodes, sometimes also called special files. These files are usually located under the directory /dev in the file system hierarchy. Until devfs is fully integrated into FreeBSD, each device node must be created statically and independent of the existence of the associated device driver. Most device nodes on the system are created by running MAKEDEV. Device drivers can roughly be broken down into two categories; character and network device drivers. Dynamic Kernel Linker Facility - KLD The kld interface allows system administrators to dynamically add and remove functionality from a running system. This allows device driver writers to load their new changes into a running kernel without constantly rebooting to test changes. The kld interface is used through the following administrator commands : kldload - loads a new kernel module kldunload - unloads a kernel module kldstat - lists the currently loadded modules Skeleton Layout of a kernel module /* * KLD Skeleton * Inspired by Andrew Reiter's Daemonnews article */ #include <sys/types.h> #include <sys/module.h> #include <sys/systm.h> /* uprintf */ #include <sys/errno.h> #include <sys/param.h> /* defines used in kernel.h */ #include <sys/kernel.h> /* types used in module initialization */ /* * Load handler that deals with the loading and unloading of a KLD. */ static int skel_loader(struct module *m, int what, void *arg) { int err = 0; switch (what) { case MOD_LOAD: /* kldload */ uprintf("Skeleton KLD loaded.\n"); break; case MOD_UNLOAD: uprintf("Skeleton KLD unloaded.\n"); break; default: err = EINVAL; break; } return(err); } /* Declare this module to the rest of the kernel */ DECLARE_MODULE(skeleton, skel_loader, SI_SUB_KLD, SI_ORDER_ANY); Makefile FreeBSD provides a makefile include that you can use to quickly compile your kernel addition. SRCS=skeleton.c KMOD=skeleton .include <bsd.kmod.mk> Simply running make with this makefile will create a file skeleton.ko that can be loaded into your system by typing : &prompt.root - kldload -v ./skeleton.ko - + kldload -v ./skeleton.ko Accessing a device driver Unix provides a common set of system calls for user applications to use. The upper layers of the kernel dispatch these calls to the corresponding device driver when a user accesses a device node. The /dev/MAKEDEV script makes most of the device nodes for your system but if you are doing your own driver development it may be necessary to create your own device nodes with mknod Creating static device nodes The mknod command requires four arguments to create a device node. You must specify the name of this device node, the type of device, the major number of the device, and the minor number of the device. Dynamic device nodes The device filesystem, or devfs, provides access to the kernel's device namespace in the global filesystem namespace. This eliminates the problems of potentially having a device driver without a static device node, or a device node without an installed device driver. Devfs is still a work in progress, but it is already working quite nice. Character Devices A character device driver is one that transfers data directly to and from a user process. This is the most common type of device driver and there are plenty of simple examples in the source tree. This simple example pseudo-device remembers whatever values you write to it and can then supply them back to you when you read from it. /* * Simple `echo' pseudo-device KLD * * Murray Stokely */ #define MIN(a,b) (((a) < (b)) ? (a) : (b)) #include <sys/types.h> #include <sys/module.h> #include <sys/systm.h> /* uprintf */ #include <sys/errno.h> #include <sys/param.h> /* defines used in kernel.h */ #include <sys/kernel.h> /* types used in module initialization */ #include <sys/conf.h> /* cdevsw struct */ #include <sys/uio.h> /* uio struct */ #include <sys/malloc.h> #define BUFFERSIZE 256 /* Function prototypes */ d_open_t echo_open; d_close_t echo_close; d_read_t echo_read; d_write_t echo_write; /* Character device entry points */ static struct cdevsw echo_cdevsw = { echo_open, echo_close, echo_read, echo_write, noioctl, nopoll, nommap, nostrategy, "echo", 33, /* reserved for lkms - /usr/src/sys/conf/majors */ nodump, nopsize, D_TTY, -1 }; typedef struct s_echo { char msg[BUFFERSIZE]; int len; } t_echo; /* vars */ static dev_t sdev; static int len; static int count; static t_echo *echomsg; MALLOC_DECLARE(M_ECHOBUF); MALLOC_DEFINE(M_ECHOBUF, "echobuffer", "buffer for echo module"); /* * This function acts is called by the kld[un]load(2) system calls to * determine what actions to take when a module is loaded or unloaded. */ static int echo_loader(struct module *m, int what, void *arg) { int err = 0; switch (what) { case MOD_LOAD: /* kldload */ sdev = make_dev(&echo_cdevsw, 0, UID_ROOT, GID_WHEEL, 0600, "echo"); /* kmalloc memory for use by this driver */ /* malloc(256,M_ECHOBUF,M_WAITOK); */ MALLOC(echomsg, t_echo *, sizeof(t_echo), M_ECHOBUF, M_WAITOK); printf("Echo device loaded.\n"); break; case MOD_UNLOAD: destroy_dev(sdev); FREE(echomsg,M_ECHOBUF); printf("Echo device unloaded.\n"); break; default: err = EINVAL; break; } return(err); } int echo_open(dev_t dev, int oflags, int devtype, struct proc *p) { int err = 0; uprintf("Opened device \"echo\" successfully.\n"); return(err); } int echo_close(dev_t dev, int fflag, int devtype, struct proc *p) { uprintf("Closing device \"echo.\"\n"); return(0); } /* * The read function just takes the buf that was saved via * echo_write() and returns it to userland for accessing. * uio(9) */ int echo_read(dev_t dev, struct uio *uio, int ioflag) { int err = 0; int amt; /* How big is this read operation? Either as big as the user wants, or as big as the remaining data */ amt = MIN(uio->uio_resid, (echomsg->len - uio->uio_offset > 0) ? echomsg->len - uio->uio_offset : 0); if ((err = uiomove(echomsg->msg + uio->uio_offset,amt,uio)) != 0) { uprintf("uiomove failed!\n"); } return err; } /* * echo_write takes in a character string and saves it * to buf for later accessing. */ int echo_write(dev_t dev, struct uio *uio, int ioflag) { int err = 0; /* Copy the string in from user memory to kernel memory */ err = copyin(uio->uio_iov->iov_base, echomsg->msg, MIN(uio->uio_iov->iov_len,BUFFERSIZE)); /* Now we need to null terminate */ *(echomsg->msg + MIN(uio->uio_iov->iov_len,BUFFERSIZE)) = 0; /* Record the length */ echomsg->len = MIN(uio->uio_iov->iov_len,BUFFERSIZE); if (err != 0) { uprintf("Write failed: bad address!\n"); } count++; return(err); } DEV_MODULE(echo,echo_loader,NULL); To install this driver you will first need to make a node on your filesystem with a command such as : - - &prompt.root mknod /dev/echo c 33 0 - + &prompt.root mknod /dev/echo c 33 0 With this driver loaded you should now be able to type something like : - - &prompt.root echo -n "Test Data" > /dev/echo +&prompt.root echo -n "Test Data" > /dev/echo &prompt.root cat /dev/echo - Test Data - + Test Data Real hardware devices in the next chapter.. Additional Resources Dynamic Kernel Linker (KLD) Facility Programming Tutorial - Daemonnews October 2000 How to Write Kernel Drivers with NEWBUS - Daemonnews July 2000 Network Drivers Drivers for network devices do not use device nodes in order to be accessed. Their selection is based on other decisions made inside the kernel and instead of calling open(), use of a network device is generally introduced by using the system call socket(2). man ifnet(), loopback device, Bill Paul's drivers, etc.. diff --git a/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.sgml index e6028b6cb3..2c1519e6b4 100644 --- a/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.sgml @@ -1,1603 +1,1587 @@ IPv6 Internals IPv6/IPsec Implementation Contributed by &a.shin;, 5 March 2000. This section should explain IPv6 and IPsec related implementation internals. These functionalities are derived from KAME project IPv6 Conformance The IPv6 related functions conforms, or tries to conform to the latest set of IPv6 specifications. For future reference we list some of the relevant documents below (NOTE: this is not a complete list - this is too hard to maintain...). For details please refer to specific chapter in the document, RFCs, manpages, or comments in the source code. Conformance tests have been performed on the KAME STABLE kit at TAHI project. Results can be viewed at http://www.tahi.org/report/KAME/ . We also attended Univ. of New Hampshire IOL tests (http://www.iol.unh.edu/) in the past, with our past snapshots. RFC1639: FTP Operation Over Big Address Records (FOOBAR) RFC2428 is preferred over RFC1639. FTP clients will first try RFC2428, then RFC1639 if failed. RFC1886: DNS Extensions to support IPv6 RFC1933: Transition Mechanisms for IPv6 Hosts and Routers IPv4 compatible address is not supported. automatic tunneling (described in 4.3 of this RFC) is not supported. &man.gif.4; interface implements IPv[46]-over-IPv[46] tunnel in a generic way, and it covers "configured tunnel" described in the spec. See 23.5.1.5 in this document for details. RFC1981: Path MTU Discovery for IPv6 RFC2080: RIPng for IPv6 usr.sbin/route6d support this. RFC2292: Advanced Sockets API for IPv6 For supported library functions/kernel APIs, see sys/netinet6/ADVAPI. RFC2362: Protocol Independent Multicast-Sparse Mode (PIM-SM) RFC2362 defines packet formats for PIM-SM. draft-ietf-pim-ipv6-01.txt is written based on this. RFC2373: IPv6 Addressing Architecture supports node required addresses, and conforms to the scope requirement. RFC2374: An IPv6 Aggregatable Global Unicast Address Format supports 64-bit length of Interface ID. RFC2375: IPv6 Multicast Address Assignments Userland applications use the well-known addresses assigned in the RFC. RFC2428: FTP Extensions for IPv6 and NATs RFC2428 is preferred over RFC1639. FTP clients will first try RFC2428, then RFC1639 if failed. RFC2460: IPv6 specification RFC2461: Neighbor discovery for IPv6 See 23.5.1.2 in this document for details. RFC2462: IPv6 Stateless Address Autoconfiguration See 23.5.1.4 in this document for details. RFC2463: ICMPv6 for IPv6 specification See 23.5.1.9 in this document for details. RFC2464: Transmission of IPv6 Packets over Ethernet Networks RFC2465: MIB for IPv6: Textual Conventions and General Group Necessary statistics are gathered by the kernel. Actual IPv6 MIB support is provided as a patchkit for ucd-snmp. RFC2466: MIB for IPv6: ICMPv6 group Necessary statistics are gathered by the kernel. Actual IPv6 MIB support is provided as patchkit for ucd-snmp. RFC2467: Transmission of IPv6 Packets over FDDI Networks RFC2497: Transmission of IPv6 packet over ARCnet Networks RFC2553: Basic Socket Interface Extensions for IPv6 IPv4 mapped address (3.7) and special behavior of IPv6 wildcard bind socket (3.8) are supported. See 23.5.1.12 in this document for details. RFC2675: IPv6 Jumbograms See 23.5.1.7 in this document for details. RFC2710: Multicast Listener Discovery for IPv6 RFC2711: IPv6 router alert option draft-ietf-ipngwg-router-renum-08: Router renumbering for IPv6 draft-ietf-ipngwg-icmp-namelookups-02: IPv6 Name Lookups Through ICMP draft-ietf-ipngwg-icmp-name-lookups-03: IPv6 Name Lookups Through ICMP draft-ietf-pim-ipv6-01.txt: PIM for IPv6 &man.pim6dd.8; implements dense mode. &man.pim6sd.8; implements sparse mode. draft-itojun-ipv6-tcp-to-anycast-00: Disconnecting TCP connection toward IPv6 anycast address draft-yamamoto-wideipv6-comm-model-00 See 23.5.1.6 in this document for details. draft-ietf-ipngwg-scopedaddr-format-00.txt : An Extension of Format for IPv6 Scoped Addresses Neighbor Discovery Neighbor Discovery is fairly stable. Currently Address Resolution, Duplicated Address Detection, and Neighbor Unreachability Detection are supported. In the near future we will be adding Proxy Neighbor Advertisement support in the kernel and Unsolicited Neighbor Advertisement transmission command as admin tool. If DAD fails, the address will be marked "duplicated" and message will be generated to syslog (and usually to console). The "duplicated" mark can be checked with &man.ifconfig.8;. It is administrators' responsibility to check for and recover from DAD failures. The behavior should be improved in the near future. Some of the network driver loops multicast packets back to itself, even if instructed not to do so (especially in promiscuous mode). In such cases DAD may fail, because DAD engine sees inbound NS packet (actually from the node itself) and considers it as a sign of duplicate. You may want to look at #if condition marked "heuristics" in sys/netinet6/nd6_nbr.c:nd6_dad_timer() as workaround (note that the code fragment in "heuristics" section is not spec conformant). Neighbor Discovery specification (RFC2461) does not talk about neighbor cache handling in the following cases: when there was no neighbor cache entry, node received unsolicited RS/NS/NA/redirect packet without link-layer address neighbor cache handling on medium without link-layer address (we need a neighbor cache entry for IsRouter bit) For first case, we implemented workaround based on discussions on IETF ipngwg mailing list. For more details, see the comments in the source code and email thread started from (IPng 7155), dated Feb 6 1999. IPv6 on-link determination rule (RFC2461) is quite different from assumptions in BSD network code. At this moment, no on-link determination rule is supported where default router list is empty (RFC2461, section 5.2, last sentence in 2nd paragraph - note that the spec misuse the word "host" and "node" in several places in the section). To avoid possible DoS attacks and infinite loops, only 10 options on ND packet is accepted now. Therefore, if you have 20 prefix options attached to RA, only the first 10 prefixes will be recognized. If this troubles you, please ask it on FREEBSD-CURRENT mailing list and/or modify nd6_maxndopt in sys/netinet6/nd6.c. If there are high demands we may provide sysctl knob for the variable. Scope Index IPv6 uses scoped addresses. Therefore, it is very important to specify scope index (interface index for link-local address, or site index for site-local address) with an IPv6 address. Without scope index, scoped IPv6 address is ambiguous to the kernel, and kernel will not be able to determine the outbound interface for a packet. Ordinary userland applications should use advanced API (RFC2292) to specify scope index, or interface index. For similar purpose, sin6_scope_id member in sockaddr_in6 structure is defined in RFC2553. However, the semantics for sin6_scope_id is rather vague. If you care about portability of your application, we suggest you to use advanced API rather than sin6_scope_id. In the kernel, an interface index for link-local scoped address is embedded into 2nd 16bit-word (3rd and 4th byte) in IPv6 address. For example, you may see something like: - - fe80:1::200:f8ff:fe01:6317 + fe80:1::200:f8ff:fe01:6317 in the routing table and interface address structure (struct in6_ifaddr). The address above is a link-local unicast address which belongs to a network interface whose interface identifier is 1. The embedded index enables us to identify IPv6 link local addresses over multiple interfaces effectively and with only a little code change. Routing daemons and configuration programs, like &man.route6d.8; and &man.ifconfig.8;, will need to manipulate the "embedded" scope index. These programs use routing sockets and ioctls (like SIOCGIFADDR_IN6) and the kernel API will return IPv6 addresses with 2nd 16bit-word filled in. The APIs are for manipulating kernel internal structure. Programs that use these APIs have to be prepared about differences in kernels anyway. When you specify scoped address to the command line, NEVER write the embedded form (such as ff02:1::1 or fe80:2::fedc). This is not supposed to work. Always use standard form, like ff02::1 or fe80::fedc, with command line option for specifying interface (like ping6 -I ne0 ff02::1). In general, if a command does not have command line option to specify outgoing interface, that command is not ready to accept scoped address. This may seem to be opposite from IPv6's premise to support "dentist office" situation. We believe that specifications need some improvements for this. Some of the userland tools support extended numeric IPv6 syntax, as documented in draft-ietf-ipngwg-scopedaddr-format-00.txt. You can specify outgoing link, by using name of the outgoing interface like "fe80::1%ne0". This way you will be able to specify link-local scoped address without much trouble. To use this extension in your program, you'll need to use &man.getaddrinfo.3;, and &man.getnameinfo.3; with NI_WITHSCOPEID. The implementation currently assumes 1-to-1 relationship between a link and an interface, which is stronger than what specs say. Plug and Play Most of the IPv6 stateless address autoconfiguration is implemented in the kernel. Neighbor Discovery functions are implemented in the kernel as a whole. Router Advertisement (RA) input for hosts is implemented in the kernel. Router Solicitation (RS) output for endhosts, RS input for routers, and RA output for routers are implemented in the userland. Assignment of link-local, and special addresses IPv6 link-local address is generated from IEEE802 address (ethernet MAC address). Each of interface is assigned an IPv6 link-local address automatically, when the interface becomes up (IFF_UP). Also, direct route for the link-local address is added to routing table. Here is an output of netstat command: - -Internet6: +Internet6: Destination Gateway Flags Netif Expire fe80:1::%ed0/64 link#1 UC ed0 -fe80:2::%ep0/64 link#2 UC ep0 - +fe80:2::%ep0/64 link#2 UC ep0 Interfaces that has no IEEE802 address (pseudo interfaces like tunnel interfaces, or ppp interfaces) will borrow IEEE802 address from other interfaces, such as ethernet interfaces, whenever possible. If there is no IEEE802 hardware attached, last-resort pseudorandom value, which is from MD5(hostname), will be used as source of link-local address. If it is not suitable for your usage, you will need to configure the link-local address manually. If an interface is not capable of handling IPv6 (such as lack of multicast support), link-local address will not be assigned to that interface. See section 2 for details. Each interface joins the solicited multicast address and the link-local all-nodes multicast addresses (e.g. fe80::1:ff01:6317 and ff02::1, respectively, on the link the interface is attached). In addition to a link-local address, the loopback address (::1) will be assigned to the loopback interface. Also, ::1/128 and ff01::/32 are automatically added to routing table, and loopback interface joins node-local multicast group ff01::1. Stateless address autoconfiguration on hosts In IPv6 specification, nodes are separated into two categories: routers and hosts. Routers forward packets addressed to others, hosts does not forward the packets. net.inet6.ip6.forwarding defines whether this node is router or host (router if it is 1, host if it is 0). When a host hears Router Advertisement from the router, a host may autoconfigure itself by stateless address autoconfiguration. This behavior can be controlled by net.inet6.ip6.accept_rtadv (host autoconfigures itself if it is set to 1). By autoconfiguration, network address prefix for the receiving interface (usually global address prefix) is added. Default route is also configured. Routers periodically generate Router Advertisement packets. To request an adjacent router to generate RA packet, a host can transmit Router Solicitation. To generate a RS packet at any time, use the rtsol command. &man.rtsold.8; daemon is also available. &man.rtsold.8; generates Router Solicitation whenever necessary, and it works great for nomadic usage (notebooks/laptops). If one wishes to ignore Router Advertisements, use sysctl to set net.inet6.ip6.accept_rtadv to 0. To generate Router Advertisement from a router, use the &man.rtadvd.8 daemon. Note that, IPv6 specification assumes the following items, and nonconforming cases are left unspecified: Only hosts will listen to router advertisements Hosts have single network interface (except loopback) Therefore, this is unwise to enable net.inet6.ip6.accept_rtadv on routers, or multi-interface host. A misconfigured node can behave strange (nonconforming configuration allowed for those who would like to do some experiments). To summarize the sysctl knob: - - accept_rtadv forwarding role of the node + accept_rtadv forwarding role of the node --- --- --- 0 0 host (to be manually configured) 0 1 router 1 0 autoconfigured host (spec assumes that host has single interface only, autoconfigured host with multiple interface is out-of-scope) 1 1 invalid, or experimental - (out-of-scope of spec) - + (out-of-scope of spec) RFC2462 has validation rule against incoming RA prefix information option, in 5.5.3 (e). This is to protect hosts from malicious (or misconfigured) routers that advertise very short prefix lifetime. There was an update from Jim Bound to ipngwg mailing list (look for "(ipng 6712)" in the archive) and it is implemented Jim's update. See 23.5.1.2 in the document for relationship between DAD and autoconfiguration. Generic tunnel interface GIF (Generic InterFace) is a pseudo interface for configured tunnel. Details are described in &man.gif.4;. Currently v6 in v6 v6 in v4 v4 in v6 v4 in v4 are available. Use &man.gifconfig.8; to assign physical (outer) source and destination address to gif interfaces. Configuration that uses same address family for inner and outer IP header (v4 in v4, or v6 in v6) is dangerous. It is very easy to configure interfaces and routing tables to perform infinite level of tunneling. Please be warned. gif can be configured to be ECN-friendly. See 23.5.4.5 for ECN-friendliness of tunnels, and &man.gif.4; for how to configure. If you would like to configure an IPv4-in-IPv6 tunnel with gif interface, read &man.gif.4; carefully. You will need to remove IPv6 link-local address automatically assigned to the gif interface. Source Address Selection Current source selection rule is scope oriented (there are some exceptions - see below). For a given destination, a source IPv6 address is selected by the following rule: If the source address is explicitly specified by the user (e.g. via the advanced API), the specified address is used. If there is an address assigned to the outgoing interface (which is usually determined by looking up the routing table) that has the same scope as the destination address, the address is used. This is the most typical case. If there is no address that satisfies the above condition, choose a global address assigned to one of the interfaces on the sending node. If there is no address that satisfies the above condition, and destination address is site local scope, choose a site local address assigned to one of the interfaces on the sending node. If there is no address that satisfies the above condition, choose the address associated with the routing table entry for the destination. This is the last resort, which may cause scope violation. For instance, ::1 is selected for ff01::1, fe80:1::200:f8ff:fe01:6317 for fe80:1::2a0:24ff:feab:839b (note that embedded interface index - described in 23.5.1.3 - helps us choose the right source address. Those embedded indices will not be on the wire). If the outgoing interface has multiple address for the scope, a source is selected longest match basis (rule 3). Suppose 3ffe:501:808:1:200:f8ff:fe01:6317 and 3ffe:2001:9:124:200:f8ff:fe01:6317 are given to the outgoing interface. 3ffe:501:808:1:200:f8ff:fe01:6317 is chosen as the source for the destination 3ffe:501:800::1. Note that the above rule is not documented in the IPv6 spec. It is considered "up to implementation" item. There are some cases where we do not use the above rule. One example is connected TCP session, and we use the address kept in tcb as the source. Another example is source address for Neighbor Advertisement. Under the spec (RFC2461 7.2.2) NA's source should be the target address of the corresponding NS's target. In this case we follow the spec rather than the above longest-match rule. For new connections (when rule 1 does not apply), deprecated addresses (addresses with preferred lifetime = 0) will not be chosen as source address if other choices are available. If no other choices are available, deprecated address will be used as a last resort. If there are multiple choice of deprecated addresses, the above scope rule will be used to choose from those deprecated addresses. If you would like to prohibit the use of deprecated address for some reason, configure net.inet6.ip6.use_deprecated to 0. The issue related to deprecated address is described in RFC2462 5.5.4 (NOTE: there is some debate underway in IETF ipngwg on how to use "deprecated" address). Jumbo Payload The Jumbo Payload hop-by-hop option is implemented and can be used to send IPv6 packets with payloads longer than 65,535 octets. But currently no physical interface whose MTU is more than 65,535 is supported, so such payloads can be seen only on the loopback interface (i.e. lo0). If you want to try jumbo payloads, you first have to reconfigure the kernel so that the MTU of the loopback interface is more than 65,535 bytes; add the following to the kernel configuration file: options "LARGE_LOMTU" #To test jumbo payload and recompile the new kernel. Then you can test jumbo payloads by the &man.ping6.8; command with -b and -s options. The -b option must be specified to enlarge the size of the socket buffer and the -s option specifies the length of the packet, which should be more than 65,535. For example, type as follows: &prompt.user; ping6 -b 70000 -s 68000 ::1 The IPv6 specification requires that the Jumbo Payload option must not be used in a packet that carries a fragment header. If this condition is broken, an ICMPv6 Parameter Problem message must be sent to the sender. specification is followed, but you cannot usually see an ICMPv6 error caused by this requirement. When an IPv6 packet is received, the frame length is checked and compared to the length specified in the payload length field of the IPv6 header or in the value of the Jumbo Payload option, if any. If the former is shorter than the latter, the packet is discarded and statistics are incremented. You can see the statistics as output of &man.netstat.8; command with `-s -p ip6' option: - - &prompt.user; netstat -s -p ip6 + &prompt.user; netstat -s -p ip6 ip6: (snip) - 1 with data size < data length - + 1 with data size < data length So, kernel does not send an ICMPv6 error unless the erroneous packet is an actual Jumbo Payload, that is, its packet size is more than 65,535 bytes. As described above, currently no physical interface with such a huge MTU is supported, so it rarely returns an ICMPv6 error. TCP/UDP over jumbogram is not supported at this moment. This is because we have no medium (other than loopback) to test this. Contact us if you need this. IPsec does not work on jumbograms. This is due to some specification twists in supporting AH with jumbograms (AH header size influences payload length, and this makes it real hard to authenticate inbound packet with jumbo payload option as well as AH). There are fundamental issues in *BSD support for jumbograms. We would like to address those, but we need more time to finalize these. To name a few: mbuf pkthdr.len field is typed as "int" in 4.4BSD, so it will not hold jumbogram with len > 2G on 32bit architecture CPUs. If we would like to support jumbogram properly, the field must be expanded to hold 4G + IPv6 header + link-layer header. Therefore, it must be expanded to at least int64_t (u_int32_t is NOT enough). We mistakingly use "int" to hold packet length in many places. We need to convert them into larger integral type. It needs a great care, as we may experience overflow during packet length computation. We mistakingly check for ip6_plen field of IPv6 header for packet payload length in various places. We should be checking mbuf pkthdr.len instead. ip6_input() will perform sanity check on jumbo payload option on input, and we can safely use mbuf pkthdr.len afterwards. TCP code needs a careful update in bunch of places, of course. Loop prevention in header processing IPv6 specification allows arbitrary number of extension headers to be placed onto packets. If we implement IPv6 packet processing code in the way BSD IPv4 code is implemented, kernel stack may overflow due to long function call chain. sys/netinet6 code is carefully designed to avoid kernel stack overflow. Because of this, sys/netinet6 code defines its own protocol switch structure, as "struct ip6protosw" (see netinet6/ip6protosw.h). There is no such update to IPv4 part (sys/netinet) for compatibility, but small change is added to its pr_input() prototype. So "struct ipprotosw" is also defined. Because of this, if you receive IPsec-over-IPv4 packet with massive number of IPsec headers, kernel stack may blow up. IPsec-over-IPv6 is okay. (Off-course, for those all IPsec headers to be processed, each such IPsec header must pass each IPsec check. So an anonymous attacker won't be able to do such an attack.) ICMPv6 After RFC2463 was published, IETF ipngwg has decided to disallow ICMPv6 error packet against ICMPv6 redirect, to prevent ICMPv6 storm on a network medium. This is already implemented into the kernel. Applications For userland programming, we support IPv6 socket API as specified in RFC2553, RFC2292 and upcoming internet drafts. TCP/UDP over IPv6 is available and quite stable. You can enjoy &man.telnet.1;, &man.ftp.1;, &man.rlogin.1;, &man.rsh.1;, &man.ssh.1, etc. These applications are protocol independent. That is, they automatically chooses IPv4 or IPv6 according to DNS. Kernel Internals While ip_forward() calls ip_output(), ip6_forward() directly calls if_output() since routers must not divide IPv6 packets into fragments. ICMPv6 should contain the original packet as long as possible up to 1280. UDP6/IP6 port unreach, for instance, should contain all extension headers and the *unchanged* UDP6 and IP6 headers. So, all IP6 functions except TCP never convert network byte order into host byte order, to save the original packet. tcp_input(), udp6_input() and icmp6_input() can't assume that IP6 header is preceding the transport headers due to extension headers. So, in6_cksum() was implemented to handle packets whose IP6 header and transport header is not continuous. TCP/IP6 nor UDP6/IP6 header structure don't exist for checksum calculation. To process IP6 header, extension headers and transport headers easily, network drivers are now required to store packets in one internal mbuf or one or more external mbufs. A typical old driver prepares two internal mbufs for 96 - 204 bytes data, however, now such packet data is stored in one external mbuf. netstat -s -p ip6 tells you whether or not your driver conforms such requirement. In the following example, "cce0" violates the requirement. (For more information, refer to Section 2.) - - Mbuf statistics: + Mbuf statistics: 317 one mbuf two or more mbuf:: lo0 = 8 cce0 = 10 3282 one ext mbuf 0 two or more ext mbuf Each input function calls IP6_EXTHDR_CHECK in the beginning to check if the region between IP6 and its header is continuous. IP6_EXTHDR_CHECK calls m_pullup() only if the mbuf has M_LOOP flag, that is, the packet comes from the loopback interface. m_pullup() is never called for packets coming from physical network interfaces. Both IP and IP6 reassemble functions never call m_pullup(). IPv4 mapped address and IPv6 wildcard socket RFC2553 describes IPv4 mapped address (3.7) and special behavior of IPv6 wildcard bind socket (3.8). The spec allows you to: Accept IPv4 connections by AF_INET6 wildcard bind socket. Transmit IPv4 packet over AF_INET6 socket by using special form of the address like ::ffff:10.1.1.1. but the spec itself is very complicated and does not specify how the socket layer should behave. Here we call the former one "listening side" and the latter one "initiating side", for reference purposes. You can perform wildcard bind on both of the address families, on the same port. The following table show the behavior of FreeBSD 4.x. - - listening side initiating side + listening side initiating side (AF_INET6 wildcard (connection to ::ffff:10.1.1.1) socket gets IPv4 conn.) --- --- FreeBSD 4.x configurable supported default: enabled The following sections will give you more details, and how you can configure the behavior. Comments on listening side: It looks that RFC2553 talks too little on wildcard bind issue, especially on the port space issue, failure mode and relationship between AF_INET/INET6 wildcard bind. There can be several separate interpretation for this RFC which conform to it but behaves differently. So, to implement portable application you should assume nothing about the behavior in the kernel. Using &man.getaddrinfo.3; is the safest way. Port number space and wildcard bind issues were discussed in detail on ipv6imp mailing list, in mid March 1999 and it looks that there's no concrete consensus (means, up to implementers). You may want to check the mailing list archives. If a server application would like to accept IPv4 and IPv6 connections, there will be two alternatives. One is using AF_INET and AF_INET6 socket (you'll need two sockets). Use &man.getaddrinfo.3; with AI_PASSIVE into ai_flags, and &man.socket.2; and &man.bind.2; to all the addresses returned. By opening multiple sockets, you can accept connections onto the socket with proper address family. IPv4 connections will be accepted by AF_INET socket, and IPv6 connections will be accepted by AF_INET6 socket. Another way is using one AF_INET6 wildcard bind socket. Use &man.getaddrinfo.3; with AI_PASSIVE into ai_flags and with AF_INET6 into ai_family, and set the 1st argument hostname to NULL. And &man.socket.2; and &man.bind.2; to the address returned. (should be IPv6 unspecified addr). You can accept either of IPv4 and IPv6 packet via this one socket. To support only IPv6 traffic on AF_INET6 wildcard binded socket portably, always check the peer address when a connection is made toward AF_INET6 listening socket. If the address is IPv4 mapped address, you may want to reject the connection. You can check the condition by using IN6_IS_ADDR_V4MAPPED() macro. To resolve this issue more easily, there is system dependent &man.setsockopt.2; option, IPV6_BINDV6ONLY, used like below. - - int on; + int on; setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY, (char *)&on, sizeof (on)) < 0)); When this call succeed, then this socket only receive IPv6 packets. Comments on initiating side: Advise to application implementers: to implement a portable IPv6 application (which works on multiple IPv6 kernels), we believe that the following is the key to the success: NEVER hardcode AF_INET nor AF_INET6. Use &man.getaddrinfo.3; and &man.getnameinfo.3; throughout the system. Never use gethostby*(), getaddrby*(), inet_*() or getipnodeby*(). (To update existing applications to be IPv6 aware easily, sometime getipnodeby*() will be useful. But if possible, try to rewrite the code to use &man.getaddrinfo.3; and &man.getnameinfo.3;.) If you would like to connect to destination, use &man.getaddrinfo.3; and try all the destination returned, like &man.telnet.1; does. Some of the IPv6 stack is shipped with buggy &man.getaddrinfo.3;. Ship a minimal working version with your application and use that as last resort. If you would like to use AF_INET6 socket for both IPv4 and IPv6 outgoing connection, you will need to use &man.getipnodebyname.3;. When you would like to update your existing application to be IPv6 aware with minimal effort, this approach might be chosen. But please note that it is a temporal solution, because &man.getipnodebyname.3; itself is not recommended as it does not handle scoped IPv6 addresses at all. For IPv6 name resolution, &man.getaddrinfo.3; is the preferred API. So you should rewrite your application to use &man.getaddrinfo.3;, when you get the time to do it. When writing applications that make outgoing connections, story goes much simpler if you treat AF_INET and AF_INET6 as totally separate address family. {set,get}sockopt issue goes simpler, DNS issue will be made simpler. We do not recommend you to rely upon IPv4 mapped address. unified tcp and inpcb code FreeBSD 4.x uses shared tcp code between IPv4 and IPv6 (from sys/netinet/tcp*) and separate udp4/6 code. It uses unified inpcb structure. The platform can be configured to support IPv4 mapped address. Kernel configuration is summarized as follows: By default, AF_INET6 socket will grab IPv4 connections in certain condition, and can initiate connection to IPv4 destination embedded in IPv4 mapped IPv6 address. You can disable it on entire system with sysctl like below. sysctl -w net.inet6.ip6.mapped_addr=0 listening side Each socket can be configured to support special AF_INET6 wildcard bind (enabled by default). You can disable it on each socket basis with &man.setsockopt.2; like below. - - int on; + int on; setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY, (char *)&on, sizeof (on)) < 0)); Wildcard AF_INET6 socket grabs IPv4 connection if and only if the following conditions are satisfied: there's no AF_INET socket that matches the IPv4 connection the AF_INET6 socket is configured to accept IPv4 traffic, i.e. getsockopt(IPV6_BINDV6ONLY) returns 0. There's no problem with open/close ordering. initiating side FreeBSD 4.x supports outgoing connection to IPv4 mapped address (::ffff:10.1.1.1), if the node is configured to support IPv4 mapped address. sockaddr_storage When RFC2553 was about to be finalized, there was discussion on how struct sockaddr_storage members are named. One proposal is to prepend "__" to the members (like "__ss_len") as they should not be touched. The other proposal was that don't prepend it (like "ss_len") as we need to touch those members directly. There was no clear consensus on it. As a result, RFC2553 defines struct sockaddr_storage as follows: - - struct sockaddr_storage { + struct sockaddr_storage { u_char __ss_len; /* address length */ u_char __ss_family; /* address family */ /* and bunch of padding */ }; On the contrary, XNET draft defines as follows: - - struct sockaddr_storage { + struct sockaddr_storage { u_char ss_len; /* address length */ u_char ss_family; /* address family */ /* and bunch of padding */ }; In December 1999, it was agreed that RFC2553bis should pick the latter (XNET) definition. Current implementation conforms to XNET definition, based on RFC2553bis discussion. If you look at multiple IPv6 implementations, you will be able to see both definitions. As an userland programmer, the most portable way of dealing with it is to: ensure ss_family and/or ss_len are available on the platform, by using GNU autoconf, have -Dss_family=__ss_family to unify all occurrences (including header file) into __ss_family, or never touch __ss_family. cast to sockaddr * and use sa_family like: - - struct sockaddr_storage ss; + struct sockaddr_storage ss; family = ((struct sockaddr *)&ss)->sa_family Network Drivers Now following two items are required to be supported by standard drivers: mbuf clustering requirement. In this stable release, we changed MINCLSIZE into MHLEN+1 for all the operating systems in order to make all the drivers behave as we expect. multicast. If &man.ifmcstat.8; yields no multicast group for a interface, that interface has to be patched. If any of the driver don't support the requirements, then the driver can't be used for IPv6 and/or IPsec communication. If you find any problem with your card using IPv6/IPsec, then, please report it to freebsd-bugs@FreeBSD.org. (NOTE: In the past we required all PCMCIA drivers to have a call to in6_ifattach(). We have no such requirement any more) Translator We categorize IPv4/IPv6 translator into 4 types: Translator A --- It is used in the early stage of transition to make it possible to establish a connection from an IPv6 host in an IPv6 island to an IPv4 host in the IPv4 ocean. Translator B --- It is used in the early stage of transition to make it possible to establish a connection from an IPv4 host in the IPv4 ocean to an IPv6 host in an IPv6 island. Translator C --- It is used in the late stage of transition to make it possible to establish a connection from an IPv4 host in an IPv4 island to an IPv6 host in the IPv6 ocean. Translator D --- It is used in the late stage of transition to make it possible to establish a connection from an IPv6 host in the IPv6 ocean to an IPv4 host in an IPv4 island. TCP relay translator for category A is supported. This is called "FAITH". We also provide IP header translator for category A. (The latter is not yet put into FreeBSD 4.x yet.) FAITH TCP relay translator FAITH system uses TCP relay daemon called &man.faithd.8; helped by the kernel. FAITH will reserve an IPv6 address prefix, and relay TCP connection toward that prefix to IPv4 destination. For example, if the reserved IPv6 prefix is 3ffe:0501:0200:ffff::, and the IPv6 destination for TCP connection is 3ffe:0501:0200:ffff::163.221.202.12, the connection will be relayed toward IPv4 destination 163.221.202.12. - - destination IPv4 node (163.221.202.12) + destination IPv4 node (163.221.202.12) ^ | IPv4 tcp toward 163.221.202.12 FAITH-relay dual stack node ^ | IPv6 TCP toward 3ffe:0501:0200:ffff::163.221.202.12 source IPv6 node &man.faithd.8; must be invoked on FAITH-relay dual stack node. For more details, consult src/usr.sbin/faithd/README IPsec IPsec is mainly organized by three components. Policy Management Key Management AH and ESP handling Policy Management The kernel implements experimental policy management code. There are two way to manage security policy. One is to configure per-socket policy using &man.setsockopt.2;. In this cases, policy configuration is described in &man.ipsec.set.policy.3;. The other is to configure kernel packet filter-based policy using PF_KEY interface, via &man.setkey.8;. The policy entry is not re-ordered with its indexes, so the order of entry when you add is very significant. Key Management The key management code implemented in this kit (sys/netkey) is a home-brew PFKEY v2 implementation. This conforms to RFC2367. The home-brew IKE daemon, "racoon" is included in the kit (kame/kame/racoon). Basically you'll need to run racoon as daemon, then setup a policy to require keys (like ping -P 'out ipsec esp/transport//use'). The kernel will contact racoon daemon as necessary to exchange keys. AH and ESP handling IPsec module is implemented as "hooks" to the standard IPv4/IPv6 processing. When sending a packet, ip{,6}_output() checks if ESP/AH processing is required by checking if a matching SPD (Security Policy Database) is found. If ESP/AH is needed, {esp,ah}{4,6}_output() will be called and mbuf will be updated accordingly. When a packet is received, {esp,ah}4_input() will be called based on protocol number, i.e. (*inetsw[proto])(). {esp,ah}4_input() will decrypt/check authenticity of the packet, and strips off daisy-chained header and padding for ESP/AH. It is safe to strip off the ESP/AH header on packet reception, since we will never use the received packet in "as is" form. By using ESP/AH, TCP4/6 effective data segment size will be affected by extra daisy-chained headers inserted by ESP/AH. Our code takes care of the case. Basic crypto functions can be found in directory "sys/crypto". ESP/AH transform are listed in {esp,ah}_core.c with wrapper functions. If you wish to add some algorithm, add wrapper function in {esp,ah}_core.c, and add your crypto algorithm code into sys/crypto. Tunnel mode is partially supported in this release, with the following restrictions: IPsec tunnel is not combined with GIF generic tunneling interface. It needs a great care because we may create an infinite loop between ip_output() and tunnelifp->if_output(). Opinion varies if it is better to unify them, or not. MTU and Don't Fragment bit (IPv4) considerations need more checking, but basically works fine. Authentication model for AH tunnel must be revisited. We'll need to improve the policy management engine, eventually. Conformance to RFCs and IDs The IPsec code in the kernel conforms (or, tries to conform) to the following standards: "old IPsec" specification documented in rfc182[5-9].txt "new IPsec" specification documented in rfc240[1-6].txt, rfc241[01].txt, rfc2451.txt and draft-mcdonald-simple-ipsec-api-01.txt (draft expired, but you can take from ftp://ftp.kame.net/pub/internet-drafts/). (NOTE: IKE specifications, rfc241[7-9].txt are implemented in userland, as "racoon" IKE daemon) Currently supported algorithms are: old IPsec AH null crypto checksum (no document, just for debugging) keyed MD5 with 128bit crypto checksum (rfc1828.txt) keyed SHA1 with 128bit crypto checksum (no document) HMAC MD5 with 128bit crypto checksum (rfc2085.txt) HMAC SHA1 with 128bit crypto checksum (no document) old IPsec ESP null encryption (no document, similar to rfc2410.txt) DES-CBC mode (rfc1829.txt) new IPsec AH null crypto checksum (no document, just for debugging) keyed MD5 with 96bit crypto checksum (no document) keyed SHA1 with 96bit crypto checksum (no document) HMAC MD5 with 96bit crypto checksum (rfc2403.txt) HMAC SHA1 with 96bit crypto checksum (rfc2404.txt) new IPsec ESP null encryption (rfc2410.txt) DES-CBC with derived IV (draft-ietf-ipsec-ciph-des-derived-01.txt, draft expired) DES-CBC with explicit IV (rfc2405.txt) 3DES-CBC with explicit IV (rfc2451.txt) BLOWFISH CBC (rfc2451.txt) CAST128 CBC (rfc2451.txt) RC5 CBC (rfc2451.txt) each of the above can be combined with: ESP authentication with HMAC-MD5(96bit) ESP authentication with HMAC-SHA1(96bit) The following algorithms are NOT supported: old IPsec AH HMAC MD5 with 128bit crypto checksum + 64bit replay prevention (rfc2085.txt) keyed SHA1 with 160bit crypto checksum + 32bit padding (rfc1852.txt) IPsec (in kernel) and IKE (in userland as "racoon") has been tested at several interoperability test events, and it is known to interoperate with many other implementations well. Also, current IPsec implementation as quite wide coverage for IPsec crypto algorithms documented in RFC (we cover algorithms without intellectual property issues only). ECN consideration on IPsec tunnels ECN-friendly IPsec tunnel is supported as described in draft-ipsec-ecn-00.txt. Normal IPsec tunnel is described in RFC2401. On encapsulation, IPv4 TOS field (or, IPv6 traffic class field) will be copied from inner IP header to outer IP header. On decapsulation outer IP header will be simply dropped. The decapsulation rule is not compatible with ECN, since ECN bit on the outer IP TOS/traffic class field will be lost. To make IPsec tunnel ECN-friendly, we should modify encapsulation and decapsulation procedure. This is described in http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt, chapter 3. IPsec tunnel implementation can give you three behaviors, by setting net.inet.ipsec.ecn (or net.inet6.ipsec6.ecn) to some value: RFC2401: no consideration for ECN (sysctl value -1) ECN forbidden (sysctl value 0) ECN allowed (sysctl value 1) Note that the behavior is configurable in per-node manner, not per-SA manner (draft-ipsec-ecn-00 wants per-SA configuration, but it looks too much for me). The behavior is summarized as follows (see source code for more detail): - encapsulate decapsulate --- --- RFC2401 copy all TOS bits drop TOS bits on outer from inner to outer. (use inner TOS bits as is) ECN forbidden copy TOS bits except for ECN drop TOS bits on outer (masked with 0xfc) from inner (use inner TOS bits as is) to outer. set ECN bits to 0. ECN allowed copy TOS bits except for ECN use inner TOS bits with some CE (masked with 0xfe) from change. if outer ECN CE bit inner to outer. is 1, enable ECN CE bit on set ECN CE bit to 0. the inner. General strategy for configuration is as follows: if both IPsec tunnel endpoint are capable of ECN-friendly behavior, you'd better configure both end to "ECN allowed" (sysctl value 1). if the other end is very strict about TOS bit, use "RFC2401" (sysctl value -1). in other cases, use "ECN forbidden" (sysctl value 0). The default behavior is "ECN forbidden" (sysctl value 0). For more information, please refer to: http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt, RFC2481 (Explicit Congestion Notification), src/sys/netinet6/{ah,esp}_input.c (Thanks goes to Kenjiro Cho kjc@csl.sony.co.jp for detailed analysis) Interoperability Here are (some of) platforms that KAME code have tested IPsec/IKE interoperability in the past. Note that both ends may have modified their implementation, so use the following list just for reference purposes. Altiga, Ashley-laurent (vpcom.com), Data Fellows (F-Secure), Ericsson ACC, FreeS/WAN, HITACHI, IBM AIX, IIJ, Intel, Microsoft WinNT, NIST (linux IPsec + plutoplus), Netscreen, OpenBSD, RedCreek, Routerware, SSH, Secure Computing, Soliton, Toshiba, VPNet, Yamaha RT100i diff --git a/en_US.ISO8859-1/books/developers-handbook/kerneldebug/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/kerneldebug/chapter.sgml index b08a119cb9..d56fa5aaba 100644 --- a/en_US.ISO8859-1/books/developers-handbook/kerneldebug/chapter.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/kerneldebug/chapter.sgml @@ -1,639 +1,638 @@ Kernel Debugging Contributed by &a.paul; and &a.joerg; Debugging a Kernel Crash Dump with <command>gdb</command> Here are some instructions for getting kernel debugging working on a crash dump. They assume that you have enough swap space for a crash dump. If you have multiple swap partitions and the first one is too small to hold the dump, you can configure your kernel to use an alternate dump device (in the config kernel line), or you can specify an alternate using the &man.dumpon.8; command. The best way to use &man.dumpon.8; is to set the dumpdev variable in /etc/rc.conf. Typically you want to specify one of the swap devices specified in /etc/fstab. Dumps to non-swap devices, tapes for example, are currently not supported. Config your kernel using config . See Kernel Configuration for details on configuring the FreeBSD kernel. Use the &man.dumpon.8; command to tell the kernel where to dump to (note that this will have to be done after configuring the partition in question as swap space via &man.swapon.8;). This is normally arranged via /etc/rc.conf and /etc/rc. Alternatively, you can hard-code the dump device via the dump clause in the config line of your kernel config file. This is deprecated and should be used only if you want a crash dump from a kernel that crashes during booting. In the following, the term gdb refers to the debugger gdb run in kernel debug mode. This can be accomplished by starting the gdb with the option . In kernel debug mode, gdb changes its prompt to (kgdb). If you are using FreeBSD 3 or earlier, you should make a stripped copy of the debug kernel, rather than installing the large debug kernel itself: &prompt.root; cp kernel kernel.debug &prompt.root; strip -g kernel This stage isn't necessary, but it is recommended. (In FreeBSD 4 and later releases this step is performed automatically at the end of the kernel make process.) When the kernel has been stripped, either automatically or by using the commands above, you may install it as usual by typing make install. Note that older releases of FreeBSD (up to but not including 3.1) used a.out kernels by default, which must have their symbol tables permanently resident in physical memory. With the larger symbol table in an unstripped debug kernel, this is wasteful. Recent FreeBSD releases use ELF kernels where this is no longer a problem. If you are testing a new kernel, for example by typing the new kernel's name at the boot prompt, but need to boot a different one in order to get your system up and running again, boot it only into single user state using the flag at the boot prompt, and then perform the following steps: &prompt.root; fsck -p &prompt.root; mount -a -t ufs # so your file system for /var/crash is writable &prompt.root; savecore -N /kernel.panicked /var/crash &prompt.root; exit # ...to multi-user This instructs &man.savecore.8; to use another kernel for symbol name extraction. It would otherwise default to the currently running kernel and most likely not do anything at all since the crash dump and the kernel symbols differ. Now, after a crash dump, go to /sys/compile/WHATEVER and run gdb . From gdb do: symbol-file kernel.debug exec-file /var/crash/kernel.0 core-file /var/crash/vmcore.0 and voila, you can debug the crash dump using the kernel sources just like you can for any other program. Here is a script log of a gdb session illustrating the procedure. Long lines have been folded to improve readability, and the lines are numbered for reference. Despite this, it is a real-world error trace taken during the development of the pcvt console driver. 1:Script started on Fri Dec 30 23:15:22 1994 2:&prompt.root; cd /sys/compile/URIAH 3:&prompt.root; gdb -k kernel /var/crash/vmcore.1 4:Reading symbol data from /usr/src/sys/compile/URIAH/kernel ...done. 5:IdlePTD 1f3000 6:panic: because you said to! 7:current pcb at 1e3f70 8:Reading in symbols for ../../i386/i386/machdep.c...done. 9:(kgdb) where 10:#0 boot (arghowto=256) (../../i386/i386/machdep.c line 767) 11:#1 0xf0115159 in panic () 12:#2 0xf01955bd in diediedie () (../../i386/i386/machdep.c line 698) 13:#3 0xf010185e in db_fncall () 14:#4 0xf0101586 in db_command (-266509132, -266509516, -267381073) 15:#5 0xf0101711 in db_command_loop () 16:#6 0xf01040a0 in db_trap () 17:#7 0xf0192976 in kdb_trap (12, 0, -272630436, -266743723) 18:#8 0xf019d2eb in trap_fatal (...) 19:#9 0xf019ce60 in trap_pfault (...) 20:#10 0xf019cb2f in trap (...) 21:#11 0xf01932a1 in exception:calltrap () 22:#12 0xf0191503 in cnopen (...) 23:#13 0xf0132c34 in spec_open () 24:#14 0xf012d014 in vn_open () 25:#15 0xf012a183 in open () 26:#16 0xf019d4eb in syscall (...) 27:(kgdb) up 10 28:Reading in symbols for ../../i386/i386/trap.c...done. 29:#10 0xf019cb2f in trap (frame={tf_es = -260440048, tf_ds = 16, tf_\ 30:edi = 3072, tf_esi = -266445372, tf_ebp = -272630356, tf_isp = -27\ 31:2630396, tf_ebx = -266427884, tf_edx = 12, tf_ecx = -266427884, tf\ 32:_eax = 64772224, tf_trapno = 12, tf_err = -272695296, tf_eip = -26\ 33:6672343, tf_cs = -266469368, tf_eflags = 66066, tf_esp = 3072, tf_\ 34:ss = -266427884}) (../../i386/i386/trap.c line 283) 35:283 (void) trap_pfault(&frame, FALSE); 36:(kgdb) frame frame->tf_ebp frame->tf_eip 37:Reading in symbols for ../../i386/isa/pcvt/pcvt_drv.c...done. 38:#0 0xf01ae729 in pcopen (dev=3072, flag=3, mode=8192, p=(struct p\ 39:roc *) 0xf07c0c00) (../../i386/isa/pcvt/pcvt_drv.c line 403) 40:403 return ((*linesw[tp->t_line].l_open)(dev, tp)); 41:(kgdb) list 42:398 43:399 tp->t_state |= TS_CARR_ON; 44:400 tp->t_cflag |= CLOCAL; /* cannot be a modem (:-) */ 45:401 46:402 #if PCVT_NETBSD || (PCVT_FREEBSD >= 200) 47:403 return ((*linesw[tp->t_line].l_open)(dev, tp)); 48:404 #else 49:405 return ((*linesw[tp->t_line].l_open)(dev, tp, flag)); 50:406 #endif /* PCVT_NETBSD || (PCVT_FREEBSD >= 200) */ 51:407 } 52:(kgdb) print tp 53:Reading in symbols for ../../i386/i386/cons.c...done. 54:$1 = (struct tty *) 0x1bae 55:(kgdb) print tp->t_line 56:$2 = 1767990816 57:(kgdb) up 58:#1 0xf0191503 in cnopen (dev=0x00000000, flag=3, mode=8192, p=(st\ 59:ruct proc *) 0xf07c0c00) (../../i386/i386/cons.c line 126) 60: return ((*cdevsw[major(dev)].d_open)(dev, flag, mode, p)); 61:(kgdb) up 62:#2 0xf0132c34 in spec_open () 63:(kgdb) up 64:#3 0xf012d014 in vn_open () 65:(kgdb) up 66:#4 0xf012a183 in open () 67:(kgdb) up 68:#5 0xf019d4eb in syscall (frame={tf_es = 39, tf_ds = 39, tf_edi =\ 69: 2158592, tf_esi = 0, tf_ebp = -272638436, tf_isp = -272629788, tf\ 70:_ebx = 7086, tf_edx = 1, tf_ecx = 0, tf_eax = 5, tf_trapno = 582, \ 71:tf_err = 582, tf_eip = 75749, tf_cs = 31, tf_eflags = 582, tf_esp \ 72:= -272638456, tf_ss = 39}) (../../i386/i386/trap.c line 673) 73:673 error = (*callp->sy_call)(p, args, rval); 74:(kgdb) up 75:Initial frame selected; you cannot go up. 76:(kgdb) quit 77:&prompt.root; exit 78:exit 79: 80:Script done on Fri Dec 30 23:18:04 1994 Comments to the above script: line 6: This is a dump taken from within DDB (see below), hence the panic comment because you said to!, and a rather long stack trace; the initial reason for going into DDB has been a page fault trap though. line 20: This is the location of function trap() in the stack trace. line 36: Force usage of a new stack frame; this is no longer necessary now. The stack frames are supposed to point to the right locations now, even in case of a trap. From looking at the code in source line 403, there is a high probability that either the pointer access for tp was messed up, or the array access was out of bounds. line 52: The pointer looks suspicious, but happens to be a valid address. line 56: However, it obviously points to garbage, so we have found our error! (For those unfamiliar with that particular piece of code: tp->t_line refers to the line discipline of the console device here, which must be a rather small integer number.) Debugging a Crash Dump with DDD Examining a kernel crash dump with a graphical debugger like ddd is also possible. Add the option to the ddd command line you would use normally. For example; &prompt.root; ddd -k /var/crash/kernel.0 /var/crash/vmcore.0 You should then be able to go about looking at the crash dump using ddd's graphical interface. Post-Mortem Analysis of a Dump What do you do if a kernel dumped core but you did not expect it, and it is therefore not compiled using config -g? Not everything is lost here. Do not panic! Of course, you still need to enable crash dumps. See above on the options you have to specify in order to do this. Go to your kernel config directory (/usr/src/sys/arch/conf) and edit your configuration file. Uncomment (or add, if it does not exist) the following line makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Rebuild the kernel. Due to the time stamp change on the Makefile, there will be some other object files rebuild, for example trap.o. With a bit of luck, the added option will not change anything for the generated code, so you will finally get a new kernel with similar code to the faulting one but some debugging symbols. You should at least verify the old and new sizes with the &man.size.1; command. If there is a mismatch, you probably need to give up here. Go and examine the dump as described above. The debugging symbols might be incomplete for some places, as can be seen in the stack trace in the example above where some functions are displayed without line numbers and argument lists. If you need more debugging symbols, remove the appropriate object files and repeat the gdb session until you know enough. All this is not guaranteed to work, but it will do it fine in most cases. On-Line Kernel Debugging Using DDB While gdb as an off-line debugger provides a very high level of user interface, there are some things it cannot do. The most important ones being breakpointing and single-stepping kernel code. If you need to do low-level debugging on your kernel, there is an on-line debugger available called DDB. It allows to setting breakpoints, single-stepping kernel functions, examining and changing kernel variables, etc. However, it cannot access kernel source files, and only has access to the global and static symbols, not to the full debug information like gdb. To configure your kernel to include DDB, add the option line options DDB to your config file, and rebuild. (See Kernel Configuration for details on configuring the FreeBSD kernel. If you have an older version of the boot blocks, your debugger symbols might not be loaded at all. Update the boot blocks; the recent ones load the DDB symbols automagically.) Once your DDB kernel is running, there are several ways to enter DDB. The first, and earliest way is to type the boot flag right at the boot prompt. The kernel will start up in debug mode and enter DDB prior to any device probing. Hence you can even debug the device probe/attach functions. The second scenario is a hot-key on the keyboard, usually Ctrl-Alt-ESC. For syscons, this can be remapped; some of the distributed maps do this, so watch out. There is an option available for serial consoles that allows the use of a serial line BREAK on the console line to enter DDB (options BREAK_TO_DEBUGGER in the kernel config file). It is not the default since there are a lot of crappy serial adapters around that gratuitously generate a BREAK condition, for example when pulling the cable. The third way is that any panic condition will branch to DDB if the kernel is configured to use it. For this reason, it is not wise to configure a kernel with DDB for a machine running unattended. The DDB commands roughly resemble some gdb commands. The first thing you probably need to do is to set a breakpoint: b function-name b address Numbers are taken hexadecimal by default, but to make them distinct from symbol names; hexadecimal numbers starting with the letters a-f need to be preceded with 0x (this is optional for other numbers). Simple expressions are allowed, for example: function-name + 0x103. To continue the operation of an interrupted kernel, simply type: c To get a stack trace, use: trace Note that when entering DDB via a hot-key, the kernel is currently servicing an interrupt, so the stack trace might be not of much use for you. If you want to remove a breakpoint, use del del address-expression The first form will be accepted immediately after a breakpoint hit, and deletes the current breakpoint. The second form can remove any breakpoint, but you need to specify the exact address; this can be obtained from: show b To single-step the kernel, try: s This will step into functions, but you can make DDB trace them until the matching return statement is reached by: n This is different from gdb's next statement; it is like gdb's finish. To examine data from memory, use (for example): x/wx 0xf0133fe0,40 x/hd db_symtab_space x/bc termbuf,10 x/s stringbuf for word/halfword/byte access, and hexadecimal/decimal/character/ string display. The number after the comma is the object count. To display the next 0x10 items, simply use: x ,10 Similarly, use x/ia foofunc,10 to disassemble the first 0x10 instructions of foofunc, and display them along with their offset from the beginning of foofunc. To modify memory, use the write command: w/b termbuf 0xa 0xb 0 w/w 0xf0010030 0 0 The command modifier (b/h/w) specifies the size of the data to be written, the first following expression is the address to write to and the remainder is interpreted as data to write to successive memory locations. If you need to know the current registers, use: show reg Alternatively, you can display a single register value by e.g. p $eax and modify it by: set $eax new-value Should you need to call some kernel functions from DDB, simply say: call func(arg1, arg2, ...) The return value will be printed. For a &man.ps.1; style summary of all running processes, use: ps Now you have examined why your kernel failed, and you wish to reboot. Remember that, depending on the severity of previous malfunctioning, not all parts of the kernel might still be working as expected. Perform one of the following actions to shut down and reboot your system: panic This will cause your kernel to dump core and reboot, so you can later analyze the core on a higher level with gdb. This command usually must be followed by another continue statement. call boot(0) Which might be a good way to cleanly shut down the running system, sync() all disks, and finally reboot. As long as the disk and file system interfaces of the kernel are not damaged, this might be a good way for an almost clean shutdown. call cpu_reset() is the final way out of disaster and almost the same as hitting the Big Red Button. If you need a short command summary, simply type: help However, it is highly recommended to have a printed copy of the &man.ddb.4; manual page ready for a debugging session. Remember that it is hard to read the on-line manual while single-stepping the kernel. On-Line Kernel Debugging Using Remote GDB This feature has been supported since FreeBSD 2.2, and it is actually a very neat one. GDB has already supported remote debugging for a long time. This is done using a very simple protocol along a serial line. Unlike the other methods described above, you will need two machines for doing this. One is the host providing the debugging environment, including all the sources, and a copy of the kernel binary with all the symbols in it, and the other one is the target machine that simply runs a similar copy of the very same kernel (but stripped of the debugging information). You should configure the kernel in question with config -g, include into the configuration, and compile it as usual. This gives a large blurb of a binary, due to the debugging information. Copy this kernel to the target machine, strip the debugging symbols off with strip -x, and boot it using the boot option. Connect the serial line of the target machine that has "flags 080" set on its sio device to any serial line of the debugging host. Now, on the debugging machine, go to the compile directory of the target kernel, and start gdb: &prompt.user; gdb -k kernel GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.16 (i386-unknown-freebsd), Copyright 1996 Free Software Foundation, Inc... (kgdb) Initialize the remote debugging session (assuming the first serial port is being used) by: (kgdb) target remote /dev/cuaa0 Now, on the target host (the one that entered DDB right before even starting the device probe), type: Debugger("Boot flags requested debugger") Stopped at Debugger+0x35: movb $0, edata+0x51bc db> gdb DDB will respond with: Next trap will enter GDB remote protocol mode Every time you type gdb, the mode will be toggled between remote GDB and local DDB. In order to force a next trap immediately, simply type s (step). Your hosting GDB will now gain control over the target kernel: Remote debugging using /dev/cuaa0 Debugger (msg=0xf01b0383 "Boot flags requested debugger") at ../../i386/i386/db_interface.c:257 (kgdb) You can use this session almost as any other GDB session, including full access to the source, running it in gud-mode inside an Emacs window (which gives you an automatic source code display in another Emacs window) etc. Debugging Loadable Modules Using GDB When debugging a panic that occurred within a module, or using remote GDB against a machine that uses dynamic modules, you need to tell GDB how to obtain symbol information for those modules. First, you need to build the module(s) with debugging information: &prompt.root; cd /sys/modules/linux &prompt.root; make clean; make COPTS=-g If you are using remote GDB, you can run kldstat on the target machine to find out where the module was loaded: &prompt.root; kldstat Id Refs Address Size Name 1 4 0xc0100000 1c1678 kernel 2 1 0xc0a9e000 6000 linprocfs.ko 3 1 0xc0ad7000 2000 warp_saver.ko - 4 1 0xc0adc000 11000 linux.ko - + 4 1 0xc0adc000 11000 linux.ko If you are debugging a crash dump, you'll need to walk the linker_files list, starting at linker_files->tqh_first and following the link.tqe_next pointers until you find the entry with the filename you are looking for. The address member of that entry is the load address of the module. Next, you need to find out the offset of the text section within the module: &prompt.root; objdump --section-headers /sys/modules/linux/linux.ko | grep text 3 .rel.text 000016e0 000038e0 000038e0 000038e0 2**2 10 .text 00007f34 000062d0 000062d0 000062d0 2**2 The one you want is the .text section, section 10 in the above example. The fourth hexadecimal field (sixth field overall) is the offset of the text section within the file. Add this offset to the load address of the module to obtain the relocation address for the module's code. In our example, we get 0xc0adc000 + 0x62d0 = 0xc0ae22d0. Use the add-symbol-file command in GDB to tell the debugger about the module: (kgdb) add-symbol-file /sys/modules/linux/linux.ko 0xc0ae22d0 add symbol table from file "/sys/modules/linux/linux.ko" at text_addr = 0xc0ae22d0? (y or n) y Reading symbols from /sys/modules/linux/linux.ko...done. (kgdb) You should now have access to all the symbols in the module. Debugging a Console Driver Since you need a console driver to run DDB on, things are more complicated if the console driver itself is failing. You might remember the use of a serial console (either with modified boot blocks, or by specifying at the Boot: prompt), and hook up a standard terminal onto your first serial port. DDB works on any configured console driver, of course also on a serial console. diff --git a/en_US.ISO8859-1/books/developers-handbook/sockets/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/sockets/chapter.sgml index f4c00f849b..acdd95e88e 100644 --- a/en_US.ISO8859-1/books/developers-handbook/sockets/chapter.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/sockets/chapter.sgml @@ -1,1789 +1,1773 @@ Sockets This chapter was written by &a.stanislav; Synopsis BSD sockets take interprocess communications to a new level. It is no longer necessary for the communicating processes to run on the same machine. They still can, but they do not have to. Not only do these processes not have to run on the same machine, they do not have to run under the same operating system. Thanks to BSD sockets, your FreeBSD software can smoothly cooperate with a program running on a Macintosh, another one running on a Sun workstation, yet another one running under Windows 2000, all connected with an Ethernet-based local area network. But your software can equally well cooperate with processes running in another building, or on another continent, inside a submarine, or a space shuttle. It can also cooperate with processes that are not part of a computer (at least not in the strict sense of the word), but of such devices as printers, digital cameras, medical equipment. Just about anything capable of digital communications. Networking and Diversity We have already hinted on the diversity of networking. Many different systems have to talk to each other. And they have to speak the same language. They also have to understand the same language the same way. People often think that body language is universal. But it is not. Back in my early teens, my father took me to Bulgaria. We were sitting at a table in a park in Sofia, when a vendor approached us trying to sell us some roasted almonds. I had not learned much Bulgarian by then, so, instead of saying no, I shook my head from side to side, the universal body language for no. The vendor quickly started serving us some almonds. I then remembered I had been told that in Bulgaria shaking your head sideways meant yes. Quickly, I started nodding my head up and down. The vendor noticed, took his almonds, and walked away. To an uninformed observer, I did not change the body language: I continued using the language of shaking and nodding my head. What changed was the meaning of the body language. At first, the vendor and I interpreted the same language as having completely different meaning. I had to adjust my own interpretation of that language so the vendor would understand. It is the same with computers: The same symbols may have different, even outright opposite meaning. Therefore, for two computers to understand each other, they must not only agree on the same language, but on the same interpretation of the language. Protocols While various programming languages tend to have complex syntax and use a number of multi-letter reserved words (which makes them easy for the human programmer to understand), the languages of data communications tend to be very terse. Instead of multi-byte words, they often use individual bits. There is a very convincing reason for it: While data travels inside your computer at speeds approaching the speed of light, it often travels considerably slower between two computers. Because the languages used in data communications are so terse, we usually refer to them as protocols rather than languages. As data travels from one computer to another, it always uses more than one protocol. These protocols are layered. The data can be compared to the inside of an onion: You have to peel off several layers of skin to get to the data. This is best illustrated with a picture: +----------------+ | Ethernet | |+--------------+| || IP || ||+------------+|| ||| TCP ||| |||+----------+||| |||| HTTP |||| ||||+--------+|||| ||||| PNG ||||| |||||+------+||||| |||||| Data |||||| |||||+------+||||| ||||+--------+|||| |||+----------+||| ||+------------+|| |+--------------+| +----------------+ Protocol Layers In this example, we are trying to get an image from a web page we are connected to via an Ethernet. The image consists of raw data, which is simply a sequence of RGB values that our software can process, i.e., convert into an image and display on our monitor. Alas, our software has no way of knowing how the raw data is organized: Is it a sequence of RGB values, or a sequence of grayscale intensities, or perhaps of CMYK encoded colors? Is the data represented by 8-bit quanta, or are they 16 bits in size, or perhaps 4 bits? How many rows and columns does the image consist of? Should certain pixels be transparent? I think you get the picture... To inform our software how to handle the raw data, it is encoded as a PNG file. It could be a GIF, or a JPEG, but it is a PNG. And PNG is a protocol. At this point, I can hear some of you yelling, No, it is not! It is a file format! Well, of course it is a file format. But from the perspective of data communications, a file format is a protocol: The file structure is a language, a terse one at that, communicating to our process how the data is organized. Ergo, it is a protocol. Alas, if all we received was the PNG file, our software would be facing a serious problem: How is it supposed to know the data is representing an image, as opposed to some text, or perhaps a sound, or what not? Secondly, how is it supposed to know the image is in the PNG format as opposed to GIF, or JPEG, or some other image format? To obtain that information, we are using another protocol: HTTP. This protocol can tell us exactly that the data represents an image, and that it uses the PNG protocol. It can also tell us some other things, but let us stay focused on protocol layers here. So, now we have some data wrapped in the PNG protocol, wrapped in the HTTP protocol. How did we get it from the server? By using TCP/IP over Ethernet, that is how. Indeed, that is three more protocols. Instead of continuing inside out, I am now going to talk about Ethernet, simply because it is easier to explain the rest that way. Ethernet is an interesting system of connecting computers in a local area network (LAN). Each computer has a network interface card (NIC), which has a unique 48-bit ID called its address. No two Ethernet NICs in the world have the same address. These NICs are all connected with each other. Whenever one computer wants to communicate with another in the same Ethernet LAN, it sends a message over the network. Every NIC sees the message. But as part of the Ethernet protocol, the data contains the address of the destination NIC (among other things). So, only one of all the network interface cards will pay attention to it, the rest will ignore it. But not all computers are connected to the same network. Just because we have received the data over our Ethernet does not mean it originated in our own local area network. It could have come to us from some other network (which may not even be Ethernet based) connected with our own network via the Internet. All data is transfered over the Internet using IP, which stands for Internet Protocol. Its basic role is to let us know where in the world the data has arrived from, and where it is supposed to go to. It does not guarantee we will receive the data, only that we will know where it came from if we do receive it. Even if we do receive the data, IP does not guarantee we will receive various chunks of data in the same order the other computer has sent it to us. So, we can receive the center of our image before we receive the upper left corner and after the lower right, for example. It is TCP (Transmission Control Protocol) that asks the sender to resend any lost data and that places it all into the proper order. All in all, it took five different protocols for one computer to communicate to another what an image looks like. We received the data wrapped into the PNG protocol, which was wrapped into the HTTP protocol, which was wrapped into the TCP protocol, which was wrapped into the IP protocol, which was wrapped into the Ethernet protocol. Oh, and by the way, there probably were several other protocols involved somewhere on the way. For example, if our LAN was connected to the Internet through a dial-up call, it used the PPP protocol over the modem which used one (or several) of the various modem protocols, et cetera, et cetera, et cetera... As a developer you should be asking by now, How am I supposed to handle it all? Luckily for you, you are not supposed to handle it all. You are supposed to handle some of it, but not all of it. Specifically, you need not worry about the physical connection (in our case Ethernet and possibly PPP, etc). Nor do you need to handle the Internet Protocol, or the Transmission Control Protocol. In other words, you do not have to do anything to receive the data from the other computer. Well, you do have to ask for it, but that is almost as simple as opening a file. Once you have received the data, it is up to you to figure out what to do with it. In our case, you would need to understand the HTTP protocol and the PNG file structure. To use an analogy, all the internetworking protocols become a gray area: Not so much because we do not understand how it works, but because we are no longer concerned about it. The sockets interface takes care of this gray area for us: +----------------+ |xxxxEthernetxxxx| |+--------------+| ||xxxxxxIPxxxxxx|| ||+------------+|| |||xxxxxTCPxxxx||| |||+----------+||| |||| HTTP |||| ||||+--------+|||| ||||| PNG ||||| |||||+------+||||| |||||| Data |||||| |||||+------+||||| ||||+--------+|||| |||+----------+||| ||+------------+|| |+--------------+| +----------------+ Sockets Covered Protocol Layers We only need to understand any protocols that tell us how to interpret the data, not how to receive it from another process, nor how to send it to another process. The Sockets Model BSD sockets are built on the basic Unix model: Everything is a file. In our example, then, sockets would let us receive an HTTP file, so to speak. It would then be up to us to extract the PNG file from it. Because of the complexity of internetworking, we cannot just use the open system call, or the open() C function. Instead, we need to take several steps to opening a socket. Once we do, however, we can start treating the socket the same way we treat any file descriptor: We can read from it, write to it, pipe it, and, eventually, close it. Essential Socket Functions While FreeBSD offers different functions to work with sockets, we only need four to open a socket. And in some cases we only need two. The Client-Server Difference Typically, one of the ends of a socket-based data communication is a server, the other is a client. The Common Elements <function>socket</function> The one function used by both, clients and servers, is &man.socket.2;. It is declared this way: int socket(int domain, int type, int protocol); The return value is of the same type as that of open, an integer. FreeBSD allocates its value from the same pool as that of file handles. That is what allows sockets to be treated the same way as files. The domain argument tells the system what protocol family you want it to use. Many of them exist, some are vendor specific, others are very common. They are declared in sys/socket.h. Use PF_INET for UDP, TCP and other Internet protocols (IPv4). Five values are defined for the type argument, again, in sys/socket.h. All of them start with SOCK_. The most common one is SOCK_STREAM, which tells the system you are asking for a reliable stream delivery service (which is TCP when used with PF_INET). If you asked for SOCK_DGRAM, you would be requesting a connectionless datagram delivery service (in our case, UDP). If you wanted to be in charge of the low-level protocols (such as IP), or even network interfaces (e.g., the Ethernet), you would need to specify SOCK_RAW. Finally, the protocol argument depends on the previous two arguments, and is not always meaningful. In that case, use 0 for its value. The Unconnected Socket Nowhere, in the socket function have we specified to what other system we should be connected. Our newly created socket remains unconnected. This is on purpose: To use a telephone analogy, we have just attached a modem to the phone line. We have neither told the modem to make a call, nor to answer if the phone rings. <varname>sockaddr</varname> Various functions of the sockets family expect the address of (or pointer to, to use C terminology) a small area of the memory. The various C declarations in the sys/socket.h refer to it as struct sockaddr. This structure is declared in the same file: /* * Structure used by kernel to store most * addresses. */ struct sockaddr { u_char sa_len; /* total length */ sa_family_t sa_family; /* address family */ char sa_data[14]; /* actually longer; address value */ }; #define SOCK_MAXADDRLEN 255 /* longest possible addresses */ Please note the vagueness with which the sa_data field is declared, just as an array of 14 bytes, with the comment hinting there can be more than 14 of them. This vagueness is quite deliberate. Sockets is a very powerful interface. While most people perhaps think of it as nothing more than the Internet interface—and most applications probably use it for that nowadays—sockets can be used for just about any kind of interprocess communications, of which the Internet (or, more precisely, IP) is only one. The sys/socket.h refers to the various types of protocols sockets will handle as address families, and lists them right before the definition of sockaddr: /* * Address families. */ #define AF_UNSPEC 0 /* unspecified */ #define AF_LOCAL 1 /* local to host (pipes, portals) */ #define AF_UNIX AF_LOCAL /* backward compatibility */ #define AF_INET 2 /* internetwork: UDP, TCP, etc. */ #define AF_IMPLINK 3 /* arpanet imp addresses */ #define AF_PUP 4 /* pup protocols: e.g. BSP */ #define AF_CHAOS 5 /* mit CHAOS protocols */ #define AF_NS 6 /* XEROX NS protocols */ #define AF_ISO 7 /* ISO protocols */ #define AF_OSI AF_ISO #define AF_ECMA 8 /* European computer manufacturers */ #define AF_DATAKIT 9 /* datakit protocols */ #define AF_CCITT 10 /* CCITT protocols, X.25 etc */ #define AF_SNA 11 /* IBM SNA */ #define AF_DECnet 12 /* DECnet */ #define AF_DLI 13 /* DEC Direct data link interface */ #define AF_LAT 14 /* LAT */ #define AF_HYLINK 15 /* NSC Hyperchannel */ #define AF_APPLETALK 16 /* Apple Talk */ #define AF_ROUTE 17 /* Internal Routing Protocol */ #define AF_LINK 18 /* Link layer interface */ #define pseudo_AF_XTP 19 /* eXpress Transfer Protocol (no AF) */ #define AF_COIP 20 /* connection-oriented IP, aka ST II */ #define AF_CNT 21 /* Computer Network Technology */ #define pseudo_AF_RTIP 22 /* Help Identify RTIP packets */ #define AF_IPX 23 /* Novell Internet Protocol */ #define AF_SIP 24 /* Simple Internet Protocol */ #define pseudo_AF_PIP 25 /* Help Identify PIP packets */ #define AF_ISDN 26 /* Integrated Services Digital Network*/ #define AF_E164 AF_ISDN /* CCITT E.164 recommendation */ #define pseudo_AF_KEY 27 /* Internal key-management function */ #define AF_INET6 28 /* IPv6 */ #define AF_NATM 29 /* native ATM access */ #define AF_ATM 30 /* ATM */ #define pseudo_AF_HDRCMPLT 31 /* Used by BPF to not rewrite headers * in interface output routine */ #define AF_NETGRAPH 32 /* Netgraph sockets */ #define AF_MAX 33 The one used for IP is AF_INET. It is a symbol for the constant 2. It is the address family listed in the sa_family field of sockaddr that decides how exactly the vaguely named bytes of sa_data will be used. Specifically, whenever the address family is AF_INET, we can use struct sockaddr_in found in netinet/in.h, wherever sockaddr is expected: /* * Socket address, internet style. */ struct sockaddr_in { u_char sin_len; u_char sin_family; u_short sin_port; struct in_addr sin_addr; char sin_zero[8]; }; We can visualize its organization this way: 0 1 2 3 +--------+--------+-----------------+ 0 | 0 | Family | Port | +--------+--------+-----------------+ 4 | IP Address | +-----------------------------------+ 8 | 0 | +-----------------------------------+ 12 | 0 | +-----------------------------------+ sockaddr_in The three important fields are sin_family, which is byte 1 of the structure, sin_port, a 16-bit value found in bytes 2 and 3, and sin_addr, a 32-bit integer representation of the IP address, stored in bytes 4-7. Now, let us try to fill it out. Let us assume we are trying to write a client for the daytime protocol, which simply states that its server will write a text string representing the current date and time to port 13. We want to use TCP/IP, so we need to specify AF_INET in the address family field. AF_INET is defined as 2. Let us use the IP address of 192.43.244.18, which is the time server of US federal government (time.nist.gov). 0 1 2 3 +--------+--------+-----------------+ 0 | 0 | 2 | 13 | +-----------------+-----------------+ 4 | 192.43.244.18 | +-----------------------------------+ 8 | 0 | +-----------------------------------+ 12 | 0 | +-----------------------------------+ Specific example of sockaddr_in By the way the sin_addr field is declared as being of the struct in_addr type, which is defined in netinet/in.h: /* * Internet address (a structure for historical reasons) */ struct in_addr { in_addr_t s_addr; }; In addition, in_addr_t is a 32-bit integer. The 192.43.244.18 is just a convenient notation of expressing a 32-bit integer by listing all of its 8-bit bytes, starting with the most significant one. So far, we have viewed sockaddr as an abstraction. Our computer does not store short integers as a single 16-bit entity, but as a sequence of 2 bytes. Similarly, it stores 32-bit integers as a sequence of 4 bytes. Suppose we coded something like this: sa.sin_family = AF_INET; sa.sin_port = 13; sa.sin_addr.s_addr = (((((192 << 8) | 43) << 8) | 244) << 8) | 18; What would the result look like? Well, that depends, of course. On a Pentium, or other x86, based computer, it would look like this: 0 1 2 3 +--------+--------+--------+--------+ 0 | 0 | 2 | 13 | 0 | +--------+--------+--------+--------+ 4 | 18 | 244 | 43 | 192 | +-----------------------------------+ 8 | 0 | +-----------------------------------+ 12 | 0 | +-----------------------------------+ sockaddr_in on an Intel system On a different system, it might look like this: 0 1 2 3 +--------+--------+--------+--------+ 0 | 0 | 2 | 0 | 13 | +--------+--------+--------+--------+ 4 | 192 | 43 | 244 | 18 | +-----------------------------------+ 8 | 0 | +-----------------------------------+ 12 | 0 | +-----------------------------------+ sockaddr_in on an MSB system And on a PDP it might look different yet. But the above two are the most common ways in use today. Ordinarily, wanting to write portable code, programmers pretend that these differences do not exist. And they get away with it (except when they code in assembly language). Alas, you cannot get away with it that easily when coding for sockets. Why? Because when communicating with another computer, you usually do not know whether it stores data most significant byte (MSB) or least significant byte (LSB) first. You might be wondering, So, will sockets not handle it for me? It will not. While that answer may surprise you at first, remember that the general sockets interface only understands the sa_len and sa_family fields of the sockaddr structure. You do not have to worry about the byte order there (of course, on FreeBSD sa_family is only 1 byte anyway, but many other Unix systems do not have sa_len and use 2 bytes for sa_family, and expect the data in whatever order is native to the computer). But the rest of the data is just sa_data[14] as far as sockets goes. Depending on the address family, sockets just forwards that data to its destination. Indeed, when we enter a port number, it is because we want the other computer to know what service we are asking for. And, when we are the server, we read the port number so we know what service the other computer is expecting from us. Either way, sockets only has to forward the port number as data. It does not interpret it in any way. Similarly, we enter the IP address to tell everyone on the way where to send our data to. Sockets, again, only forwards it as data. That is why, we (the programmers, not the sockets) have to distinguish between the byte order used by our computer and a conventional byte order to send the data in to the other computer. We will call the byte order our computer uses the host byte order, or just the host order. There is a convention of sending the multi-byte data over IP MSB first. This, we will refer to as the network byte order, or simply the network order. Now, if we compiled the above code for an Intel based computer, our host byte order would produce: 0 1 2 3 +--------+--------+--------+--------+ 0 | 0 | 2 | 13 | 0 | +--------+--------+--------+--------+ 4 | 18 | 244 | 43 | 192 | +-----------------------------------+ 8 | 0 | +-----------------------------------+ 12 | 0 | +-----------------------------------+ Host byte order on an Intel system But the network byte order requires that we store the data MSB first: 0 1 2 3 +--------+--------+--------+--------+ 0 | 0 | 2 | 0 | 13 | +--------+--------+--------+--------+ 4 | 192 | 43 | 244 | 18 | +-----------------------------------+ 8 | 0 | +-----------------------------------+ 12 | 0 | +-----------------------------------+ Network byte order Unfortunately, our host order is the exact opposite of the network order. We have several ways of dealing with it. One would be to reverse the values in our code: sa.sin_family = AF_INET; sa.sin_port = 13 << 8; sa.sin_addr.s_addr = (((((18 << 8) | 244) << 8) | 43) << 8) | 192; This will trick our compiler into storing the data in the network byte order. In some cases, this is exactly the way to do it (e.g., when programming in assembly language). In most cases, however, it can cause a problem. Suppose, you wrote a sockets-based program in C. You know it is going to run on a Pentium, so you enter all your constants in reverse and force them to the network byte order. It works well. Then, some day, your trusted old Pentium becomes a rusty old Pentium. You replace it with a system whose host order is the same as the network order. You need to recompile all your software. All of your software continues to perform well, except the one program you wrote. You have since forgotten that you had forced all of your constants to the opposite of the host order. You spend some quality time tearing out your hair, calling the names of all gods you ever heard of (and some you made up), hitting your monitor with a nerf bat, and performing all the other traditional ceremonies of trying to figure out why something that has worked so well is suddenly not working at all. Eventually, you figure it out, say a couple of swear words, and start rewriting your code. Luckily, you are not the first one to face the problem. Someone else has created the &man.htons.3; and &man.htonl.3; C functions to convert a short and long respectively from the host byte order to the network byte order, and the &man.ntohs.3; and &man.ntohl.3; C functions to go the other way. On MSB-first systems these functions do nothing. On LSB-first systems they convert values to the proper order. So, regardless of what system your software is compiled on, your data will end up in the correct order if you use these functions. Client Functions Typically, the client initiates the connection to the server. The client knows which server it is about to call: It knows its IP address, and it knows the port the server resides at. It is akin to you picking up the phone and dialing the number (the address), then, after someone answers, asking for the person in charge of wingdings (the port). <function>connect</function> Once a client has created a socket, it needs to connect it to a specific port on a remote system. It uses &man.connect.2;: int connect(int s, const struct sockaddr *name, socklen_t namelen); The s argument is the socket, i.e., the value returned by the socket function. The name is a pointer to sockaddr, the structure we have talked about extensively. Finaly, namelen informs the system how many bytes are in our sockaddr structure. If connect is successful, it returns 0. Otherwise it returns -1 and stores the error code in errno. There are many reasons why connect may fail. For example, with an attempt to an Internet connection, the IP address may not exist, or it may be down, or just too busy, or it may not have a server listening at the specified port. Or it may outright refuse any request for specific code. Our First Client We now know enough to write a very simple client, one that will get current time from 192.43.244.18 and print it to stdout. /* * daytime.c * * Programmed by G. Adam Stanislav */ #include <stdio.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> int main() { register int s; register int bytes; struct sockaddr_in sa; char buffer[BUFSIZ+1]; if ((s = socket(PF_INET, SOCK_STREAM, 0)) < 0) { perror("socket"); return 1; } bzero(&sa, sizeof sa); sa.sin_family = AF_INET; sa.sin_port = htons(13); sa.sin_addr.s_addr = htonl((((((192 << 8) | 43) << 8) | 244) << 8) | 18); if (connect(s, (struct sockaddr *)&sa, sizeof sa) < 0) { perror("connect"); close(s); return 2; } while ((bytes = read(s, buffer, BUFSIZ)) > 0) write(1, buffer, bytes); close(s); return 0; } Go ahead, enter it in your editor, save it as daytime.c, then compile and run it: - -&prompt.user; cc -O3 -o daytime daytime.c +&prompt.user; cc -O3 -o daytime daytime.c &prompt.user; ./daytime 52079 01-06-19 02:29:25 50 0 1 543.9 UTC(NIST) * -&prompt.user; - +&prompt.user; In this case, the date was June 19, 2001, the time was 02:29:25 UTC. Naturally, your results will vary. Server Functions The typical server does not initiate the connection. Instead, it waits for a client to call it and request services. It does not know when the client will call, nor how many clients will call. It may be just sitting there, waiting patiently, one moment, The next moment, it can find itself swamped with requests from a number of clients, all calling in at the same time. The sockets interface offers three basic functions to handle this. <function>bind</function> Ports are like extensions to a phone line: After you dial a number, you dial the extension to get to a specific person or department. There are 65535 IP ports, but a server usually processes requests that come in on only one of them. It is like telling the phone room operator that we are now at work and available to answer the phone at a specific extension. We use &man.bind.2; to tell sockets which port we want to serve. int bind(int s, const struct sockaddr *addr, socklen_t addrlen); Beside specifying the port in addr, the server may include its IP address. However, it can just use the symbolic constant INADDR_ANY to indicate it will serve all requests to the specified port regardless of what its IP address is. This symbol, along with several similar ones, is declared in netinet/in.h #define INADDR_ANY (u_int32_t)0x00000000 Suppose we were writing a server for the daytime protocol over TCP/IP. Recall that it uses port 13. Our sockaddr_in structure would look like this: 0 1 2 3 +--------+--------+--------+--------+ 0 | 0 | 2 | 0 | 13 | +--------+--------+--------+--------+ 4 | 0 | +-----------------------------------+ 8 | 0 | +-----------------------------------+ 12 | 0 | +-----------------------------------+ Example Server sockaddr_in <function>listen</function> To continue our office phone analogy, after you have told the phone central operator what extension you will be at, you now walk into your office, and make sure your own phone is plugged in and the ringer is turned on. Plus, you make sure your call waiting is activated, so you can hear the phone ring even while you are talking to someone. The server ensures all of that with the &man.listen.2; function. int listen(int s, int backlog); In here, the backlog variable tells sockets how many incoming requests to accept while you are busy processing the last request. In other words, it determines the maximum size of the queue of pending connections. <function>accept</function> After you hear the phone ringing, you accept the call by answering the call. You have now established a connection with your client. This connection remains active until either you or your client hang up. The server accepts the connection by using the &man.accept.2; function. int accept(int s, struct sockaddr *addr, socklen_t *addrlen); Note that this time addrlen is a pointer. This is necessary because in this case it is the socket that fills out addr, the sockaddr_in structure. The return value is an integer. Indeed, the accept returns a new socket. You will use this new socket to communicate with the client. What happens to the old socket? It continues to listen for more requests (remember the backlog variable we passed to listen?) until we close it. Now, the new socket is meant only for communications. It is fully connected. We cannot pass it to listen again, trying to accept additional connections. Our First Server Our first server will be somewhat more complex than our first client was: Not only do we have more sockets functions to use, but we need to write it as a daemon. This is best achieved by creating a child process after binding the port. The main process then exits and returns control to the shell (or whatever program invoked it). The child calls listen, then starts an endless loop, which accepts a connection, serves it, and eventually closes its socket. /* * daytimed - a port 13 server * * Programmed by G. Adam Stanislav * June 19, 2001 */ #include <stdio.h> #include <time.h> #include <unistd.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #define BACKLOG 4 int main() { register int s, c; int b; struct sockaddr_in sa; time_t t; struct tm *tm; FILE *client; if ((s = socket(PF_INET, SOCK_STREAM, 0)) < 0) { perror("socket"); return 1; } bzero(&sa, sizeof sa); sa.sin_family = AF_INET; sa.sin_port = htons(13); if (INADDR_ANY) sa.sin_addr.s_addr = htonl(INADDR_ANY); if (bind(s, (struct sockaddr *)&sa, sizeof sa) < 0) { perror("bind"); return 2; } switch (fork()) { case -1: perror("fork"); return 3; break; default: close s; return 0; break; case 0: break; } listen(s, BACKLOG); for (;;) { b = sizeof sa; if ((c = accept(s, (struct sockaddr *)&sa, &b)) < 0) { perror("daytimed accept"); return 4; } if ((client = fdopen(c, "w")) == NULL) { perror("daytimed fdopen"); return 5; } if ((t = time(NULL)) < 0) { perror("daytimed time"); return 6; } tm = gmtime(&t); fprintf(client, "%.4i-%.2i-%.2iT%.2i:%.2i:%.2iZ\n", tm->tm_year + 1900, tm->tm_mon + 1, tm->tm_mday, tm->tm_hour, tm->tm_min, tm->tm_sec); fclose(client); } } We start by creating a socket. Then we fill out the sockaddr_in structure in sa. Note the conditional use of INADDR_ANY: if (INADDR_ANY) sa.sin_addr.s_addr = htonl(INADDR_ANY); Its value is 0. Since we have just used bzero on the entire structure, it would be redundant to set it to 0 again. But if we port our code to some other system where INADDR_ANY is perhaps not a zero, we need to assign it to sa.sin_addr.s_addr. Most modern C compilers are clever enough to notice that INADDR_ANY is a constant. As long as it is a zero, they will optimize the entire conditional statement out of the code. After we have called bind successfully, we are ready to become a daemon: We use fork to create a child process. In both, the parent and the child, the s variable is our socket. The parent process will not need it, so it calls close, then it returns 0 to inform its own parent it had terminated successfully. Meanwhile, the child process continues working in the background. It calls listen and sets its backlog to 4. It does not need a large value here because daytime is not a protocol many clients request all the time, and because it can process each request instantly anyway. Finally, the daemon starts an endless loop, which performs the following steps: Call accept. It waits here until a client contacts it. At that point, it receives a new socket, c, which it can use to communicate with this particular client. It uses the C function fdopen to turn the socket from a low-level file descriptor to a C-style FILE pointer. This will allow the use of fprintf later on. It checks the time, and prints it in the ISO 8601 format to the client file. It then uses fclose to close the file. That will automatically close the socket as well. We can generalize this, and use it as a model for many other servers: +-----------------+ | Create Socket | +-----------------+ | +-----------------+ | Bind Port | Daemon Process +-----------------+ | +--------+ +-------------+-->| Init | | | +--------+ +-----------------+ | | | Exit | | +--------+ +-----------------+ | | Listen | | +--------+ | | | +--------+ | | Accept | | +--------+ | | | +--------+ | | Serve | | +--------+ | | | +--------+ | | Close | |<--------+ Sequential Server This flowchart is good for sequential servers, i.e., servers that can serve one client at a time, just as we were able to with our daytime server. This is only possible whenever there is no real conversation going on between the client and the server: As soon as the server detects a connection to the client, it sends out some data and closes the connection. The entire operation may take nanoseconds, and it is finished. The advantage of this flowchart is that, except for the brief moment after the parent forks and before it exits, there is always only one process active: Our server does not take up much memory and other system resources. Note that we have added initialize daemon in our flowchart. We did not need to initialize our own daemon, but this is a good place in the flow of the program to set up any signal handlers, open any files we may need, etc. Just about everything in the flow chart can be used literally on many different servers. The serve entry is the exception. We think of it as a black box, i.e., something you design specifically for your own server, and just plug it into the rest. Not all protocols are that simple. Many receive a request from the client, reply to it, then receive another request from the same client. Because of that, they do not know in advance how long they will be serving the client. Such servers usually start a new process for each client. While the new process is serving its client, the daemon can continue listening for more connections. Now, go ahead, save the above source code as daytimed.c (it is customary to end the names of daemons with the letter d). After you have compiled it, try running it: - -&prompt.user; ./daytimed +&prompt.user; ./daytimed bind: Permission denied -&prompt.user; - +&prompt.user; What happened here? As you will recall, the daytime protocol uses port 13. But all ports below 1024 are reserved to the superuser (otherwise, anyone could start a daemon pretending to serve a commonly used port, while causing a security breach). Try again, this time as the superuser: - -&prompt.root; ./daytimed -&prompt.root; - +&prompt.root; ./daytimed +&prompt.root; What... Nothing? Let us try again: - -&prompt.root; ./daytimed +&prompt.root; ./daytimed bind: Address already in use -&prompt.root; - +&prompt.root; Every port can only be bound by one program at a time. Our first attempt was indeed successful: It started the child daemon and returned quietly. It is still running and will continue to run until you either kill it, or any of its system calls fail, or you reboot the system. Fine, we know it is running in the background. But is it working? How do we know it is a proper daytime server? Simple: - -&prompt.user; telnet localhost 13 +&prompt.user; telnet localhost 13 Trying ::1... telnet: connect to address ::1: Connection refused Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 2001-06-19T21:04:42Z Connection closed by foreign host. -&prompt.user; - +&prompt.user; telnet tried the new IPv6, and failed. It retried with IPv4 and succeeded. The daemon works. If you have access to another Unix system via telnet, you can use it to test accessing the server remotely. My computer does not have a static IP address, so this is what I did: - -&prompt.user; who +&prompt.user; who whizkid ttyp0 Jun 19 16:59 (216.127.220.143) xxx ttyp1 Jun 19 16:06 (xx.xx.xx.xx) &prompt.user; telnet 216.127.220.143 13 Trying 216.127.220.143... Connected to r47.bfm.org. Escape character is '^]'. 2001-06-19T21:31:11Z Connection closed by foreign host. -&prompt.user; - +&prompt.user; Again, it worked. Will it work using the domain name? - -&prompt.user; telnet r47.bfm.org 13 +&prompt.user; telnet r47.bfm.org 13 Trying 216.127.220.143... Connected to r47.bfm.org. Escape character is '^]'. 2001-06-19T21:31:40Z Connection closed by foreign host. -&prompt.user; - +&prompt.user; By the way, telnet prints the Connection closed by foreign host message after our daemon has closed the socket. This shows us that, indeed, using fclose(client); in our code works as advertised. Helper Functions FreeBSD C library contains many helper functions for sockets programming. For example, in our sample client we hard coded the time.nist.gov IP address. But we do not always know the IP address. Even if we do, our software is more flexible if it allows the user to enter the IP address, or even the domain name. <function>gethostbyname</function> While there is no way to pass the domain name directly to any of the sockets functions, the FreeBSD C library comes with the &man.gethostbyname.3 and &man.gethostbyname2.3 functions, declared in netdb.h. struct hostent * gethostbyname(const char *name); struct hostent * gethostbyname2(const char *name, int af); Both return a pointer to the hostent structure, with much information about the domain. For our purposes, the h_addr_list[0] field of the structure points at h_length bytes of the correct address, already stored in the network byte order. This allows us to create a much more flexible—and much more useful—version of our daytime program: /* * daytime.c * * Programmed by G. Adam Stanislav * 19 June 2001 */ #include <stdio.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <netdb.h> int main(int argc, char *argv[]) { register int s; register int bytes; struct sockaddr_in sa; struct hostent *he; char buf[BUFSIZ+1]; char *host; if ((s = socket(PF_INET, SOCK_STREAM, 0)) < 0) { perror("socket"); return 1; } bzero(&sa, sizeof sa); sa.sin_family = AF_INET; sa.sin_port = htons(13); host = (argc > 1) ? (char *)argv[1] : "time.nist.gov"; if ((he = gethostbyname(host)) == NULL) { herror(host); return 2; } bcopy(he->h_addr_list[0],&sa.sin_addr, he->h_length); if (connect(s, (struct sockaddr *)&sa, sizeof sa) < 0) { perror("connect"); return 3; } while ((bytes = read(s, buf, BUFSIZ)) > 0) write(1, buf, bytes); close(s); return 0; } We now can type a domain name (or an IP address, it works both ways) on the command line, and the program will try to connect to its daytime server. Otherwise, it will still default to time.nist.gov. However, even in this case we will use gethostbyname rather than hard coding 192.43.244.18. That way, even if its IP address changes in the future, we will still find it. Since it takes virtually no time to get the time from your local server, you could run daytime twice in a row: First to get the time from time.nist.gov, the second time from your own system. You can then compare the results and see how exact your system clock is: - -&prompt.user; daytime ; daytime localhost +&prompt.user; daytime ; daytime localhost 52080 01-06-20 04:02:33 50 0 0 390.2 UTC(NIST) * 2001-06-20T04:02:35Z -&prompt.user; - +&prompt.user; As you can see, my system was two seconds ahead of the NIST time. <function>getservbyname</function> Sometimes you may not be sure what port a certain service uses. The &man.getservbyname.3; function, also declared in netdb.h comes in very handy in those cases: struct servent * getservbyname(const char *name, const char *proto); The servent structure contains the s_port, which contains the proper port, already in network byte order. Had we not known the correct port for the daytime service, we could have found it this way: struct servent *se; ... if ((se = getservbyname("daytime", "tcp")) == NULL { fprintf(stderr, "Cannot determine which port to use.\n"); return 7; } sa.sin_port = se->s_port; You usually do know the port. But if you are developing a new protocol, you may be testing it on an unofficial port. Some day, you will register the protocol and its port (if nowehere else, at least in your /etc/services, which is where getservbyname looks). Instead of returning an error in the above code, you just use the temporary port number. Once you have listed the protocol in /etc/services, your software will find its port without you having to rewrite the code. Concurrent Servers Unlike a sequential server, a concurrent server has to be able to serve more than one client at a time. For example, a chat server may be serving a specific client for hours—it cannot wait till it stops serving a client before it serves the next one. This requires a significant change in our flowchart: +-----------------+ | Create Socket | +-----------------+ | +-----------------+ | Bind Port | Daemon Process +-----------------+ | +--------+ +-------------+-->| Init | | | +--------+ +-----------------+ | | | Exit | | +--------+ +-----------------+ | | Listen | | +--------+ | | | +--------+ | | Accept | | +--------+ | | +------------------+ | +------>| Close Top Socket | | | +------------------+ | +--------+ | | | Close | +------------------+ | +--------+ | Serve | | | +------------------+ |<--------+ | +------------------+ | Close Acc Socket | +--------+ +------------------+ | Signal | | +--------+ +------------------+ | Exit | +------------------+ Concurrent Server We moved the serve from the daemon process to its own server process. However, because each child process inherits all open files (and a socket is treated just like a file), the new process inherits not only the accepted handle, i.e., the socket returned by the accept call, but also the top socket, i.e., the one opened by the top process right at the beginning. However, the server process does not need this socket and should close it immediately. Similarly, the daemon process no longer needs the accepted socket, and not only should, but must close it—otherwise, it will run out of available file descriptors sooner or later. After the server process is done serving, it should close the accepted socket. Instead of returning to accept, it now exits. Under Unix, a process does not really exit. Instead, it returns to its parent. Typically, a parent process waits for its child process, and obtains a return value. However, our daemon process cannot simply stop and wait. That would defeat the whole purpose of creating additional processes. But if it never does wait, its children will become zombies—no loger functional but still roaming around. For that reason, the daemon process needs to set signal handlers in its initialize daemon phase. At least a SIGCHLD signal has to be processed, so the daemon can remove the zombie return values from the system and release the system resources they are taking up. That is why our flowchart now contains a process signals box, which is not connected to any other box. By the way, many servers also process SIGHUP, and typically interpret as the signal from the superuser that they should reread their configuration files. This allows us to change settings without having to kill and restart these servers. diff --git a/en_US.ISO8859-1/books/developers-handbook/tools/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/tools/chapter.sgml index 2ca3ecd8f2..2ec1bb5f83 100644 --- a/en_US.ISO8859-1/books/developers-handbook/tools/chapter.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/tools/chapter.sgml @@ -1,2309 +1,2287 @@ Programming Tools This chapter was written by &a.jraynard;. Modifications for the Developers' Handbook by &a.murray;. Synopsis This document is an introduction to using some of the programming tools supplied with FreeBSD, although much of it will be applicable to many other versions of Unix. It does not attempt to describe coding in any detail. Most of the document assumes little or no previous programming knowledge, although it is hoped that most programmers will find something of value in it Introduction FreeBSD offers an excellent development environment. Compilers for C, C++, and Fortran and an assembler come with the basic system, not to mention a Perl interpreter and classic Unix tools such as sed and awk. If that is not enough, there are many more compilers and interpreters in the Ports collection. FreeBSD is very compatible with standards such as POSIX and ANSI C, as well with its own BSD heritage, so it is possible to write applications that will compile and run with little or no modification on a wide range of platforms. However, all this power can be rather overwhelming at first if you've never written programs on a Unix platform before. This document aims to help you get up and running, without getting too deeply into more advanced topics. The intention is that this document should give you enough of the basics to be able to make some sense of the documentation. Most of the document requires little or no knowledge of programming, although it does assume a basic competence with using Unix and a willingness to learn! Introduction to Programming A program is a set of instructions that tell the computer to do various things; sometimes the instruction it has to perform depends on what happened when it performed a previous instruction. This section gives an overview of the two main ways in which you can give these instructions, or commands as they are usually called. One way uses an interpreter, the other a compiler. As human languages are too difficult for a computer to understand in an unambiguous way, commands are usually written in one or other languages specially designed for the purpose. Interpreters With an interpreter, the language comes as an environment, where you type in commands at a prompt and the environment executes them for you. For more complicated programs, you can type the commands into a file and get the interpreter to load the file and execute the commands in it. If anything goes wrong, many interpreters will drop you into a debugger to help you track down the problem. The advantage of this is that you can see the results of your commands immediately, and mistakes can be corrected readily. The biggest disadvantage comes when you want to share your programs with someone. They must have the same interpreter, or you must have some way of giving it to them, and they need to understand how to use it. Also users may not appreciate being thrown into a debugger if they press the wrong key! From a performance point of view, interpreters can use up a lot of memory, and generally do not generate code as efficiently as compilers. In my opinion, interpreted languages are the best way to start if you have not done any programming before. This kind of environment is typically found with languages like Lisp, Smalltalk, Perl and Basic. It could also be argued that the Unix shell (sh, csh) is itself an interpreter, and many people do in fact write shell scripts to help with various housekeeping tasks on their machine. Indeed, part of the original Unix philosophy was to provide lots of small utility programs that could be linked together in shell scripts to perform useful tasks. Interpreters available with FreeBSD Here is a list of interpreters that are available as FreeBSD packages, with a brief discussion of some of the more popular interpreted languages. To get one of these packages, all you need to do is to click on the hotlink for the package, then run - &prompt.root; pkg_add package name - + &prompt.root; pkg_add package name as root. Obviously, you will need to have a fully functional FreeBSD 2.1.0 or later system for the package to work! BASIC Short for Beginner's All-purpose Symbolic Instruction Code. Developed in the 1950s for teaching University students to program and provided with every self-respecting personal computer in the 1980s, BASIC has been the first programming language for many programmers. It's also the foundation for Visual Basic. The Bywater Basic Interpreter and the Phil Cockroft's Basic Interpreter (formerly Rabbit Basic) are available as FreeBSD FreeBSD packages Lisp A language that was developed in the late 1950s as an alternative to the number-crunching languages that were popular at the time. Instead of being based on numbers, Lisp is based on lists; in fact the name is short for List Processing. Very popular in AI (Artificial Intelligence) circles. Lisp is an extremely powerful and sophisticated language, but can be rather large and unwieldy. FreeBSD has GNU Common Lisp available as a package. Perl Very popular with system administrators for writing scripts; also often used on World Wide Web servers for writing CGI scripts. The latest version (version 5) comes with FreeBSD. Scheme A dialect of Lisp that is rather more compact and cleaner than Common Lisp. Popular in Universities as it is simple enough to teach to undergraduates as a first language, while it has a high enough level of abstraction to be used in research work. FreeBSD has packages of the Elk Scheme Interpreter, the MIT Scheme Interpreter and the SCM Scheme Interpreter. Icon The Icon Programming Language. Logo Brian Harvey's LOGO Interpreter. Python The Python Object-Oriented Programming Language Compilers Compilers are rather different. First of all, you write your code in a file (or files) using an editor. You then run the compiler and see if it accepts your program. If it did not compile, grit your teeth and go back to the editor; if it did compile and gave you a program, you can run it either at a shell command prompt or in a debugger to see if it works properly. If you run it in the shell, you may get a core dump. Obviously, this is not quite as direct as using an interpreter. However it allows you to do a lot of things which are very difficult or even impossible with an interpreter, such as writing code which interacts closely with the operating system—or even writing your own operating system! It's also useful if you need to write very efficient code, as the compiler can take its time and optimise the code, which would not be acceptable in an interpreter. And distributing a program written for a compiler is usually more straightforward than one written for an interpreter—you can just give them a copy of the executable, assuming they have the same operating system as you. Compiled languages include Pascal, C and C++. C and C++ are rather unforgiving languages, and best suited to more experienced programmers; Pascal, on the other hand, was designed as an educational language, and is quite a good language to start with. Unfortunately, FreeBSD doesn't have any Pascal support, except for a Pascal-to-C converter in the ports. As the edit-compile-run-debug cycle is rather tedious when using separate programs, many commercial compiler makers have produced Integrated Development Environments (IDEs for short). FreeBSD does not have an IDE as such; however it is possible to use Emacs for this purpose. This is discussed in . Compiling with <command>cc</command> This section deals only with the GNU compiler for C and C++, since that comes with the base FreeBSD system. It can be invoked by either cc or gcc. The details of producing a program with an interpreter vary considerably between interpreters, and are usually well covered in the documentation and on-line help for the interpreter. Once you've written your masterpiece, the next step is to convert it into something that will (hopefully!) run on FreeBSD. This usually involves several steps, each of which is done by a separate program. Pre-process your source code to remove comments and do other tricks like expanding macros in C. Check the syntax of your code to see if you have obeyed the rules of the language. If you have not, it will complain! Convert the source code into assembly language—this is very close to machine code, but still understandable by humans. Allegedly. To be strictly accurate, cc converts the source code into its own, machine-independent p-code instead of assembly language at this stage. Convert the assembly language into machine code—yep, we are talking bits and bytes, ones and zeros here. Check that you have used things like functions and global variables in a consistent way. For example, if you have called a non-existent function, it will complain. If you are trying to produce an executable from several source code files, work out how to fit them all together. Work out how to produce something that the system's run-time loader will be able to load into memory and run. Finally, write the executable on the file system. The word compiling is often used to refer to just steps 1 to 4—the others are referred to as linking. Sometimes step 1 is referred to as pre-processing and steps 3-4 as assembling. Fortunately, almost all this detail is hidden from you, as cc is a front end that manages calling all these programs with the right arguments for you; simply typing - &prompt.user; cc foobar.c - + &prompt.user; cc foobar.c will cause foobar.c to be compiled by all the steps above. If you have more than one file to compile, just do something like - &prompt.user; cc foo.c bar.c - + &prompt.user; cc foo.c bar.c Note that the syntax checking is just that—checking the syntax. It will not check for any logical mistakes you may have made, like putting the program into an infinite loop, or using a bubble sort when you meant to use a binary sort. In case you didn't know, a binary sort is an efficient way of sorting things into order and a bubble sort isn't. There are lots and lots of options for cc, which are all in the man page. Here are a few of the most important ones, with examples of how to use them. The output name of the file. If you do not use this option, cc will produce an executable called a.out. The reasons for this are buried in the mists of history. &prompt.user; cc foobar.c executable is a.out &prompt.user; cc -o foobar foobar.c executable is foobar Just compile the file, do not link it. Useful for toy programs where you just want to check the syntax, or if you are using a Makefile. &prompt.user; cc -c foobar.c This will produce an object file (not an executable) called foobar.o. This can be linked together with other object files into an executable. Create a debug version of the executable. This makes the compiler put information into the executable about which line of which source file corresponds to which function call. A debugger can use this information to show the source code as you step through the program, which is very useful; the disadvantage is that all this extra information makes the program much bigger. Normally, you compile with while you are developing a program and then compile a release version without when you're satisfied it works properly. &prompt.user; cc -g foobar.c This will produce a debug version of the program. Note, we didn't use the flag to specify the executable name, so we will get an executable called a.out. Producing a debug version called foobar is left as an exercise for the reader! Create an optimised version of the executable. The compiler performs various clever tricks to try and produce an executable that runs faster than normal. You can add a number after the to specify a higher level of optimisation, but this often exposes bugs in the compiler's optimiser. For instance, the version of cc that comes with the 2.1.0 release of FreeBSD is known to produce bad code with the option in some circumstances. Optimisation is usually only turned on when compiling a release version. &prompt.user; cc -O -o foobar foobar.c This will produce an optimised version of foobar. The following three flags will force cc to check that your code complies to the relevant international standard, often referred to as the ANSI standard, though strictly speaking it is an ISO standard. Enable all the warnings which the authors of cc believe are worthwhile. Despite the name, it will not enable all the warnings cc is capable of. Turn off most, but not all, of the non-ANSI C features provided by cc. Despite the name, it does not guarantee strictly that your code will comply to the standard. Turn off all cc's non-ANSI C features. Without these flags, cc will allow you to use some of its non-standard extensions to the standard. Some of these are very useful, but will not work with other compilers—in fact, one of the main aims of the standard is to allow people to write code that will work with any compiler on any system. This is known as portable code. Generally, you should try to make your code as portable as possible, as otherwise you may have to completely re-write the program later to get it to work somewhere else—and who knows what you may be using in a few years time? - &prompt.user; cc -Wall -ansi -pedantic -o foobar foobar.c - + &prompt.user; cc -Wall -ansi -pedantic -o foobar foobar.c This will produce an executable foobar after checking foobar.c for standard compliance. Specify a function library to be used during when linking. The most common example of this is when compiling a program that uses some of the mathematical functions in C. Unlike most other platforms, these are in a separate library from the standard C one and you have to tell the compiler to add it. The rule is that if the library is called libsomething.a, you give cc the argument . For example, the math library is libm.a, so you give cc the argument . A common gotcha with the math library is that it has to be the last library on the command line. &prompt.user; cc -o foobar foobar.c -lm This will link the math library functions into foobar. If you are compiling C++ code, you need to add , or if you are using FreeBSD 2.2 or later, to the command line argument to link the C++ library functions. Alternatively, you can run c++ instead of cc, which does this for you. c++ can also be invoked as g++ on FreeBSD. &prompt.user; cc -o foobar foobar.cc -lg++ For FreeBSD 2.1.6 and earlier &prompt.user; cc -o foobar foobar.cc -lstdc++ For FreeBSD 2.2 and later &prompt.user; c++ -o foobar foobar.cc Each of these will both produce an executable foobar from the C++ source file foobar.cc. Note that, on Unix systems, C++ source files traditionally end in .C, .cxx or .cc, rather than the MS-DOS style .cpp (which was already used for something else). gcc used to rely on this to work out what kind of compiler to use on the source file; however, this restriction no longer applies, so you may now call your C++ files .cpp with impunity! Common <command>cc</command> Queries and Problems I am trying to write a program which uses the sin() function and I get an error like this. What does it mean? /var/tmp/cc0143941.o: Undefined symbol `_sin' referenced from text segment When using mathematical functions like sin(), you have to tell cc to link in the math library, like so: &prompt.user; cc -o foobar foobar.c -lm All right, I wrote this simple program to practice using . All it does is raise 2.1 to the power of 6. #include <stdio.h> int main() { float f; f = pow(2.1, 6); printf("2.1 ^ 6 = %f\n", f); return 0; } and I compiled it as: &prompt.user; cc temp.c -lm like you said I should, but I get this when I run it: &prompt.user; ./a.out 2.1 ^ 6 = 1023.000000 This is not the right answer! What is going on? When the compiler sees you call a function, it checks if it has already seen a prototype for it. If it has not, it assumes the function returns an int, which is definitely not what you want here. So how do I fix this? The prototypes for the mathematical functions are in math.h. If you include this file, the compiler will be able to find the prototype and it will stop doing strange things to your calculation! #include <math.h> #include <stdio.h> int main() { ... After recompiling it as you did before, run it: &prompt.user; ./a.out 2.1 ^ 6 = 85.766121 If you are using any of the mathematical functions, always include math.h and remember to link in the math library. I compiled a file called foobar.c and I cannot find an executable called foobar. Where's it gone? Remember, cc will call the executable a.out unless you tell it differently. Use the option: &prompt.user; cc -o foobar foobar.c OK, I have an executable called foobar, I can see it when I run ls, but when I type in foobar at the command prompt it tells me there is no such file. Why can it not find it? Unlike MS-DOS, Unix does not look in the current directory when it is trying to find out which executable you want it to run, unless you tell it to. Either type ./foobar, which means run the file called foobar in the current directory, or change your PATH environment variable so that it looks something like bin:/usr/bin:/usr/local/bin:. The dot at the end means look in the current directory if it is not in any of the others. I called my executable test, but nothing happens when I run it. What is going on? Most Unix systems have a program called test in /usr/bin and the shell is picking that one up before it gets to checking the current directory. Either type: &prompt.user; ./test or choose a better name for your program! I compiled my program and it seemed to run all right at first, then there was an error and it said something about core dumped. What does that mean? The name core dump dates back to the very early days of Unix, when the machines used core memory for storing data. Basically, if the program failed under certain conditions, the system would write the contents of core memory to disk in a file called core, which the programmer could then pore over to find out what went wrong. Fascinating stuff, but what I am supposed to do now? Use gdb to analyse the core (see ). When my program dumped core, it said something about a segmentation fault. What's that? This basically means that your program tried to perform some sort of illegal operation on memory; Unix is designed to protect the operating system and other programs from rogue programs. Common causes for this are: Trying to write to a NULL pointer, eg char *foo = NULL; strcpy(foo, "bang!"); Using a pointer that hasn't been initialised, eg char *foo; strcpy(foo, "bang!"); The pointer will have some random value that, with luck, will point into an area of memory that isn't available to your program and the kernel will kill your program before it can do any damage. If you're unlucky, it'll point somewhere inside your own program and corrupt one of your data structures, causing the program to fail mysteriously. Trying to access past the end of an array, eg int bar[20]; bar[27] = 6; Trying to store something in read-only memory, eg char *foo = "My string"; strcpy(foo, "bang!"); Unix compilers often put string literals like "My string" into read-only areas of memory. Doing naughty things with malloc() and free(), eg char bar[80]; free(bar); or char *foo = malloc(27); free(foo); free(foo); Making one of these mistakes will not always lead to an error, but they are always bad practice. Some systems and compilers are more tolerant than others, which is why programs that ran well on one system can crash when you try them on an another. Sometimes when I get a core dump it says bus error. It says in my Unix book that this means a hardware problem, but the computer still seems to be working. Is this true? No, fortunately not (unless of course you really do have a hardware problem…). This is usually another way of saying that you accessed memory in a way you shouldn't have. This dumping core business sounds as though it could be quite useful, if I can make it happen when I want to. Can I do this, or do I have to wait until there's an error? Yes, just go to another console or xterm, do &prompt.user; ps to find out the process ID of your program, and do &prompt.user; kill -ABRT pid where pid is the process ID you looked up. This is useful if your program has got stuck in an infinite loop, for instance. If your program happens to trap SIGABRT, there are several other signals which have a similar effect. Alternatively, you can create a core dump from inside your program, by calling the abort() function. See the man page of &man.abort.3; to learn more. If you want to create a core dump from outside your program, but don't want the process to terminate, you can use the gcore program. See the man page of &man.gcore.1 for more information. Make What is <command>make</command>? When you're working on a simple program with only one or two source files, typing in - &prompt.user; cc file1.c file2.c - + &prompt.user; cc file1.c file2.c is not too bad, but it quickly becomes very tedious when there are several files—and it can take a while to compile, too. One way to get around this is to use object files and only recompile the source file if the source code has changed. So we could have something like: - &prompt.user; cc file1.o file2.o … file37.c &hellip - + &prompt.user; cc file1.o file2.o … file37.c &hellip if we'd changed file37.c, but not any of the others, since the last time we compiled. This may speed up the compilation quite a bit, but doesn't solve the typing problem. Or we could write a shell script to solve the typing problem, but it would have to re-compile everything, making it very inefficient on a large project. What happens if we have hundreds of source files lying about? What if we're working in a team with other people who forget to tell us when they've changed one of their source files that we use? Perhaps we could put the two solutions together and write something like a shell script that would contain some kind of magic rule saying when a source file needs compiling. Now all we need now is a program that can understand these rules, as it's a bit too complicated for the shell. This program is called make. It reads in a file, called a makefile, that tells it how different files depend on each other, and works out which files need to be re-compiled and which ones don't. For example, a rule could say something like if fromboz.o is older than fromboz.c, that means someone must have changed fromboz.c, so it needs to be re-compiled. The makefile also has rules telling make how to re-compile the source file, making it a much more powerful tool. Makefiles are typically kept in the same directory as the source they apply to, and can be called makefile, Makefile or MAKEFILE. Most programmers use the name Makefile, as this puts it near the top of a directory listing, where it can easily be seen. They don't use the MAKEFILE form as block capitals are often used for documentation files like README. Example of using <command>make</command> Here's a very simple make file: foo: foo.c cc -o foo foo.c It consists of two lines, a dependency line and a creation line. The dependency line here consists of the name of the program (known as the target), followed by a colon, then whitespace, then the name of the source file. When make reads this line, it looks to see if foo exists; if it exists, it compares the time foo was last modified to the time foo.c was last modified. If foo does not exist, or is older than foo.c, it then looks at the creation line to find out what to do. In other words, this is the rule for working out when foo.c needs to be re-compiled. The creation line starts with a tab (press the tab key) and then the command you would type to create foo if you were doing it at a command prompt. If foo is out of date, or does not exist, make then executes this command to create it. In other words, this is the rule which tells make how to re-compile foo.c. So, when you type make, it will make sure that foo is up to date with respect to your latest changes to foo.c. This principle can be extended to Makefiles with hundreds of targets—in fact, on FreeBSD, it is possible to compile the entire operating system just by typing make world in the appropriate directory! Another useful property of makefiles is that the targets don't have to be programs. For instance, we could have a make file that looks like this: foo: foo.c cc -o foo foo.c install: cp foo /home/me We can tell make which target we want to make by typing: - &prompt.user; make target - + &prompt.user; make target make will then only look at that target and ignore any others. For example, if we type make foo with the makefile above, make will ignore the install target. If we just type make on its own, make will always look at the first target and then stop without looking at any others. So if we typed make here, it will just go to the foo target, re-compile foo if necessary, and then stop without going on to the install target. Notice that the install target doesn't actually depend on anything! This means that the command on the following line is always executed when we try to make that target by typing make install. In this case, it will copy foo into the user's home directory. This is often used by application makefiles, so that the application can be installed in the correct directory when it has been correctly compiled. This is a slightly confusing subject to try and explain. If you don't quite understand how make works, the best thing to do is to write a simple program like hello world and a make file like the one above and experiment. Then progress to using more than one source file, or having the source file include a header file. The touch command is very useful here—it changes the date on a file without you having to edit it. Make and include-files C code often starts with a list of files to include, for example stdio.h. Some of these files are system-include files, some of them are from the project you're now working on: #include <stdio.h> #include "foo.h" int main(.... To make sure that this file is recompiled the moment foo.h is changed, you have to add it in your Makefile: foo: foo.c foo.h The moment your project is getting bigger and you have more and more own include-files to maintain, it will be a pain to keep track of all include files and the files which are depending on it. If you change an include-file but forget to recompile all the files which are depending on it, the results will be devastating. gcc has an option to analyze your files and to produce a list of include-files and their dependencies: . If you add this to your Makefile: depend: gcc -E -MM *.c > .depend and run make depend, the file .depend will appear with a list of object-files, C-files and the include-files: foo.o: foo.c foo.h If you change foo.h, next time you run make all files depending on foo.h will be recompiled. Don't forget to run make depend each time you add an include-file to one of your files. FreeBSD Makefiles Makefiles can be rather complicated to write. Fortunately, BSD-based systems like FreeBSD come with some very powerful ones as part of the system. One very good example of this is the FreeBSD ports system. Here's the essential part of a typical ports Makefile: MASTER_SITES= ftp://freefall.cdrom.com/pub/FreeBSD/LOCAL_PORTS/ DISTFILES= scheme-microcode+dist-7.3-freebsd.tgz .include <bsd.port.mk> Now, if we go to the directory for this port and type make, the following happens: A check is made to see if the source code for this port is already on the system. If it isn't, an FTP connection to the URL in MASTER_SITES is set up to download the source. The checksum for the source is calculated and compared it with one for a known, good, copy of the source. This is to make sure that the source was not corrupted while in transit. Any changes required to make the source work on FreeBSD are applied—this is known as patching. Any special configuration needed for the source is done. (Many Unix program distributions try to work out which version of Unix they are being compiled on and which optional Unix features are present—this is where they are given the information in the FreeBSD ports scenario). The source code for the program is compiled. In effect, we change to the directory where the source was unpacked and do make—the program's own make file has the necessary information to build the program. We now have a compiled version of the program. If we wish, we can test it now; when we feel confident about the program, we can type make install. This will cause the program and any supporting files it needs to be copied into the correct location; an entry is also made into a package database, so that the port can easily be uninstalled later if we change our mind about it. Now I think you'll agree that's rather impressive for a four line script! The secret lies in the last line, which tells make to look in the system makefile called bsd.port.mk. It's easy to overlook this line, but this is where all the clever stuff comes from—someone has written a makefile that tells make to do all the things above (plus a couple of other things I didn't mention, including handling any errors that may occur) and anyone can get access to that just by putting a single line in their own make file! If you want to have a look at these system makefiles, they're in /usr/share/mk, but it's probably best to wait until you've had a bit of practice with makefiles, as they are very complicated (and if you do look at them, make sure you have a flask of strong coffee handy!) More advanced uses of <command>make</command> Make is a very powerful tool, and can do much more than the simple example above shows. Unfortunately, there are several different versions of make, and they all differ considerably. The best way to learn what they can do is probably to read the documentation—hopefully this introduction will have given you a base from which you can do this. The version of make that comes with FreeBSD is the Berkeley make; there is a tutorial for it in /usr/share/doc/psd/12.make. To view it, do - &prompt.user; zmore paper.ascii.gz - + &prompt.user; zmore paper.ascii.gz in that directory. Many applications in the ports use GNU make, which has a very good set of info pages. If you have installed any of these ports, GNU make will automatically have been installed as gmake. It's also available as a port and package in its own right. To view the info pages for GNU make, you will have to edit the dir file in the /usr/local/info directory to add an entry for it. This involves adding a line like * Make: (make). The GNU Make utility. to the file. Once you have done this, you can type info and then select make from the menu (or in Emacs, do C-h i). Debugging The Debugger The debugger that comes with FreeBSD is called gdb (GNU debugger). You start it up by typing - &prompt.user; gdb progname - + &prompt.user; gdb progname although most people prefer to run it inside Emacs. You can do this by: - M-x gdb RET progname RET - + M-x gdb RET progname RET Using a debugger allows you to run the program under more controlled circumstances. Typically, you can step through the program a line at a time, inspect the value of variables, change them, tell the debugger to run up to a certain point and then stop, and so on. You can even attach to a program that's already running, or load a core file to investigate why the program crashed. It's even possible to debug the kernel, though that's a little trickier than the user applications we'll be discussing in this section. gdb has quite good on-line help, as well as a set of info pages, so this section will concentrate on a few of the basic commands. Finally, if you find its text-based command-prompt style off-putting, there's a graphical front-end for it xxgdb in the ports collection. This section is intended to be an introduction to using gdb and does not cover specialised topics such as debugging the kernel. Running a program in the debugger You'll need to have compiled the program with the option to get the most out of using gdb. It will work without, but you'll only see the name of the function you're in, instead of the source code. If you see a line like: - … (no debugging symbols found) … - + … (no debugging symbols found) … when gdb starts up, you'll know that the program wasn't compiled with the option. At the gdb prompt, type break main. This will tell the debugger to skip over the preliminary set-up code in the program and start at the beginning of your code. Now type run to start the program—it will start at the beginning of the set-up code and then get stopped by the debugger when it calls main(). (If you've ever wondered where main() gets called from, now you know!). You can now step through the program, a line at a time, by pressing n. If you get to a function call, you can step into it by pressing s. Once you're in a function call, you can return from stepping into a function call by pressing f. You can also use up and down to take a quick look at the caller. Here's a simple example of how to spot a mistake in a program with gdb. This is our program (with a deliberate mistake): #include <stdio.h> int bazz(int anint); main() { int i; printf("This is my program\n"); bazz(i); return 0; } int bazz(int anint) { printf("You gave me %d\n", anint); return anint; } This program sets i to be 5 and passes it to a function bazz() which prints out the number we gave it. When we compile and run the program we get &prompt.user; cc -g -o temp temp.c &prompt.user; ./temp This is my program -anint = 4231 - +anint = 4231 That wasn't what we expected! Time to see what's going on! &prompt.user; gdb temp GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.13 (i386-unknown-freebsd), Copyright 1994 Free Software Foundation, Inc. (gdb) break main Skip the set-up code Breakpoint 1 at 0x160f: file temp.c, line 9. gdb puts breakpoint at main() (gdb) run Run as far as main() Starting program: /home/james/tmp/temp Program starts running Breakpoint 1, main () at temp.c:9 gdb stops at main() (gdb) n Go to next line This is my program Program prints out (gdb) s step into bazz() bazz (anint=4231) at temp.c:17 gdb displays stack frame -(gdb) - +(gdb) Hang on a minute! How did anint get to be 4231? Didn't we set it to be 5 in main()? Let's move up to main() and have a look. (gdb) up Move up call stack #1 0x1625 in main () at temp.c:11 gdb displays stack frame (gdb) p i Show us the value of i -$1 = 4231 gdb displays 4231 - +$1 = 4231 gdb displays 4231 Oh dear! Looking at the code, we forgot to initialise i. We meant to put … main() { int i; i = 5; printf("This is my program\n"); &hellip but we left the i=5; line out. As we didn't initialise i, it had whatever number happened to be in that area of memory when the program ran, which in this case happened to be 4231. gdb displays the stack frame every time we go into or out of a function, even if we're using up and down to move around the call stack. This shows the name of the function and the values of its arguments, which helps us keep track of where we are and what's going on. (The stack is a storage area where the program stores information about the arguments passed to functions and where to go when it returns from a function call). Examining a core file A core file is basically a file which contains the complete state of the process when it crashed. In the good old days, programmers had to print out hex listings of core files and sweat over machine code manuals, but now life is a bit easier. Incidentally, under FreeBSD and other 4.4BSD systems, a core file is called progname.core instead of just core, to make it clearer which program a core file belongs to. To examine a core file, start up gdb in the usual way. Instead of typing break or run, type - (gdb) core progname.core - + (gdb) core progname.core If you're not in the same directory as the core file, you'll have to do dir /path/to/core/file first. You should see something like this: &prompt.user; gdb a.out GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.13 (i386-unknown-freebsd), Copyright 1994 Free Software Foundation, Inc. (gdb) core a.out.core Core was generated by `a.out'. Program terminated with signal 11, Segmentation fault. Cannot access memory at address 0x7020796d. #0 0x164a in bazz (anint=0x5) at temp.c:17 -(gdb) - +(gdb) In this case, the program was called a.out, so the core file is called a.out.core. We can see that the program crashed due to trying to access an area in memory that was not available to it in a function called bazz. Sometimes it's useful to be able to see how a function was called, as the problem could have occurred a long way up the call stack in a complex program. The bt command causes gdb to print out a back-trace of the call stack: (gdb) bt #0 0x164a in bazz (anint=0x5) at temp.c:17 #1 0xefbfd888 in end () #2 0x162c in main () at temp.c:11 -(gdb) - +(gdb) The end() function is called when a program crashes; in this case, the bazz() function was called from main(). Attaching to a running program One of the neatest features about gdb is that it can attach to a program that's already running. Of course, that assumes you have sufficient permissions to do so. A common problem is when you are stepping through a program that forks, and you want to trace the child, but the debugger will only let you trace the parent. What you do is start up another gdb, use ps to find the process ID for the child, and do - (gdb) attach pid - + (gdb) attach pid in gdb, and then debug as usual. That's all very well, you're probably thinking, but by the time I've done that, the child process will be over the hill and far away. Fear not, gentle reader, here's how to do it (courtesy of the gdb info pages): &hellip if ((pid = fork()) < 0) /* _Always_ check this */ error(); else if (pid == 0) { /* child */ int PauseMode = 1; while (PauseMode) sleep(10); /* Wait until someone attaches to us */ &hellip } else { /* parent */ - &hellip - + &hellip Now all you have to do is attach to the child, set PauseMode to 0, and wait for the sleep() call to return! Using Emacs as a Development Environment Emacs Unfortunately, Unix systems don't come with the kind of everything-you-ever-wanted-and-lots-more-you-didn't-in-one-gigantic-package integrated development environments that other systems have. At least, not unless you pay out very large sums of money. However, it is possible to set up your own environment. It may not be as pretty, and it may not be quite as integrated, but you can set it up the way you want it. And it's free. And you have the source to it. The key to it all is Emacs. Now there are some people who loathe it, but many who love it. If you're one of the former, I'm afraid this section will hold little of interest to you. Also, you'll need a fair amount of memory to run it—I'd recommend 8MB in text mode and 16MB in X as the bare minimum to get reasonable performance. Emacs is basically a highly customisable editor—indeed, it has been customised to the point where it's more like an operating system than an editor! Many developers and sysadmins do in fact spend practically all their time working inside Emacs, leaving it only to log out. It's impossible even to summarise everything Emacs can do here, but here are some of the features of interest to developers: Very powerful editor, allowing search-and-replace on both strings and regular expressions (patterns), jumping to start/end of block expression, etc, etc. Pull-down menus and online help. Language-dependent syntax highlighting and indentation. Completely customisable. You can compile and debug programs within Emacs. On a compilation error, you can jump to the offending line of source code. Friendly-ish front-end to the info program used for reading GNU hypertext documentation, including the documentation on Emacs itself. Friendly front-end to gdb, allowing you to look at the source code as you step through your program. You can read Usenet news and mail while your program is compiling. And doubtless many more that I've overlooked. Emacs can be installed on FreeBSD using the Emacs port. Once it's installed, start it up and do C-h t to read an Emacs tutorial—that means hold down the control key, press h, let go of the control key, and then press t. (Alternatively, you can you use the mouse to select Emacs Tutorial from the Help menu). Although Emacs does have menus, it's well worth learning the key bindings, as it's much quicker when you're editing something to press a couple of keys than to try and find the mouse and then click on the right place. And, when you're talking to seasoned Emacs users, you'll find they often casually throw around expressions like M-x replace-s RET foo RET bar RET so it's useful to know what they mean. And in any case, Emacs has far too many useful functions for them to all fit on the menu bars. Fortunately, it's quite easy to pick up the key-bindings, as they're displayed next to the menu item. My advice is to use the menu item for, say, opening a file until you understand how it works and feel confident with it, then try doing C-x C-f. When you're happy with that, move on to another menu command. If you can't remember what a particular combination of keys does, select Describe Key from the Help menu and type it in—Emacs will tell you what it does. You can also use the Command Apropos menu item to find out all the commands which contain a particular word in them, with the key binding next to it. By the way, the expression above means hold down the Meta key, press x, release the Meta key, type replace-s (short for replace-string—another feature of Emacs is that you can abbreviate commands), press the return key, type foo (the string you want replaced), press the return key, type bar (the string you want to replace foo with) and press return again. Emacs will then do the search-and-replace operation you've just requested. If you're wondering what on earth the Meta key is, it's a special key that many Unix workstations have. Unfortunately, PC's don't have one, so it's usually the alt key (or if you're unlucky, the escape key). Oh, and to get out of Emacs, do C-x C-c (that means hold down the control key, press x, press c and release the control key). If you have any unsaved files open, Emacs will ask you if you want to save them. (Ignore the bit in the documentation where it says C-z is the usual way to leave Emacs—that leaves Emacs hanging around in the background, and is only really useful if you're on a system which doesn't have virtual terminals). Configuring Emacs Emacs does many wonderful things; some of them are built in, some of them need to be configured. Instead of using a proprietary macro language for configuration, Emacs uses a version of Lisp specially adapted for editors, known as Emacs Lisp. This can be quite useful if you want to go on and learn something like Common Lisp, as it's considerably smaller than Common Lisp (although still quite big!). The best way to learn Emacs Lisp is to download the Emacs Tutorial However, there's no need to actually know any Lisp to get started with configuring Emacs, as I've included a sample .emacs file, which should be enough to get you started. Just copy it into your home directory and restart Emacs if it's already running; it will read the commands from the file and (hopefully) give you a useful basic setup. A sample <filename>.emacs</filename> file Unfortunately, there's far too much here to explain it in detail; however there are one or two points worth mentioning. Everything beginning with a ; is a comment and is ignored by Emacs. In the first line, the -*- Emacs-Lisp -*- is so that we can edit the .emacs file itself within Emacs and get all the fancy features for editing Emacs Lisp. Emacs usually tries to guess this based on the filename, and may not get it right for .emacs. The tab key is bound to an indentation function in some modes, so when you press the tab key, it will indent the current line of code. If you want to put a tab character in whatever you're writing, hold the control key down while you're pressing the tab key. This file supports syntax highlighting for C, C++, Perl, Lisp and Scheme, by guessing the language from the filename. Emacs already has a pre-defined function called next-error. In a compilation output window, this allows you to move from one compilation error to the next by doing M-n; we define a complementary function, previous-error, that allows you to go to a previous error by doing M-p. The nicest feature of all is that C-c C-c will open up the source file in which the error occurred and jump to the appropriate line. We enable Emacs's ability to act as a server, so that if you're doing something outside Emacs and you want to edit a file, you can just type in &prompt.user; emacsclient filename and then you can edit the file in your Emacs! Many Emacs users set their EDITOR environment to emacsclient so this happens every time they need to edit a file. A sample <filename>.emacs</filename> file ;; -*-Emacs-Lisp-*- ;; This file is designed to be re-evaled; use the variable first-time ;; to avoid any problems with this. (defvar first-time t "Flag signifying this is the first time that .emacs has been evaled") ;; Meta (global-set-key "\M- " 'set-mark-command) (global-set-key "\M-\C-h" 'backward-kill-word) (global-set-key "\M-\C-r" 'query-replace) (global-set-key "\M-r" 'replace-string) (global-set-key "\M-g" 'goto-line) (global-set-key "\M-h" 'help-command) ;; Function keys (global-set-key [f1] 'manual-entry) (global-set-key [f2] 'info) (global-set-key [f3] 'repeat-complex-command) (global-set-key [f4] 'advertised-undo) (global-set-key [f5] 'eval-current-buffer) (global-set-key [f6] 'buffer-menu) (global-set-key [f7] 'other-window) (global-set-key [f8] 'find-file) (global-set-key [f9] 'save-buffer) (global-set-key [f10] 'next-error) (global-set-key [f11] 'compile) (global-set-key [f12] 'grep) (global-set-key [C-f1] 'compile) (global-set-key [C-f2] 'grep) (global-set-key [C-f3] 'next-error) (global-set-key [C-f4] 'previous-error) (global-set-key [C-f5] 'display-faces) (global-set-key [C-f8] 'dired) (global-set-key [C-f10] 'kill-compilation) ;; Keypad bindings (global-set-key [up] "\C-p") (global-set-key [down] "\C-n") (global-set-key [left] "\C-b") (global-set-key [right] "\C-f") (global-set-key [home] "\C-a") (global-set-key [end] "\C-e") (global-set-key [prior] "\M-v") (global-set-key [next] "\C-v") (global-set-key [C-up] "\M-\C-b") (global-set-key [C-down] "\M-\C-f") (global-set-key [C-left] "\M-b") (global-set-key [C-right] "\M-f") (global-set-key [C-home] "\M-<") (global-set-key [C-end] "\M->") (global-set-key [C-prior] "\M-<") (global-set-key [C-next] "\M->") ;; Mouse (global-set-key [mouse-3] 'imenu) ;; Misc (global-set-key [C-tab] "\C-q\t") ; Control tab quotes a tab. (setq backup-by-copying-when-mismatch t) ;; Treat 'y' or <CR> as yes, 'n' as no. (fset 'yes-or-no-p 'y-or-n-p) (define-key query-replace-map [return] 'act) (define-key query-replace-map [?\C-m] 'act) ;; Load packages (require 'desktop) (require 'tar-mode) ;; Pretty diff mode (autoload 'ediff-buffers "ediff" "Intelligent Emacs interface to diff" t) (autoload 'ediff-files "ediff" "Intelligent Emacs interface to diff" t) (autoload 'ediff-files-remote "ediff" "Intelligent Emacs interface to diff") (if first-time (setq auto-mode-alist (append '(("\\.cpp$" . c++-mode) ("\\.hpp$" . c++-mode) ("\\.lsp$" . lisp-mode) ("\\.scm$" . scheme-mode) ("\\.pl$" . perl-mode) ) auto-mode-alist))) ;; Auto font lock mode (defvar font-lock-auto-mode-list (list 'c-mode 'c++-mode 'c++-c-mode 'emacs-lisp-mode 'lisp-mode 'perl-mode 'scheme-mode) "List of modes to always start in font-lock-mode") (defvar font-lock-mode-keyword-alist '((c++-c-mode . c-font-lock-keywords) (perl-mode . perl-font-lock-keywords)) "Associations between modes and keywords") (defun font-lock-auto-mode-select () "Automatically select font-lock-mode if the current major mode is in font-lock-auto-mode-list" (if (memq major-mode font-lock-auto-mode-list) (progn (font-lock-mode t)) ) ) (global-set-key [M-f1] 'font-lock-fontify-buffer) ;; New dabbrev stuff ;(require 'new-dabbrev) (setq dabbrev-always-check-other-buffers t) (setq dabbrev-abbrev-char-regexp "\\sw\\|\\s_") (add-hook 'emacs-lisp-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) nil) (set (make-local-variable 'dabbrev-case-replace) nil))) (add-hook 'c-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) nil) (set (make-local-variable 'dabbrev-case-replace) nil))) (add-hook 'text-mode-hook '(lambda () (set (make-local-variable 'dabbrev-case-fold-search) t) (set (make-local-variable 'dabbrev-case-replace) t))) ;; C++ and C mode... (defun my-c++-mode-hook () (setq tab-width 4) (define-key c++-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key c++-mode-map "\C-ce" 'c-comment-edit) (setq c++-auto-hungry-initial-state 'none) (setq c++-delete-function 'backward-delete-char) (setq c++-tab-always-indent t) (setq c-indent-level 4) (setq c-continued-statement-offset 4) (setq c++-empty-arglist-indent 4)) (defun my-c-mode-hook () (setq tab-width 4) (define-key c-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key c-mode-map "\C-ce" 'c-comment-edit) (setq c-auto-hungry-initial-state 'none) (setq c-delete-function 'backward-delete-char) (setq c-tab-always-indent t) ;; BSD-ish indentation style (setq c-indent-level 4) (setq c-continued-statement-offset 4) (setq c-brace-offset -4) (setq c-argdecl-indent 0) (setq c-label-offset -4)) ;; Perl mode (defun my-perl-mode-hook () (setq tab-width 4) (define-key c++-mode-map "\C-m" 'reindent-then-newline-and-indent) (setq perl-indent-level 4) (setq perl-continued-statement-offset 4)) ;; Scheme mode... (defun my-scheme-mode-hook () (define-key scheme-mode-map "\C-m" 'reindent-then-newline-and-indent)) ;; Emacs-Lisp mode... (defun my-lisp-mode-hook () (define-key lisp-mode-map "\C-m" 'reindent-then-newline-and-indent) (define-key lisp-mode-map "\C-i" 'lisp-indent-line) (define-key lisp-mode-map "\C-j" 'eval-print-last-sexp)) ;; Add all of the hooks... (add-hook 'c++-mode-hook 'my-c++-mode-hook) (add-hook 'c-mode-hook 'my-c-mode-hook) (add-hook 'scheme-mode-hook 'my-scheme-mode-hook) (add-hook 'emacs-lisp-mode-hook 'my-lisp-mode-hook) (add-hook 'lisp-mode-hook 'my-lisp-mode-hook) (add-hook 'perl-mode-hook 'my-perl-mode-hook) ;; Complement to next-error (defun previous-error (n) "Visit previous compilation error message and corresponding source code." (interactive "p") (next-error (- n))) ;; Misc... (transient-mark-mode 1) (setq mark-even-if-inactive t) (setq visible-bell nil) (setq next-line-add-newlines nil) (setq compile-command "make") (setq suggest-key-bindings nil) (put 'eval-expression 'disabled nil) (put 'narrow-to-region 'disabled nil) (put 'set-goal-column 'disabled nil) ;; Elisp archive searching (autoload 'format-lisp-code-directory "lispdir" nil t) (autoload 'lisp-dir-apropos "lispdir" nil t) (autoload 'lisp-dir-retrieve "lispdir" nil t) (autoload 'lisp-dir-verify "lispdir" nil t) ;; Font lock mode (defun my-make-face (face colour &optional bold) "Create a face from a colour and optionally make it bold" (make-face face) (copy-face 'default face) (set-face-foreground face colour) (if bold (make-face-bold face)) ) (if (eq window-system 'x) (progn (my-make-face 'blue "blue") (my-make-face 'red "red") (my-make-face 'green "dark green") (setq font-lock-comment-face 'blue) (setq font-lock-string-face 'bold) (setq font-lock-type-face 'bold) (setq font-lock-keyword-face 'bold) (setq font-lock-function-name-face 'red) (setq font-lock-doc-string-face 'green) (add-hook 'find-file-hooks 'font-lock-auto-mode-select) (setq baud-rate 1000000) (global-set-key "\C-cmm" 'menu-bar-mode) (global-set-key "\C-cms" 'scroll-bar-mode) (global-set-key [backspace] 'backward-delete-char) ; (global-set-key [delete] 'delete-char) (standard-display-european t) (load-library "iso-transl"))) ;; X11 or PC using direct screen writes (if window-system (progn ;; (global-set-key [M-f1] 'hilit-repaint-command) ;; (global-set-key [M-f2] [?\C-u M-f1]) (setq hilit-mode-enable-list '(not text-mode c-mode c++-mode emacs-lisp-mode lisp-mode scheme-mode) hilit-auto-highlight nil hilit-auto-rehighlight 'visible hilit-inhibit-hooks nil hilit-inhibit-rebinding t) (require 'hilit19) (require 'paren)) (setq baud-rate 2400) ; For slow serial connections ) ;; TTY type terminal (if (and (not window-system) (not (equal system-type 'ms-dos))) (progn (if first-time (progn (keyboard-translate ?\C-h ?\C-?) (keyboard-translate ?\C-? ?\C-h))))) ;; Under UNIX (if (not (equal system-type 'ms-dos)) (progn (if first-time (server-start)))) ;; Add any face changes here (add-hook 'term-setup-hook 'my-term-setup-hook) (defun my-term-setup-hook () (if (eq window-system 'pc) (progn ;; (set-face-background 'default "red") ))) ;; Restore the "desktop" - do this as late as possible (if first-time (progn (desktop-load-default) (desktop-read))) ;; Indicate that this file has been read at least once (setq first-time nil) ;; No need to debug anything now (setq debug-on-error nil) ;; All done (message "All done, %s%s" (user-login-name) ".") Extending the Range of Languages Emacs Understands Now, this is all very well if you only want to program in the languages already catered for in the .emacs file (C, C++, Perl, Lisp and Scheme), but what happens if a new language called whizbang comes out, full of exciting features? The first thing to do is find out if whizbang comes with any files that tell Emacs about the language. These usually end in .el, short for Emacs Lisp. For example, if whizbang is a FreeBSD port, we can locate these files by doing - &prompt.user; find /usr/ports/lang/whizbang -name "*.el" -print - + &prompt.user; find /usr/ports/lang/whizbang -name "*.el" -print and install them by copying them into the Emacs site Lisp directory. On FreeBSD 2.1.0-RELEASE, this is /usr/local/share/emacs/site-lisp. So for example, if the output from the find command was - /usr/ports/lang/whizbang/work/misc/whizbang.el - + /usr/ports/lang/whizbang/work/misc/whizbang.el we would do - &prompt.root; cp /usr/ports/lang/whizbang/work/misc/whizbang.el /usr/local/share/emacs/site-lisp - + &prompt.root; cp /usr/ports/lang/whizbang/work/misc/whizbang.el /usr/local/share/emacs/site-lisp Next, we need to decide what extension whizbang source files have. Let's say for the sake of argument that they all end in .wiz. We need to add an entry to our .emacs file to make sure Emacs will be able to use the information in whizbang.el. Find the auto-mode-alist entry in .emacs and add a line for whizbang, such as: … ("\\.lsp$" . lisp-mode) ("\\.wiz$" . whizbang-mode) ("\\.scm$" . scheme-mode) … This means that Emacs will automatically go into whizbang-mode when you edit a file ending in .wiz. Just below this, you'll find the font-lock-auto-mode-list entry. Add whizbang-mode to it like so: ;; Auto font lock mode (defvar font-lock-auto-mode-list (list 'c-mode 'c++-mode 'c++-c-mode 'emacs-lisp-mode 'whizbang-mode 'lisp-mode 'perl-mode 'scheme-mode) "List of modes to always start in font-lock-mode") This means that Emacs will always enable font-lock-mode (ie syntax highlighting) when editing a .wiz file. And that's all that's needed. If there's anything else you want done automatically when you open up a .wiz file, you can add a whizbang-mode hook (see my-scheme-mode-hook for a simple example that adds auto-indent). Further Reading Brian Harvey and Matthew Wright Simply Scheme MIT 1994. ISBN 0-262-08226-8 Randall Schwartz Learning Perl O'Reilly 1993 ISBN 1-56592-042-2 Patrick Henry Winston and Berthold Klaus Paul Horn Lisp (3rd Edition) Addison-Wesley 1989 ISBN 0-201-08319-1 Brian W. Kernighan and Rob Pike The Unix Programming Environment Prentice-Hall 1984 ISBN 0-13-937681-X Brian W. Kernighan and Dennis M. Ritchie The C Programming Language (2nd Edition) Prentice-Hall 1988 ISBN 0-13-110362-8 Bjarne Stroustrup The C++ Programming Language Addison-Wesley 1991 ISBN 0-201-53992-6 W. Richard Stevens Advanced Programming in the Unix Environment Addison-Wesley 1992 ISBN 0-201-56317-7 W. Richard Stevens Unix Network Programming Prentice-Hall 1990 ISBN 0-13-949876-1 diff --git a/en_US.ISO8859-1/books/developers-handbook/x86/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/x86/chapter.sgml index d5b2df1551..c7a74aa21d 100644 --- a/en_US.ISO8859-1/books/developers-handbook/x86/chapter.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/x86/chapter.sgml @@ -1,6539 +1,6488 @@ x86 Assembly Language Programming This chapter was written by G. Adam Stanislav. Whiz Kid Technomagic Synopsis Assembly language programing under Unix is highly undocumented. It is generally assumed that no one would ever want to use it because various Unix systems run on different microprocessors, so everything should be written in C for portability. In reality, C portability is quite a myth. Even C programs need to be modified when ported from one Unix to another, regardless of what processor each runs on. Typically, such a program is full of conditional statements depending on the system it is compiled for. Even if we believe that all of Unix software should be written in C, or some other high-level language, we still need assembly language programmers: Who else would write the section of C library that accesses the kernel? In this chapter I will attempt to show you how you can use assembly language writing Unix programs, specifically under FreeBSD. This chapter does not explain the basics of assembly language. There are enough resources about that (for a complete online course in assembly language, see Randall Hyde's Art of Assembly Language; or if you prefer a printed book, take a look at Jeff Duntemann's Assembly Language Step-by-Step). However, once the chapter is finished, any assembly language programmer will be able to write programs for FreeBSD quickly and efficiently. Copyright © 2000-2001 G. Adam Stanislav. All rights reserved. The Tools The Assembler The most important tool for assembly language programming is the assembler, the software that converts assembly language code into machine language. Two very different assemblers are available for FreeBSD. One is as1, which uses the traditional Unix assembly language syntax. It comes with the system. The other is /usr/ports/devel/nasm. It uses the Intel syntax. Its main advantage is that it can assemble code for many operating systems. It needs to be installed separately, but is completely free. This chapter uses nasm syntax because most assembly language programmers coming to FreeBSD from other operating systems will find it easier to understand. And, because, quite frankly, that is what I am used to. The Linker The output of the assembler, like that of any compiler, needs to be linked to form an executable file. The standard ld1 linker comes with FreeBSD. It works with the code assembled with either assembler. System Calls Default Calling Convention By default, the FreeBSD kernel uses the C calling convention. Further, although the kernel is accessed using int 80h, it is assumed the program will call a function that issues int 80h, rather than issuing int 80h directly. This convention is very convenient, and quite superior to the Microsoft convention used by MS DOS. Why? Because the Unix convention allows any program written in any language to access the kernel. An assembly language program can do that as well. For example, we could open a file: kernel: int 80h ; Call kernel ret open: push dword mode push dword flags push dword path mov eax, 5 call kernel add esp, byte 12 ret This is a very clean and portable way of coding. If you need to port the code to a Unix system which uses a different interrupt, or a different way of passing parameters, all you need to change is the kernel procedure. But assembly language programmers like to shave off cycles. The above example requires a call/ret combination. We can eliminate it by pushing an extra dword: open: push dword mode push dword flags push dword path mov eax, 5 push eax ; Or any other dword int 80h add esp, byte 16 The 5 that we have placed in EAX identifies the kernel function, in this case open. Alternate Calling Convention FreeBSD is an extremely flexible system. It offers other ways of calling the kernel. For it to work, however, the system must have Linux emulation installed. Linux is a Unix-like system. However, its kernel uses the same system-call convention of passing parameters in registers MS DOS does. As with the Unix convention, the function number is placed in EAX. The parameters, however, are not passed on the stack but in EBX, ECX, EDX, ESI, EDI, EBP: open: mov eax, 5 mov ebx, path mov ecx, flags mov edx, mode int 80h This convention has a great disadvantage over the Unix way, at least as far as assembly language programming is concerned: Every time you make a kernel call you must push the registers, then pop them later. This makes your code bulkier and slower. Nevertheless, FreeBSD gives you a choice. If you do choose the Linux convention, you must let the system know about it. After your program is assembled and linked, you need to brand the executable: - -&prompt.user; brandelf -f Linux filename - +&prompt.user; brandelf -f Linux filename Which Convention Should You Use? If you are coding specifically for FreeBSD, you should always use the Unix convention: It is faster, you can store global variables in registers, you do not have to brand the executable, and you do not impose the installation of the Linux emulation package on the target system. If you want to create portable code that can also run on Linux, you will probably still want to give the FreeBSD users as efficient a code as possible. I will show you how you can accomplish that after I have explained the basics. Call Numbers To tell the kernel which system service you are calling, place its number in EAX. Of course, you need to know what the number is. The <filename>syscalls</filename> File The numbers are listed in syscalls. locate syscalls finds this file in several different formats, all produced automatically from syscalls.master. You can find the master file for the default Unix calling convention in /usr/src/sys/kern/syscalls.master. If you need to use the other convention implemented in the Linux emulation mode, read /usr/src/sys/i386/linux/syscalls.master. Not only do FreeBSD and Linux use different calling conventions, they sometimes use different numbers for the same functions. syscalls.master describes how the call is to be made: 0 STD NOHIDE { int nosys(void); } syscall nosys_args int 1 STD NOHIDE { void exit(int rval); } exit rexit_args void 2 STD POSIX { int fork(void); } 3 STD POSIX { ssize_t read(int fd, void *buf, size_t nbyte); } 4 STD POSIX { ssize_t write(int fd, const void *buf, size_t nbyte); } 5 STD POSIX { int open(char *path, int flags, int mode); } 6 STD POSIX { int close(int fd); } etc... It is the leftmost column that tells us the number to place in EAX. The rightmost column tells us what parameters to push. They are pushed from right to left. For example, to open a file, we need to push the mode first, then flags, then the address at which the path is stored. Return Values A system call would not be useful most of the time if it did not return some kind of a value: The file descriptor of an open file, the number of bytes read to a buffer, the system time, etc. Additionally, the system needs to inform us if an error occurs: A file does not exist, system resources are exhausted, we passed an invalid parameter, etc. Man Pages The traditional place to look for information about various system calls under Unix systems are the man pages. FreeBSD describes its system calls in section 2, sometimes in section 3. For example, open2 says:
If successful, open() returns a non-negative integer, termed a file descriptor. It returns -1 on failure, and sets errno to indicate the error.
The assembly language programmer new to Unix and FreeBSD will immediately ask the puzzling question: Where is errno and how do I get to it? The information presented in the man pages applies to C programs. The assembly language programmer needs additional information. Where Are the Return Values? Unfortunately, it depends... For most system calls it is in EAX, but not for all. A good rule of thumb, when working with a system call for the first time, is to look for the return value in EAX. If it is not there, you need further research. I am aware of one system call that returns the value in EDX: SYS_fork. All others I have worked with use EAX. But I have not worked with them all yet. If you cannot find the answer here or anywhere else, study libc source code and see how it interfaces with the kernel. Where Is <varname>errno</varname>? Actually, nowhere... errno is part of the C language, not the Unix kernel. When accessing kernel services directly, the error code is returned in EAX, the same register the proper return value generally ends up in. This makes perfect sense. If there is no error, there is no error code. If there is an error, there is no return value. One register can contain either. Determining an Error Occurred When using the standard FreeBSD calling convention, the carry flag is cleared upon success, set upon failure. When using the Linux emulation mode, the signed value in EAX is non-negative upon success, and contains the return value. In case of an error, the value is negative, i.e., -errno. Creating Portable Code Portability is generally not one of the strengths of assembly language. Yet, writing assembly language programs for different platforms is possible, especially with nasm. I have written assembly language libraries that can be assembled for such different operating systems as Windows and FreeBSD. It is all the more possible when you want your code to run on two platforms which, while different, are based on similar architectures. For example, FreeBSD is Unix, Linux is Unix-like. I only mentioned three differences between them (from an assembly language programmer's perspective): The calling convention, the function numbers, and the way of returning values. Dealing with Function Numbers In many cases the function numbers are the same. However, even when they are not, the problem is easy to deal with: Instead of using numbers in your code, use constants which you have declared differently depending on the target architecture: %ifdef LINUX %define SYS_execve 11 %else %define SYS_execve 59 %endif Dealing with Conventions Both, the calling convention, and the return value (the errno problem) can be resolved with macros: %ifdef LINUX %macro system 0 call kernel %endmacro align 4 kernel: push ebx push ecx push edx push esi push edi push ebp mov ebx, [esp+32] mov ecx, [esp+36] mov edx, [esp+40] mov esi, [esp+44] mov ebp, [esp+48] int 80h pop ebp pop edi pop esi pop edx pop ecx pop ebx or eax, eax js .errno clc ret .errno: neg eax stc ret %else %macro system 0 int 80h %endmacro %endif Dealing with Other Portability Issues The above solutions can handle most cases of writing code portable between FreeBSD and Linux. Nevertheless, with some kernel services the differences are deeper. In that case, you need to write two different handlers for those particular system calls, and use conditional assembly. Luckily, most of your code does something other than calling the kernel, so usually you will only need a few such conditional sections in your code. Using a Library You can avoid portability issues in your main code altogether by writing a library of system calls. Create a separate library for FreeBSD, a different one for Linux, and yet other libraries for more operating systems. In your library, write a separate function (or procedure, if you prefer the traditional assembly language terminology) for each system call. Use the C calling convention of passing parameters. But still use EAX to pass the call number in. In that case, your FreeBSD library can be very simple, as many seemingly different functions can be just labels to the same code: sys.open: sys.close: [etc...] int 80h ret Your Linux library will require more different functions. But even here you can group system calls using the same number of parameters: sys.exit: sys.close: [etc... one-parameter functions] push ebx mov ebx, [esp+12] int 80h pop ebx jmp sys.return ... sys.return: or eax, eax js sys.err clc ret sys.err: neg eax stc ret The library approach may seem inconvenient at first because it requires you to produce a separate file your code depends on. But it has many advantages: For one, you only need to write it once and can use it for all your programs. You can even let other assembly language programmers use it, or perhaps use one written by someone else. But perhaps the greatest advantage of the library is that your code can be ported to other systems, even by other programmers, by simply writing a new library without any changes to your code. If you do not like the idea of having a library, you can at least place all your system calls in a separate assembly language file and link it with your main program. Here, again, all porters have to do is create a new object file to link with your main program. Using an Include File If you are releasing your software as (or with) source code, you can use macros and place them in a separate file, which you include in your code. Porters of your software will simply write a new include file. No library or external object file is necessary, yet your code is portable without any need to edit the code. This is the approach we will use throughout this chapter. We will name our include file system.inc, and add to it whenever we deal with a new system call. We can start our system.inc by declaring the standard file descriptors: %define stdin 0 %define stdout 1 %define stderr 2 Next, we create a symbolic name for each system call: %define SYS_nosys 0 %define SYS_exit 1 %define SYS_fork 2 %define SYS_read 3 %define SYS_write 4 ; [etc...] We add a short, non-global procedure with a long name, so we do not accidentally reuse the name in our code: section .text align 4 access.the.bsd.kernel: int 80h ret We create a macro which takes one argument, the syscall number: %macro system 1 mov eax, %1 call access.the.bsd.kernel %endmacro Finally, we create macros for each syscall. These macros take no arguments. %macro sys.exit 0 system SYS_exit %endmacro %macro sys.fork 0 system SYS_fork %endmacro %macro sys.read 0 system SYS_read %endmacro %macro sys.write 0 system SYS_write %endmacro ; [etc...] Go ahead, enter it into your editor and save it as system.inc. We will add more to it as we discuss more syscalls. Our First Program We are now ready for our first program, the mandatory Hello, World! 1: %include 'system.inc' 2: 3: section .data 4: hello db 'Hello, World!', 0Ah 5: hbytes equ $-hello 6: 7: section .text 8: global _start 9: _start: 10: push dword hbytes 11: push dword hello 12: push dword stdout 13: sys.write 14: 15: push dword 0 16: sys.exit Here is what it does: Line 1 includes the defines, the macros, and the code from system.inc. Lines 3-5 are the data: Line 3 starts the data section/segment. Line 4 contains the string "Hello, World!" followed by a new line (0Ah). Line 5 creates a constant that contains the length of the string from line 4 in bytes. Lines 7-16 contain the code. Note that FreeBSD uses the elf file format for its executables, which requires every program to start at the point labeled _start (or, more precisely, the linker expects that). This label has to be global. Lines 10-13 ask the system to write hbytes bytes of the hello string to stdout. Lines 15-16 ask the system to end the program with the return value of 0. The SYS_exit syscall never returns, so the code ends there. If you have come to Unix from MS DOS assembly language background, you may be used to writing directly to the video hardware. You will never have to worry about this in FreeBSD, or any other flavor of Unix. As far as you are concerned, you are writing to a file known as stdout. This can be the video screen, or a telnet terminal, or an actual file, or even the input of another program. Which one it is, is for the system to figure out. Assembling the Code Type the code (except the line numbers) in an editor, and save it in a file named hello.asm. You need nasm to assemble it. Installing <application>nasm</application> If you do not have nasm, type: - -&prompt.user; su +&prompt.user; su Password:your root password &prompt.root; cd /usr/ports/devel/nasm &prompt.root; make install &prompt.root; exit -&prompt.user; - +&prompt.user; You may type make install clean instead of just make install if you do not want to keep nasm source code. Either way, FreeBSD will automatically download nasm from the Internet, compile it, and install it on your system. If your system is not FreeBSD, you need to get nasm from its home page. You can still use it to assemble FreeBSD code. Now you can assemble, link, and run the code: - -&prompt.user; nasm -f elf hello.asm +&prompt.user; nasm -f elf hello.asm &prompt.user; ld -s -o hello hello.o &prompt.user; ./hello Hello, World! -&prompt.user; - +&prompt.user; Writing Unix Filters A common type of Unix application is a filter—a program that reads data from the stdin, processes it somehow, then writes the result to stdout. In this chapter, we shall develop a simple filter, and learn how to read from stdin and write to stdout. This filter will convert each byte of its input into a hexadecimal number followed by a blank space. %include 'system.inc' section .data hex db '0123456789ABCDEF' buffer db 0, 0, ' ' section .text global _start _start: ; read a byte from stdin push dword 1 push dword buffer push dword stdin sys.read add esp, byte 12 or eax, eax je .done ; convert it to hex movzx eax, byte [buffer] mov edx, eax shr dl, 4 mov dl, [hex+edx] mov [buffer], dl and al, 0Fh mov al, [hex+eax] mov [buffer+1], al ; print it push dword 3 push dword buffer push dword stdout sys.write add esp, byte 12 jmp short _start .done: push dword 0 sys.exit In the data section we create an array called hex. It contains the 16 hexadecimal digits in ascending order. The array is followed by a buffer which we will use for both input and output. The first two bytes of the buffer are initially set to 0. This is where we will write the two hexadecimal digits (the first byte also is where we will read the input). The third byte is a space. The code section consists of four parts: Reading the byte, converting it to a hexadecimal number, writing the result, and eventually exiting the program. To read the byte, we ask the system to read one byte from stdin, and store it in the first byte of the buffer. The system returns the number of bytes read in EAX. This will be 1 while data is coming, or 0, when no more input data is available. Therefore, we check the value of EAX. If it is 0, we jump to .done, otherwise we continue. For simplicity sake, we are ignoring the possibility of an error condition at this time. The hexadecimal conversion reads the byte from the buffer into EAX, or actually just AL, while clearing the remaining bits of EAX to zeros. We also copy the byte to EDX because we need to convert the upper four bits (nibble) separately from the lower four bits. We store the result in the first two bytes of the buffer. Next, we ask the system to write the three bytes of the buffer, i.e., the two hexadecimal digits and the blank space, to stdout. We then jump back to the beginning of the program and process the next byte. Once there is no more input left, we ask the system to exit our program, returning a zero, which is the traditional value meaning the program was successful. Go ahead, and save the code in a file named hex.asm, then type the following (the ^D means press the control key and type D while holding the control key down): - -&prompt.user; nasm -f elf hex.asm +&prompt.user; nasm -f elf hex.asm &prompt.user; ld -s -o hex hex.o &prompt.user; ./hex Hello, World! 48 65 6C 6C 6F 2C 20 57 6F 72 6C 64 21 0A Here I come! -48 65 72 65 20 49 20 63 6F 6D 65 21 0A ^D &prompt.user; - +48 65 72 65 20 49 20 63 6F 6D 65 21 0A ^D &prompt.user; If you are migrating to Unix from MS DOS, you may be wondering why each line ends with 0A instead of 0D 0A. This is because Unix does not use the cr/lf convention, but a "new line" convention, which is 0A in hexadecimal. Can we improve this? Well, for one, it is a bit confusing because once we have converted a line of text, our input no longer starts at the begining of the line. We can modify it to print a new line instead of a space after each 0A: %include 'system.inc' section .data hex db '0123456789ABCDEF' buffer db 0, 0, ' ' section .text global _start _start: mov cl, ' ' .loop: ; read a byte from stdin push dword 1 push dword buffer push dword stdin sys.read add esp, byte 12 or eax, eax je .done ; convert it to hex movzx eax, byte [buffer] mov [buffer+2], cl cmp al, 0Ah jne .hex mov [buffer+2], al .hex: mov edx, eax shr dl, 4 mov dl, [hex+edx] mov [buffer], dl and al, 0Fh mov al, [hex+eax] mov [buffer+1], al ; print it push dword 3 push dword buffer push dword stdout sys.write add esp, byte 12 jmp short .loop .done: push dword 0 sys.exit We have stored the space in the CL register. We can do this safely because, unlike Microsoft Windows, Unix system calls do not modify the value of any register they do not use to return a value in. That means we only need to set CL once. We have, therefore, added a new label .loop and jump to it for the next byte instead of jumping at _start. We have also added the .hex label so we can either have a blank space or a new line as the third byte of the buffer. Once you have changed hex.asm to reflect these changes, type: - -&prompt.user; nasm -f elf hex.asm +&prompt.user; nasm -f elf hex.asm &prompt.user; ld -s -o hex hex.o &prompt.user; ./hex Hello, World! 48 65 6C 6C 6F 2C 20 57 6F 72 6C 64 21 0A Here I come! 48 65 72 65 20 49 20 63 6F 6D 65 21 0A -^D &prompt.user; - +^D &prompt.user; That looks better. But this code is quite inefficient! We are making a system call for every single byte twice (once to read it, another time to write the output). Buffered Input and Output We can improve the efficiency of our code by buffering our input and output. We create an input buffer and read a whole sequence of bytes at one time. Then we fetch them one by one from the buffer. We also create an output buffer. We store our output in it until it is full. At that time we ask the kernel to write the contents of the buffer to stdout. The program ends when there is no more input. But we still need to ask the kernel to write the contents of our output buffer to stdout one last time, otherwise some of our output would make it to the output buffer, but never be sent out. Do not forget that, or you will be wondering why some of your output is missing. %include 'system.inc' %define BUFSIZE 2048 section .data hex db '0123456789ABCDEF' section .bss ibuffer resb BUFSIZE obuffer resb BUFSIZE section .text global _start _start: sub eax, eax sub ebx, ebx sub ecx, ecx mov edi, obuffer .loop: ; read a byte from stdin call getchar ; convert it to hex mov dl, al shr al, 4 mov al, [hex+eax] call putchar mov al, dl and al, 0Fh mov al, [hex+eax] call putchar mov al, ' ' cmp dl, 0Ah jne .put mov al, dl .put: call putchar jmp short .loop align 4 getchar: or ebx, ebx jne .fetch call read .fetch: lodsb dec ebx ret read: push dword BUFSIZE mov esi, ibuffer push esi push dword stdin sys.read add esp, byte 12 mov ebx, eax or eax, eax je .done sub eax, eax ret align 4 .done: call write ; flush output buffer push dword 0 sys.exit align 4 putchar: stosb inc ecx cmp ecx, BUFSIZE je write ret align 4 write: sub edi, ecx ; start of buffer push ecx push edi push dword stdout sys.write add esp, byte 12 sub eax, eax sub ecx, ecx ; buffer is empty now ret We now have a third section in the source code, named .bss. This section is not included in our executable file, and, therefore, cannot be initialized. We use resb instead of db. It simply reserves the requested size of uninitialized memory for our use. We take advantage of the fact that the system does not modify the registers: We use registers for what, otherwise, would have to be global variables stored in the .data section. This is also why the Unix convention of passing parameters to system calls on the stack is superior to the Microsoft convention of passing them in the registers: We can keep the registers for our own use. We use EDI and ESI as pointers to the next byte to be read from or written to. We use EBX and ECX to keep count of the number of bytes in the two buffers, so we know when to dump the output to, or read more input from, the system. Let us see how it works now: - -&prompt.user; nasm -f elf hex.asm +&prompt.user; nasm -f elf hex.asm &prompt.user; ld -s -o hex hex.o &prompt.user; ./hex Hello, World! Here I come! 48 65 6C 6C 6F 2C 20 57 6F 72 6C 64 21 0A 48 65 72 65 20 49 20 63 6F 6D 65 21 0A -^D &prompt.user; - +^D &prompt.user; Not what you expected? The program did not print the output until we pressed ^D. That is easy to fix by inserting three lines of code to write the output every time we have converted a new line to 0A. I have marked the three lines with > (do not copy the > in your hex.asm). %include 'system.inc' %define BUFSIZE 2048 section .data hex db '0123456789ABCDEF' section .bss ibuffer resb BUFSIZE obuffer resb BUFSIZE section .text global _start _start: sub eax, eax sub ebx, ebx sub ecx, ecx mov edi, obuffer .loop: ; read a byte from stdin call getchar ; convert it to hex mov dl, al shr al, 4 mov al, [hex+eax] call putchar mov al, dl and al, 0Fh mov al, [hex+eax] call putchar mov al, ' ' cmp dl, 0Ah jne .put mov al, dl .put: call putchar > cmp al, 0Ah > jne .loop > call write jmp short .loop align 4 getchar: or ebx, ebx jne .fetch call read .fetch: lodsb dec ebx ret read: push dword BUFSIZE mov esi, ibuffer push esi push dword stdin sys.read add esp, byte 12 mov ebx, eax or eax, eax je .done sub eax, eax ret align 4 .done: call write ; flush output buffer push dword 0 sys.exit align 4 putchar: stosb inc ecx cmp ecx, BUFSIZE je write ret align 4 write: sub edi, ecx ; start of buffer push ecx push edi push dword stdout sys.write add esp, byte 12 sub eax, eax sub ecx, ecx ; buffer is empty now ret Now, let us see how it works: - -&prompt.user; nasm -f elf hex.asm +&prompt.user; nasm -f elf hex.asm &prompt.user; ld -s -o hex hex.o &prompt.user; ./hex Hello, World! 48 65 6C 6C 6F 2C 20 57 6F 72 6C 64 21 0A Here I come! 48 65 72 65 20 49 20 63 6F 6D 65 21 0A -^D &prompt.user; - +^D &prompt.user; Not bad for a 644-byte executable, is it! This approach to buffered input/output still contains a hidden danger. I will discuss—and fix—it later, when I talk about the dark side of buffering. How to Unread a Character This may be a somewhat advanced topic, mostly of interest to programmers familiar with the theory of compilers. If you wish, you may skip to the next section, and perhaps read this later. While our sample program does not require it, more sophisticated filters often need to look ahead. In other words, they may need to see what the next character is (or even several characters). If the next character is of a certain value, it is part of the token currently being processed. Otherwise, it is not. For example, you may be parsing the input stream for a textual string (e.g., when implementing a language compiler): If a character is followed by another character, or perhaps a digit, it is part of the token you are processing. If it is followed by white space, or some other value, then it is not part of the current token. This presents an interesting problem: How to return the next character back to the input stream, so it can be read again later? One possible solution is to store it in a character variable, then set a flag. We can modify getchar to check the flag, and if it is set, fetch the byte from that variable instead of the input buffer, and reset the flag. But, of course, that slows us down. The C language has an ungetc() function, just for that purpose. Is there a quick way to implement it in our code? I would like you to scroll back up and take a look at the getchar procedure and see if you can find a nice and fast solution before reading the next paragraph. Then come back here and see my own solution. The key to returning a character back to the stream is in how we are getting the characters to start with: First we check if the buffer is empty by testing the value of EBX. If it is zero, we call the read procedure. If we do have a character available, we use lodsb, then decrease the value of EBX. The lodsb instruction is effectively identical to: mov al, [esi] inc esi The byte we have fetched remains in the buffer until the next time read is called. We do not know when that happens, but we do know it will not happen until the next call to getchar. Hence, to "return" the last-read byte back to the stream, all we have to do is decrease the value of ESI and increase the value of EBX: ungetc: dec esi inc ebx ret But, be careful! We are perfectly safe doing this if our look-ahead is at most one character at a time. If we are examining more than one upcoming character and call ungetc several times in a row, it will work most of the time, but not all the time (and will be tough to debug). Why? Because as long as getchar does not have to call read, all of the pre-read bytes are still in the buffer, and our ungetc works without a glitch. But the moment getchar calls read, the contents of the buffer change. We can always rely on ungetc working properly on the last character we have read with getchar, but not on anything we have read before that. If your program reads more than one byte ahead, you have at least two choices: If possible, modify the program so it only reads one byte ahead. This is the simplest solution. If that option is not available, first of all determine the maximum number of characters your program needs to return to the input stream at one time. Increase that number slightly, just to be sure, preferably to a multiple of 16—so it aligns nicely. Then modify the .bss section of your code, and create a small "spare" buffer right before your input buffer, something like this: section .bss resb 16 ; or whatever the value you came up with ibuffer resb BUFSIZE obuffer resb BUFSIZE You also need to modify your ungetc to pass the value of the byte to unget in AL: ungetc: dec esi inc ebx mov [esi], al ret With this modification, you can call ungetc up to 17 times in a row safely (the first call will still be within the buffer, the remaining 16 may be either within the buffer or within the "spare"). Command Line Arguments Our hex program will be more useful if it can read the names of an input and output file from its command line, i.e., if it can process the command line arguments. But... Where are they? Before a Unix system starts a program, it pushes some data on the stack, then jumps at the _start label of the program. Yes, I said jumps, not calls. That means the data can be accessed by reading [esp+offset], or by simply popping it. The value at the top of the stack contains the number of command line arguments. It is traditionally called argc, for "argument count." Command line arguments follow next, all argc of them. These are typically referred to as argv, for "argument value(s)." That is, we get argv[0], argv[1], ..., argv[argc-1]. These are not the actual arguments, but pointers to arguments, i.e., memory addresses of the actual arguments. The arguments themselves are NUL-terminated character strings. The argv list is followed by a NULL pointer, which is simply a 0. There is more, but this is enough for our purposes right now. If you have come from the MS DOS programming environment, the main difference is that each argument is in a separate string. The second difference is that there is no practical limit on how many arguments there can be. Armed with this knowledge, we are almost ready for the next version of hex.asm. First, however, we need to add a few lines to system.inc: First, we need to add two new entries to our list of system call numbers: %define SYS_open 5 %define SYS_close 6 Then we add two new macros at the end of the file: %macro sys.open 0 system SYS_open %endmacro %macro sys.close 0 system SYS_close %endmacro Here, then, is our modified source code: %include 'system.inc' %define BUFSIZE 2048 section .data fd.in dd stdin fd.out dd stdout hex db '0123456789ABCDEF' section .bss ibuffer resb BUFSIZE obuffer resb BUFSIZE section .text align 4 err: push dword 1 ; return failure sys.exit align 4 global _start _start: add esp, byte 8 ; discard argc and argv[0] pop ecx jecxz .init ; no more arguments ; ECX contains the path to input file push dword 0 ; O_RDONLY push ecx sys.open jc err ; open failed add esp, byte 8 mov [fd.in], eax pop ecx jecxz .init ; no more arguments ; ECX contains the path to output file push dword 420 ; file mode (644 octal) push dword 0200h | 0400h | 01h ; O_CREAT | O_TRUNC | O_WRONLY push ecx sys.open jc err add esp, byte 12 mov [fd.out], eax .init: sub eax, eax sub ebx, ebx sub ecx, ecx mov edi, obuffer .loop: ; read a byte from input file or stdin call getchar ; convert it to hex mov dl, al shr al, 4 mov al, [hex+eax] call putchar mov al, dl and al, 0Fh mov al, [hex+eax] call putchar mov al, ' ' cmp dl, 0Ah jne .put mov al, dl .put: call putchar cmp al, dl jne .loop call write jmp short .loop align 4 getchar: or ebx, ebx jne .fetch call read .fetch: lodsb dec ebx ret read: push dword BUFSIZE mov esi, ibuffer push esi push dword [fd.in] sys.read add esp, byte 12 mov ebx, eax or eax, eax je .done sub eax, eax ret align 4 .done: call write ; flush output buffer ; close files push dword [fd.in] sys.close push dword [fd.out] sys.close ; return success push dword 0 sys.exit align 4 putchar: stosb inc ecx cmp ecx, BUFSIZE je write ret align 4 write: sub edi, ecx ; start of buffer push ecx push edi push dword [fd.out] sys.write add esp, byte 12 sub eax, eax sub ecx, ecx ; buffer is empty now ret In our .data section we now have two new variables, fd.in and fd.out. We store the input and output file descriptors here. In the .text section we have replaced the references to stdin and stdout with [fd.in] and [fd.out]. The .text section now starts with a simple error handler, which does nothing but exit the program with a return value of 1. The error handler is before _start so we are within a short distance from where the errors occur. Naturally, the program execution still begins at _start. First, we remove argc and argv[0] from the stack: They are of no interest to us (in this program, that is). We pop argv[1] to ECX. This register is particularly suited for pointers, as we can handle NULL pointers with jecxz. If argv[1] is not NULL, we try to open the file named in the first argument. Otherwise, we continue the program as before: Reading from stdin, writing to stdout. If we fail to open the input file (e.g., it does not exist), we jump to the error handler and quit. If all went well, we now check for the second argument. If it is there, we open the output file. Otherwise, we send the output to stdout. If we fail to open the output file (e.g., it exists and we do not have the write permission), we, again, jump to the error handler. The rest of the code is the same as before, except we close the input and output files before exiting, and, as mentioned, we use [fd.in] and [fd.out]. Our executable is now a whopping 768 bytes long. Can we still improve it? Of course! Every program can be improved. Here are a few ideas of what we could do: Have our error handler print a message to stderr. Add error handlers to the read and write functions. Close stdin when we open an input file, stdout when we open an output file. Add command line switches, such as -i and -o, so we can list the input and output files in any order, or perhaps read from stdin and write to a file. Print a usage message if command line arguments are incorrect. I shall leave these enhancements as an exercise to the reader: You already know everything you need to know to implement them. Unix Environment An important Unix concept is the environment, which is defined by environment variables. Some are set by the system, others by you, yet others by the shell, or any program that loads another program. How to Find Environment Variables I said earlier that when a program starts executing, the stack contains argc followed by the NULL-terminated argv array, followed by something else. The "something else" is the environment, or, to be more precise, a NULL-terminated array of pointers to environment variables. This is often referred to as env. The structure of env is the same as that of argv, a list of memory addresses followed by a NULL (0). In this case, there is no "envc"—we figure out where the array ends by searching for the final NULL. The variables usually come in the name=value format, but sometimes the =value part may be missing. We need to account for that possibility. webvars I could just show you some code that prints the environment the same way the Unix env command does. But I thought it would be more interesting to write a simple assembly language CGI utility. CGI: A Quick Overview I have a detailed CGI tutorial on my web site, but here is a very quick overview of CGI: The web server communicates with the CGI program by setting environment variables. The CGI program sends its output to stdout. The web server reads it from there. It must start with an HTTP header followed by two blank lines. It then prints the HTML code, or whatever other type of data it is producing. While certain environment variables use standard names, others vary, depending on the web server. That makes webvars quite a useful diagnostic tool. The Code Our webvars program, then, must send out the HTTP header followed by some HTML mark-up. It then must read the environment variables one by one and send them out as part of the HTML page. The code follows. I placed comments and explanations right inside the code: ;;;;;;; webvars.asm ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; ; Copyright (c) 2000 G. Adam Stanislav ; All rights reserved. ; ; Redistribution and use in source and binary forms, with or without ; modification, are permitted provided that the following conditions ; are met: ; 1. Redistributions of source code must retain the above copyright ; notice, this list of conditions and the following disclaimer. ; 2. Redistributions in binary form must reproduce the above copyright ; notice, this list of conditions and the following disclaimer in the ; documentation and/or other materials provided with the distribution. ; ; THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND ; ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE ; IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE ; FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS ; OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) ; HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT ; LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY ; OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF ; SUCH DAMAGE. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; ; Version 1.0 ; ; Started: 8-Dec-2000 ; Updated: 8-Dec-2000 ; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; %include 'system.inc' section .data http db 'Content-type: text/html', 0Ah, 0Ah db '<?xml version="1.0" encoding="UTF-8"?>', 0Ah db '<!DOCTYPE html PUBLIC "-//W3C/DTD XHTML Strict//EN" ' db '"DTD/xhtml1-strict.dtd">', 0Ah db '<html xmlns="http://www.w3.org/1999/xhtml" ' db 'xml.lang="en" lang="en">', 0Ah db '<head>', 0Ah db '<title>Web Environment</title>', 0Ah db '<meta name="author" content="G. Adam Stanislav" />', 0Ah db '</head>', 0Ah, 0Ah db '<body bgcolor="#ffffff" text="#000000" link="#0000ff" ' db 'vlink="#840084" alink="#0000ff">', 0Ah db '<div class="webvars">', 0Ah db '<h1>Web Environment</h1>', 0Ah db '<p>The following <b>environment variables</b> are defined ' db 'on this web server:</p>', 0Ah, 0Ah db '<table align="center" width="80" border="0" cellpadding="10" ' db 'cellspacing="0" class="webvars">', 0Ah httplen equ $-http left db '<tr>', 0Ah db '<td class="name"><tt>' leftlen equ $-left middle db '</tt></td>', 0Ah db '<td class="value"><tt><b>' midlen equ $-middle undef db '<i>(undefined)</i>' undeflen equ $-undef right db '</b></tt></td>', 0Ah db '</tr>', 0Ah rightlen equ $-right wrap db '</table>', 0Ah db '</div>', 0Ah db '</body>', 0Ah db '</html>', 0Ah, 0Ah wraplen equ $-wrap section .text global _start _start: ; First, send out all the http and xhtml stuff that is ; needed before we start showing the environment push dword httplen push dword http push dword stdout sys.write ; Now find how far on the stack the environment pointers ; are. We have 12 bytes we have pushed before "argc" mov eax, [esp+12] ; We need to remove the following from the stack: ; ; The 12 bytes we pushed for sys.write ; The 4 bytes of argc ; The EAX*4 bytes of argv ; The 4 bytes of the NULL after argv ; ; Total: ; 20 + eax * 4 ; ; Because stack grows down, we need to ADD that many bytes ; to ESP. lea esp, [esp+20+eax*4] cld ; This should already be the case, but let's be sure. ; Loop through the environment, printing it out .loop: pop edi or edi, edi ; Done yet? je near .wrap ; Print the left part of HTML push dword leftlen push dword left push dword stdout sys.write ; It may be tempting to search for the '=' in the env string next. ; But it is possible there is no '=', so we search for the ; terminating NUL first. mov esi, edi ; Save start of string sub ecx, ecx not ecx ; ECX = FFFFFFFF sub eax, eax repne scasb not ecx ; ECX = string length + 1 mov ebx, ecx ; Save it in EBX ; Now is the time to find '=' mov edi, esi ; Start of string mov al, '=' repne scasb not ecx add ecx, ebx ; Length of name push ecx push esi push dword stdout sys.write ; Print the middle part of HTML table code push dword midlen push dword middle push dword stdout sys.write ; Find the length of the value not ecx lea ebx, [ebx+ecx-1] ; Print "undefined" if 0 or ebx, ebx jne .value mov ebx, undeflen mov edi, undef .value: push ebx push edi push dword stdout sys.write ; Print the right part of the table row push dword rightlen push dword right push dword stdout sys.write ; Get rid of the 60 bytes we have pushed add esp, byte 60 ; Get the next variable jmp .loop .wrap: ; Print the rest of HTML push dword wraplen push dword wrap push dword stdout sys.write ; Return success push dword 0 sys.exit This code produces a 1,396-byte executable. Most of it is data, i.e., the HTML mark-up we need to send out. Assemble and link it as usual: &prompt.user; nasm -f elf webvars.asm -&prompt.user; ld -s -o webvars webvars.o - +&prompt.user; ld -s -o webvars webvars.o To use it, you need to upload webvars to your web server. Depending on how your web server is set up, you may have to store it in a special cgi-bin directory, or perhaps rename it with a .cgi extension. Then you need to use your browser to view its output. To see its output on my web server, please go to http://www.int80h.org/webvars/. If curious about the additional environment variables present in a password protected web directory, go to http://www.int80h.org/private/, using the name asm and password programmer. Working with Files We have already done some basic file work: We know how to open and close them, how to read and write them using buffers. But Unix offers much more functionality when it comes to files. We will examine some of it in this section, and end up with a nice file conversion utility. Indeed, let us start at the end, that is, with the file conversion utility. It always makes programming easier when we know from the start what the end product is supposed to do. One of the first programs I wrote for Unix was tuc, a text-to-Unix file converter. It converts a text file from other operating systems to a Unix text file. In other words, it changes from different kind of line endings to the newline convention of Unix. It saves the output in a different file. Optionally, it converts a Unix text file to a DOS text file. I have used tuc extensively, but always only to convert from some other OS to Unix, never the other way. I have always wished it would just overwrite the file instead of me having to send the output to a different file. Most of the time, I end up using it like this: - -&prompt.user; tuc myfile tempfile -&prompt.user; mv tempfile myfile - +&prompt.user; tuc myfile tempfile +&prompt.user; mv tempfile myfile It would be nice to have a ftuc, i.e., fast tuc, and use it like this: - -&prompt.user; ftuc myfile - +&prompt.user; ftuc myfile In this chapter, then, we will write ftuc in assembly language (the original tuc is in C), and study various file-oriented kernel services in the process. At first sight, such a file conversion is very simple: All you have to do is strip the carriage returns, right? If you answered yes, think again: That approach will work most of the time (at least with MS DOS text files), but will fail occasionally. The problem is that not all non-Unix text files end their line with the carriage return / line feed sequence. Some use carriage returns without line feeds. Others combine several blank lines into a single carriage return followed by several line feeds. And so on. A text file converter, then, must be able to handle any possible line endings: carriage return / line feed carriage return line feed / carriage return line feed It should also handle files that use some kind of a combination of the above (e.g., carriage return followed by several line feeds). Finite State Machine The problem is easily solved by the use of a technique called finite state machine, originally developed by the designers of digital electronic circuits. A finite state machine is a digital circuit whose output is dependent not only on its input but on its previous input, i.e., on its state. The microprocessor is an example of a finite state machine: Our assembly language code is assembled to machine language in which some assembly language code produces a single byte of machine language, while others produce several bytes. As the microprocessor fetches the bytes from the memory one by one, some of them simply change its state rather than produce some output. When all the bytes of the op code are fetched, the microprocessor produces some output, or changes the value of a register, etc. Because of that, all software is essentially a sequence of state instructions for the microprocessor. Nevertheless, the concept of finite state machine is useful in software design as well. Our text file converter can be designed as a finite state machine with three possible states. We could call them states 0-2, but it will make our life easier if we give them symbolic names: ordinary cr lf Our program will start in the ordinary state. During this state, the program action depends on its input as follows: If the input is anything other than a carriage return or line feed, the input is simply passed on to the output. The state remains unchanged. If the input is a carriage return, the state is changed to cr. The input is then discarded, i.e., no output is made. If the input is a line feed, the state is changed to lf. The input is then discarded. Whenever we are in the cr state, it is because the last input was a carriage return, which was unprocessed. What our software does in this state again depends on the current input: If the input is anything other than a carriage return or line feed, output a line feed, then output the input, then change the state to ordinary. If the input is a carriage return, we have received two (or more) carriage returns in a row. We discard the input, we output a line feed, and leave the state unchanged. If the input is a line feed, we output the line feed and change the state to ordinary. Note that this is not the same as the first case above – if we tried to combine them, we would be outputting two line feeds instead of one. Finally, we are in the lf state after we have received a line feed that was not preceded by a carriage return. This will happen when our file already is in Unix format, or whenever several lines in a row are expressed by a single carriage return followed by several line feeds, or when line ends with a line feed / carriage return sequence. Here is how we need to handle our input in this state: If the input is anything other than a carriage return or line feed, we output a line feed, then output the input, then change the state to ordinary. This is exactly the same action as in the cr state upon receiving the same kind of input. If the input is a carriage return, we discard the input, we output a line feed, then change the state to ordinary. If the input is a line feed, we output the line feed, and leave the state unchanged. The Final State The above finite state machine works for the entire file, but leaves the possibility that the final line end will be ignored. That will happen whenever the file ends with a single carriage return or a single line feed. I did not think of it when I wrote tuc, just to discover that occasionally it strips the last line ending. This problem is easily fixed by checking the state after the entire file was processed. If the state is not ordinary, we simply need to output one last line feed. Now that we have expressed our algorithm as a finite state machine, we could easily design a dedicated digital electronic circuit (a "chip") to do the conversion for us. Of course, doing so would be considerably more expensive than writing an assembly language program. The Output Counter Because our file conversion program may be combining two characters into one, we need to use an output counter. We initialize it to 0, and increase it every time we send a character to the output. At the end of the program, the counter will tell us what size we need to set the file to. Implementing FSM in Software The hardest part of working with a finite state machine is analyzing the problem and expressing it as a finite state machine. That accomplished, the software almost writes itself. In a high-level language, such as C, there are several main approaches. One is to use a switch statement which chooses what function should be run. For example, switch (state) { default: case REGULAR: regular(inputchar); break; case CR: cr(inputchar); break; case LF: lf(inputchar); break; } Another approach is by using an array of function pointers, something like this: (output[state])(inputchar); Yet another is to have state be a function pointer, set to point at the appropriate function: (*state)(inputchar); This is the approach we will use in our program because it is very easy to do in assembly language, and very fast, too. We will simply keep the address of the right procedure in EBX, and then just issue: call ebx This is possibly faster than hardcoding the address in the code because the microprocessor does not have to fetch the address from the memory—it is already stored in one of its registers. I said possibly because with the caching modern microprocessors do, either way may be equally fast. Memory Mapped Files Because our program works on a single file, we cannot use the approach that worked for us before, i.e., to read from an input file and to write to an output file. Unix allows us to map a file, or a section of a file, into memory. To do that, we first need to open the file with the appropriate read/write flags. Then we use the mmap system call to map it into the memory. One nice thing about mmap is that it automatically works with virtual memory: We can map more of the file into the memory than we have physical memory available, yet still access it through regular memory op codes, such as mov, lods, and stos. Whatever changes we make to the memory image of the file will be written to the file by the system. We do not even have to keep the file open: As long as it stays mapped, we can read from it and write to it. The 32-bit Intel microprocessors can access up to four gigabytes of memory – physical or virtual. The FreeBSD system allows us to use up to a half of it for file mapping. For simplicity sake, in this tutorial we will only convert files that can be mapped into the memory in their entirety. There are probably not too many text files that exceed two gigabytes in size. If our program encounters one, it will simply display a message suggesting we use the original tuc instead. If you examine your copy of syscalls.master, you will find two separate syscalls named mmap. This is because of evolution of Unix: There was the traditional BSD mmap, syscall 71. That one was superceded by the POSIX mmap, syscall 197. The FreeBSD system supports both because older programs were written by using the original BSD version. But new software uses the POSIX version, which is what we will use. The syscalls.master file lists the POSIX version like this: 197 STD BSD { caddr_t mmap(caddr_t addr, size_t len, int prot, \ int flags, int fd, long pad, off_t pos); } This differs slightly from what mmap2 says. That is because mmap2 describes the C version. The difference is in the long pad argument, which is not present in the C version. However, the FreeBSD syscalls add a 32-bit pad after pushing a 64-bit argument. In this case, off_t is a 64-bit value. When we are finished working with a memory-mapped file, we unmap it with the munmap syscall: For an in-depth treatment of mmap, see W. Richard Stevens' Unix Network Programming, Volume 2, Chapter 12. Determining File Size Because we need to tell mmap how many bytes of the file to map into the memory, and because we want to map the entire file, we need to determine the size of the file. We can use the fstat syscall to get all the information about an open file that the system can give us. That includes the file size. Again, syscalls.master lists two versions of fstat, a traditional one (syscall 62), and a POSIX one (syscall 189). Naturally, we will use the POSIX version: 189 STD POSIX { int fstat(int fd, struct stat *sb); } This is a very straightforward call: We pass to it the address of a stat structure and the descriptor of an open file. It will fill out the contents of the stat structure. I do, however, have to say that I tried to declare the stat structure in the .bss section, and fstat did not like it: It set the carry flag indicating an error. After I changed the code to allocate the structure on the stack, everything was working fine. Changing the File Size Because our program may combine carriage return / line feed sequences into straight line feeds, our output may be smaller than our input. However, since we are placing our output into the same file we read the input from, we may have to change the size of the file. The ftruncate system call allows us to do just that. Despite its somewhat misleading name, the ftruncate system call can be used to both truncate the file (make it smaller) and to grow it. And yes, we will find two versions of ftruncate in syscalls.master, an older one (130), and a newer one (201). We will use the newer one: 201 STD BSD { int ftruncate(int fd, int pad, off_t length); } Please note that this one contains a int pad again. ftuc We now know everything we need to write ftuc. We start by adding some new lines in system.inc. First, we define some constants and structures, somewhere at or near the beginning of the file: ;;;;;;; open flags %define O_RDONLY 0 %define O_WRONLY 1 %define O_RDWR 2 ;;;;;;; mmap flags %define PROT_NONE 0 %define PROT_READ 1 %define PROT_WRITE 2 %define PROT_EXEC 4 ;; %define MAP_SHARED 0001h %define MAP_PRIVATE 0002h ;;;;;;; stat structure struc stat st_dev resd 1 ; = 0 st_ino resd 1 ; = 4 st_mode resw 1 ; = 8, size is 16 bits st_nlink resw 1 ; = 10, ditto st_uid resd 1 ; = 12 st_gid resd 1 ; = 16 st_rdev resd 1 ; = 20 st_atime resd 1 ; = 24 st_atimensec resd 1 ; = 28 st_mtime resd 1 ; = 32 st_mtimensec resd 1 ; = 36 st_ctime resd 1 ; = 40 st_ctimensec resd 1 ; = 44 st_size resd 2 ; = 48, size is 64 bits st_blocks resd 2 ; = 56, ditto st_blksize resd 1 ; = 64 st_flags resd 1 ; = 68 st_gen resd 1 ; = 72 st_lspare resd 1 ; = 76 st_qspare resd 4 ; = 80 endstruc We define the new syscalls: %define SYS_mmap 197 %define SYS_munmap 73 %define SYS_fstat 189 %define SYS_ftruncate 201 We add the macros for their use: %macro sys.mmap 0 system SYS_mmap %endmacro %macro sys.munmap 0 system SYS_munmap %endmacro %macro sys.ftruncate 0 system SYS_ftruncate %endmacro %macro sys.fstat 0 system SYS_fstat %endmacro And here is our code: ;;;;;;; Fast Text-to-Unix Conversion (ftuc.asm) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; ;; Started: 21-Dec-2000 ;; Updated: 22-Dec-2000 ;; ;; Copyright 2000 G. Adam Stanislav. ;; All rights reserved. ;; ;;;;;;; v.1 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; %include 'system.inc' section .data db 'Copyright 2000 G. Adam Stanislav.', 0Ah db 'All rights reserved.', 0Ah usg db 'Usage: ftuc filename', 0Ah usglen equ $-usg co db "ftuc: Can't open file.", 0Ah colen equ $-co fae db 'ftuc: File access error.', 0Ah faelen equ $-fae ftl db 'ftuc: File too long, use regular tuc instead.', 0Ah ftllen equ $-ftl mae db 'ftuc: Memory allocation error.', 0Ah maelen equ $-mae section .text align 4 memerr: push dword maelen push dword mae jmp short error align 4 toolong: push dword ftllen push dword ftl jmp short error align 4 facerr: push dword faelen push dword fae jmp short error align 4 cantopen: push dword colen push dword co jmp short error align 4 usage: push dword usglen push dword usg error: push dword stderr sys.write push dword 1 sys.exit align 4 global _start _start: pop eax ; argc pop eax ; program name pop ecx ; file to convert jecxz usage pop eax or eax, eax ; Too many arguments? jne usage ; Open the file push dword O_RDWR push ecx sys.open jc cantopen mov ebp, eax ; Save fd sub esp, byte stat_size mov ebx, esp ; Find file size push ebx push ebp ; fd sys.fstat jc facerr mov edx, [ebx + st_size + 4] ; File is too long if EDX != 0 ... or edx, edx jne near toolong mov ecx, [ebx + st_size] ; ... or if it is above 2 GB or ecx, ecx js near toolong ; Do nothing if the file is 0 bytes in size jecxz .quit ; Map the entire file in memory push edx push edx ; starting at offset 0 push edx ; pad push ebp ; fd push dword MAP_SHARED push dword PROT_READ | PROT_WRITE push ecx ; entire file size push edx ; let system decide on the address sys.mmap jc near memerr mov edi, eax mov esi, eax push ecx ; for SYS_munmap push edi ; Use EBX for state machine mov ebx, ordinary mov ah, 0Ah cld .loop: lodsb call ebx loop .loop cmp ebx, ordinary je .filesize ; Output final lf mov al, ah stosb inc edx .filesize: ; truncate file to new size push dword 0 ; high dword push edx ; low dword push eax ; pad push ebp sys.ftruncate ; close it (ebp still pushed) sys.close add esp, byte 16 sys.munmap .quit: push dword 0 sys.exit align 4 ordinary: cmp al, 0Dh je .cr cmp al, ah je .lf stosb inc edx ret align 4 .cr: mov ebx, cr ret align 4 .lf: mov ebx, lf ret align 4 cr: cmp al, 0Dh je .cr cmp al, ah je .lf xchg al, ah stosb inc edx xchg al, ah ; fall through .lf: stosb inc edx mov ebx, ordinary ret align 4 .cr: mov al, ah stosb inc edx ret align 4 lf: cmp al, ah je .lf cmp al, 0Dh je .cr xchg al, ah stosb inc edx xchg al, ah stosb inc edx mov ebx, ordinary ret align 4 .cr: mov ebx, ordinary mov al, ah ; fall through .lf: stosb inc edx ret Do not use this program on files stored on a disk formated by MS DOS or Windows. There seems to be a subtle bug in the FreeBSD code when using mmap on these drives mounted under FreeBSD: If the file is over a certain size, mmap will just fill the memory with zeros, and then copy them to the file overwriting its contents. One-Pointed Mind As a student of Zen, I like the idea of a one-pointed mind: Do one thing at a time, and do it well. This, indeed, is very much how Unix works as well. While a typical Windows application is attempting to do everything imaginable (and is, therefore, riddled with bugs), a typical Unix program does only one thing, and it does it well. The typical Unix user then essentially assembles his own applications by writing a shell script which combines the various existing programs by piping the output of one program to the input of another. When writing your own Unix software, it is generally a good idea to see what parts of the problem you need to solve can be handled by existing programs, and only write your own programs for that part of the problem that you do not have an existing solution for. CSV I will illustrate this principle with a specific real-life example I was faced with recently: I needed to extract the 11th field of each record from a database I downloaded from a web site. The database was a CSV file, i.e., a list of comma-separated values. That is quite a standard format for sharing data among people who may be using different database software. The first line of the file contains the list of various fields separated by commas. The rest of the file contains the data listed line by line, with values separated by commas. I tried awk, using the comma as a separator. But because several lines contained a quoted comma, awk was extracting the wrong field from those lines. Therefore, I needed to write my own software to extract the 11th field from the CSV file. However, going with the Unix spirit, I only needed to write a simple filter that would do the following: Remove the first line from the file; Change all unquoted commas to a different character; Remove all quotation marks. Strictly speaking, I could use sed to remove the first line from the file, but doing so in my own program was very easy, so I decided to do it and reduce the size of the pipeline. At any rate, writing a program like this took me about 20 minutes. Writing a program that extracts the 11th field from the CSV file would take a lot longer, and I could not reuse it to extract some other field from some other database. This time I decided to let it do a little more work than a typical tutorial program would: It parses its command line for options; It displays proper usage if it finds wrong arguments; It produces meaningful error messages. Here is its usage message: - -Usage: csv [-t<delim>] [-c<comma>] [-p] [-o <outfile>] [-i <infile>] - +Usage: csv [-t<delim>] [-c<comma>] [-p] [-o <outfile>] [-i <infile>] All parameters are optional, and can appear in any order. The -t parameter declares what to replace the commas with. The tab is the default here. For example, -t; will replace all unquoted commas with semicolons. I did not need the -c option, but it may come in handy in the future. It lets me declare that I want a character other than a comma replaced with something else. For example, -c@ will replace all at signs (useful if you want to split a list of email addresses to their user names and domains). The -p option preserves the first line, i.e., it does not delete it. By default, we delete the first line because in a CSV file it contains the field names rather than data. The -i and -o options let me specify the input and the output files. Defaults are stdin and stdout, so this is a regular Unix filter. I made sure that both -i filename and -ifilename are accepted. I also made sure that only one input and one output files may be specified. To get the 11th field of each record, I can now do: - -&prompt.user; csv '-t;' data.csv | awk '-F;' '{print $11}' - +&prompt.user; csv '-t;' data.csv | awk '-F;' '{print $11}' The code stores the options (except for the file descriptors) in EDX: The comma in DH, the new separator in DL, and the flag for the -p option in the highest bit of EDX, so a check for its sign will give us a quick decision what to do. Here is the code: ;;;;;;; csv.asm ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; ; Convert a comma-separated file to a something-else separated file. ; ; Started: 31-May-2001 ; Updated: 1-Jun-2001 ; ; Copyright (c) 2001 G. Adam Stanislav ; All rights reserved. ; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; %include 'system.inc' %define BUFSIZE 2048 section .data fd.in dd stdin fd.out dd stdout usg db 'Usage: csv [-t<delim>] [-c<comma>] [-p] [-o <outfile>] [-i <infile>]', 0Ah usglen equ $-usg iemsg db "csv: Can't open input file", 0Ah iemlen equ $-iemsg oemsg db "csv: Can't create output file", 0Ah oemlen equ $-oemsg section .bss ibuffer resb BUFSIZE obuffer resb BUFSIZE section .text align 4 ierr: push dword iemlen push dword iemsg push dword stderr sys.write push dword 1 ; return failure sys.exit align 4 oerr: push dword oemlen push dword oemsg push dword stderr sys.write push dword 2 sys.exit align 4 usage: push dword usglen push dword usg push dword stderr sys.write push dword 3 sys.exit align 4 global _start _start: add esp, byte 8 ; discard argc and argv[0] mov edx, (',' << 8) | 9 .arg: pop ecx or ecx, ecx je near .init ; no more arguments ; ECX contains the pointer to an argument cmp byte [ecx], '-' jne usage inc ecx mov ax, [ecx] .o: cmp al, 'o' jne .i ; Make sure we are not asked for the output file twice cmp dword [fd.out], stdout jne usage ; Find the path to output file - it is either at [ECX+1], ; i.e., -ofile -- ; or in the next argument, ; i.e., -o file inc ecx or ah, ah jne .openoutput pop ecx jecxz usage .openoutput: push dword 420 ; file mode (644 octal) push dword 0200h | 0400h | 01h ; O_CREAT | O_TRUNC | O_WRONLY push ecx sys.open jc near oerr add esp, byte 12 mov [fd.out], eax jmp short .arg .i: cmp al, 'i' jne .p ; Make sure we are not asked twice cmp dword [fd.in], stdin jne near usage ; Find the path to the input file inc ecx or ah, ah jne .openinput pop ecx or ecx, ecx je near usage .openinput: push dword 0 ; O_RDONLY push ecx sys.open jc near ierr ; open failed add esp, byte 8 mov [fd.in], eax jmp .arg .p: cmp al, 'p' jne .t or ah, ah jne near usage or edx, 1 << 31 jmp .arg .t: cmp al, 't' ; redefine output delimiter jne .c or ah, ah je near usage mov dl, ah jmp .arg .c: cmp al, 'c' jne near usage or ah, ah je near usage mov dh, ah jmp .arg align 4 .init: sub eax, eax sub ebx, ebx sub ecx, ecx mov edi, obuffer ; See if we are to preserve the first line or edx, edx js .loop .firstline: ; get rid of the first line call getchar cmp al, 0Ah jne .firstline .loop: ; read a byte from stdin call getchar ; is it a comma (or whatever the user asked for)? cmp al, dh jne .quote ; Replace the comma with a tab (or whatever the user wants) mov al, dl .put: call putchar jmp short .loop .quote: cmp al, '"' jne .put ; Print everything until you get another quote or EOL. If it ; is a quote, skip it. If it is EOL, print it. .qloop: call getchar cmp al, '"' je .loop cmp al, 0Ah je .put call putchar jmp short .qloop align 4 getchar: or ebx, ebx jne .fetch call read .fetch: lodsb dec ebx ret read: jecxz .read call write .read: push dword BUFSIZE mov esi, ibuffer push esi push dword [fd.in] sys.read add esp, byte 12 mov ebx, eax or eax, eax je .done sub eax, eax ret align 4 .done: call write ; flush output buffer ; close files push dword [fd.in] sys.close push dword [fd.out] sys.close ; return success push dword 0 sys.exit align 4 putchar: stosb inc ecx cmp ecx, BUFSIZE je write ret align 4 write: jecxz .ret ; nothing to write sub edi, ecx ; start of buffer push ecx push edi push dword [fd.out] sys.write add esp, byte 12 sub eax, eax sub ecx, ecx ; buffer is empty now .ret: ret Much of it is taken from hex.asm above. But there is one important difference: I no longer call write whenever I am outputing a line feed. Yet, the code can be used interactively. I have found a better solution for the interactive problem since I first started writing this chapter. I wanted to make sure each line is printed out separately only when needed. After all, there is no need to flush out every line when used non-interactively. The new solution I use now is to call write every time I find the input buffer empty. That way, when running in the interactive mode, the program reads one line from the user's keyboard, processes it, and sees its input buffer is empty. It flushes its output and reads the next line. The Dark Side of Buffering This change prevents a mysterious lockup in a very specific case. I refer to it as the dark side of buffering, mostly because it presents a danger that is not quite obvious. It is unlikely to happen with a program like the csv above, so let us consider yet another filter: In this case we expect our input to be raw data representing color values, such as the red, green, and blue intensities of a pixel. Our output will be the negative of our input. Such a filter would be very simple to write. Most of it would look just like all the other filters we have written so far, so I am only going to show you its inner loop: .loop: call getchar not al ; Create a negative call putchar jmp short .loop Because this filter works with raw data, it is unlikely to be used interactively. But it could be called by image manipulation software. And, unless it calls write before each call to read, chances are it will lock up. Here is what might happen: The image editor will load our filter using the C function popen(). It will read the first row of pixels from a bitmap or pixmap. It will write the first row of pixels to the pipe leading to the fd.in of our filter. Our filter will read each pixel from its input, turn it to a negative, and write it to its output buffer. Our filter will call getchar to fetch the next pixel. getchar will find an empty input buffer, so it will call read. read will call the SYS_read system call. The kernel will suspend our filter until the image editor sends more data to the pipe. The image editor will read from the other pipe, connected to the fd.out of our filter so it can set the first row of the output image before it sends us the second row of the input. The kernel suspends the image editor until it receives some output from our filter, so it can pass it on to the image editor. At this point our filter waits for the image editor to send it more data to process, while the image editor is waiting for our filter to send it the result of the processing of the first row. But the result sits in our output buffer. The filter and the image editor will continue waiting for each other forever (or, at least, until they are killed). Our software has just entered a race condition. This problem does not exist if our filter flushes its output buffer before asking the kernel for more input data. Using the <acronym>FPU</acronym> Strangely enough, most of assembly language literature does not even mention the existence of the FPU, or floating point unit, let alone discuss programming it. Yet, never does assembly language shine more than when we create highly optimized FPU code by doing things that can be done only in assembly language. Organization of the <acronym>FPU</acronym> The FPU consists of 8 80–bit floating–point registers. These are organized in a stack fashion—you can push a value on TOS (top of stack) and you can pop it. That said, the assembly language op codes are not push and pop because those are already taken. You can push a value on TOS by using fld, fild, and fbld. Several other op codes let you push many common constants—such as pi—on the TOS. Similarly, you can pop a value by using fst, fstp, fist, fistp, and fbstp. Actually, only the op codes that end with a p will literally pop the value, the rest will store it somewhere else without removing it from the TOS. We can transfer the data between the TOS and the computer memory either as a 32–bit, 64–bit, or 80–bit real, a 16–bit, 32–bit, or 64–bit integer, or an 80–bit packed decimal. The 80–bit packed decimal is a special case of binary coded decimal which is very convenient when converting between the ASCII representation of data and the internal data of the FPU. It allows us to use 18 significant digits. No matter how we represent data in the memory, the FPU always stores it in the 80–bit real format in its registers. Its internal precision is at least 19 decimal digits, so even if we choose to display results as ASCII in the full 18–digit precision, we are still showing correct results. We can perform mathematical operations on the TOS: We can calculate its sine, we can scale it (i.e., we can multiply or divide it by a power of 2), we can calculate its base–2 logarithm, and many other things. We can also multiply or divide it by, add it to, or subtract it from, any of the FPU registers (including itself). The official Intel op code for the TOS is st, and for the registers st(0)–st(7). st and st(0), then, refer to the same register. For whatever reasons, the original author of nasm has decided to use different op codes, namely st0–st7. In other words, there are no parentheses, and the TOS is always st0, never just st. The Packed Decimal Format The packed decimal format uses 10 bytes (80 bits) of memory to represent 18 digits. The number represented there is always an integer. You can use it to get decimal places by multiplying the TOS by a power of 10 first. The highest bit of the highest byte (byte 9) is the sign bit: If it is set, the number is negative, otherwise, it is positive. The rest of the bits of this byte are unused/ignored. The remaining 9 bytes store the 18 digits of the number: 2 digits per byte. The more significant digit is stored in the high nibble (4 bits), the less significant digit in the low nibble. That said, you might think that -1234567 would be stored in the memory like this (using hexadecimal notation): 80 00 00 00 00 00 01 23 45 67 Alas it is not! As with everything else of Intel make, even the packed decimal is little–endian. That means our -1234567 is stored like this: 67 45 23 01 00 00 00 00 00 80 Remember that, or you will be pulling your hair out in desperation! The book to read—if you can find it—is Richard Startz' 8087/80287/80387 for the IBM PC & Compatibles. Though it does seem to take the fact about the little–endian storage of the packed decimal for granted. I kid you not about the desperation of trying to figure out what was wrong with the filter I show below before it occurred to me I should try the little–endian order even for this type of data. Excursion to Pinhole Photography To write meaningful software, we must not only understand our programming tools, but also the field we are creating software for. Our next filter will help us whenever we want to build a pinhole camera, so, we need some background in pinhole photography before we can continue. The Camera The easiest way to describe any camera ever built is as some empty space enclosed in some lightproof material, with a small hole in the enclosure. The enclosure is usually sturdy (e.g., a box), though sometimes it is flexible (the bellows). It is quite dark inside the camera. However, the hole lets light rays in through a single point (though in some cases there may be several). These light rays form an image, a representation of whatever is outside the camera, in front of the hole. If some light sensitive material (such as film) is placed inside the camera, it can capture the image. The hole often contains a lens, or a lens assembly, often called the objective. The Pinhole But, strictly speaking, the lens is not necessary: The original cameras did not use a lens but a pinhole. Even today, pinholes are used, both as a tool to study how cameras work, and to achieve a special kind of image. The image produced by the pinhole is all equally sharp. Or blurred. There is an ideal size for a pinhole: If it is either larger or smaller, the image loses its sharpness. Focal Length This ideal pinhole diameter is a function of the square root of focal length, which is the distance of the pinhole from the film. D = PC * sqrt(FL) In here, D is the ideal diameter of the pinhole, FL is the focal length, and PC is a pinhole constant. According to Jay Bender, its value is 0.04, while Kenneth Connors has determined it to be 0.037. Others have proposed other values. Plus, this value is for the daylight only: Other types of light will require a different constant, whose value can only be determined by experimentation. The F–Number The f–number is a very useful measure of how much light reaches the film. A light meter can determine that, for example, to expose a film of specific sensitivity with f5.6 may require the exposure to last 1/1000 sec. It does not matter whether it is a 35–mm camera, or a 6x9cm camera, etc. As long as we know the f–number, we can determine the proper exposure. The f–number is easy to calculate: F = FL / D In other words, the f–number equals the focal length divided by the diameter of the pinhole. It also means a higher f–number either implies a smaller pinhole or a larger focal distance, or both. That, in turn, implies, the higher the f–number, the longer the exposure has to be. Furthermore, while pinhole diameter and focal distance are one–dimensional measurements, both, the film and the pinhole, are two–dimensional. That means that if you have measured the exposure at f–number A as t, then the exposure at f–number B is: t * (B / A)² Normalized F–Number While many modern cameras can change the diameter of their pinhole, and thus their f–number, quite smoothly and gradually, such was not always the case. To allow for different f–numbers, cameras typically contained a metal plate with several holes of different sizes drilled to them. Their sizes were chosen according to the above formula in such a way that the resultant f–number was one of standard f–numbers used on all cameras everywhere. For example, a very old Kodak Duaflex IV camera in my possession has three such holes for f–numbers 8, 11, and 16. A more recently made camera may offer f–numbers of 2.8, 4, 5.6, 8, 11, 16, 22, and 32 (as well as others). These numbers were not chosen arbitrarily: They all are powers of the square root of 2, though they may be rounded somewhat. The F–Stop A typical camera is designed in such a way that setting any of the normalized f–numbers changes the feel of the dial. It will naturally stop in that position. Because of that, these positions of the dial are called f–stops. Since the f–numbers at each stop are powers of the square root of 2, moving the dial by 1 stop will double the amount of light required for proper exposure. Moving it by 2 stops will quadruple the required exposure. Moving the dial by 3 stops will require the increase in exposure 8 times, etc. Designing the Pinhole Software We are now ready to decide what exactly we want our pinhole software to do. Processing Program Input Since its main purpose is to help us design a working pinhole camera, we will use the focal length as the input to the program. This is something we can determine without software: Proper focal length is determined by the size of the film and by the need to shoot "regular" pictures, wide angle pictures, or telephoto pictures. Most of the programs we have written so far worked with individual characters, or bytes, as their input: The hex program converted individual bytes into a hexadecimal number, the csv program either let a character through, or deleted it, or changed it to a different character, etc. One program, ftuc used the state machine to consider at most two input bytes at a time. But our pinhole program cannot just work with individual characters, it has to deal with larger syntactic units. For example, if we want the program to calculate the pinhole diameter (and other values we will discuss later) at the focal lengths of 100 mm, 150 mm, and 210 mm, we may want to enter something like this: - -100, 150, 210 - +100, 150, 210 Our program needs to consider more than a single byte of input at a time. When it sees the first 1, it must understand it is seeing the first digit of a decimal number. When it sees the 0 and the other 0, it must know it is seeing more digits of the same number. When it encounters the first comma, it must know it is no longer receiving the digits of the first number. It must be able to convert the digits of the first number into the value of 100. And the digits of the second number into the value of 150. And, of course, the digits of the third number into the numeric value of 210. We need to decide what delimiters to accept: Do the input numbers have to be separated by a comma? If so, how do we treat two numbers separated by something else? Personally, I like to keep it simple. Something either is a number, so I process it. Or it is not a number, so I discard it. I don't like the computer complaining about me typing in an extra character when it is obvious that it is an extra character. Duh! Plus, it allows me to break up the monotony of computing and type in a query instead of just a number: -What is the best pinhole diameter for the focal length of 150? - +What is the best pinhole diameter for the focal length of 150? There is no reason for the computer to spit out a number of complaints: - -Syntax error: What +Syntax error: What Syntax error: is Syntax error: the -Syntax error: best - +Syntax error: best Et cetera, et cetera, et cetera. Secondly, I like the # character to denote the start of a comment which extends to the end of the line. This does not take too much effort to code, and lets me treat input files for my software as executable scripts. In our case, we also need to decide what units the input should come in: We choose millimeters because that is how most photographers measure the focus length. Finally, we need to decide whether to allow the use of the decimal point (in which case we must also consider the fact that much of the world uses a decimal comma). In our case allowing for the decimal point/comma would offer a false sense of precision: There is little if any noticeable difference between the focus lengths of 50 and 51, so allowing the user to input something like 50.5 is not a good idea. This is my opinion, mind you, but I am the one writing this program. You can make other choices in yours, of course. Offering Options The most important thing we need to know when building a pinhole camera is the diameter of the pinhole. Since we want to shoot sharp images, we will use the above formula to calculate the pinhole diameter from focal length. As experts are offering several different values for the PC constant, we will need to have the choice. It is traditional in Unix programming to have two main ways of choosing program parameters, plus to have a default for the time the user does not make a choice. Why have two ways of choosing? One is to allow a (relatively) permanent choice that applies automatically each time the software is run without us having to tell it over and over what we want it to do. The permanent choices may be stored in a configuration file, typically found in the user's home directory. The file usually has the same name as the application but is started with a dot. Often "rc" is added to the file name. So, ours could be ~/.pinhole or ~/.pinholerc. (The ~/ means current user's home directory.) The configuration file is used mostly by programs that have many configurable parameters. Those that have only one (or a few) often use a different method: They expect to find the parameter in an environment variable. In our case, we might look at an environment variable named PINHOLE. Usually, a program uses one or the other of the above methods. Otherwise, if a configuration file said one thing, but an environment variable another, the program might get confused (or just too complicated). Because we only need to choose one such parameter, we will go with the second method and search the environment for a variable named PINHOLE. The other way allows us to make ad hoc decisions: "Though I usually want you to use 0.039, this time I want 0.03872." In other words, it allows us to override the permanent choice. This type of choice is usually done with command line parameters. Finally, a program always needs a default. The user may not make any choices. Perhaps he does not know what to choose. Perhaps he is "just browsing." Preferably, the default will be the value most users would choose anyway. That way they do not need to choose. Or, rather, they can choose the default without an additional effort. Given this system, the program may find conflicting options, and handle them this way: If it finds an ad hoc choice (e.g., command line parameter), it should accept that choice. It must ignore any permanent choice and any default. Otherwise, if it finds a permanent option (e.g., an environment variable), it should accept it, and ignore the default. Otherwise, it should use the default. We also need to decide what format our PC option should have. At first site, it seems obvious to use the PINHOLE=0.04 format for the environment variable, and -p0.04 for the command line. Allowing that is actually a security risk. The PC constant is a very small number. Naturally, we will test our software using various small values of PC. But what will happen if someone runs the program choosing a huge value? It may crash the program because we have not designed it to handle huge numbers. Or, we may spend more time on the program so it can handle huge numbers. We might do that if we were writing commercial software for computer illiterate audience. Or, we might say, "Tough! The user should know better."" Or, we just may make it impossible for the user to enter a huge number. This is the approach we will take: We will use an implied 0. prefix. In other words, if the user wants 0.04, we will expect him to type -p04, or set PINHOLE=04 in his environment. So, if he says -p9999999, we will interpret it as 0.9999999—still ridiculous but at least safer. Secondly, many users will just want to go with either Bender's constant or Connors' constant. To make it easier on them, we will interpret -b as identical to -p04, and -c as identical to -p037. The Output We need to decide what we want our software to send to the output, and in what format. Since our input allows for an unspecified number of focal length entries, it makes sense to use a traditional database–style output of showing the result of the calculation for each focal length on a separate line, while separating all values on one line by a tab character. Optionally, we should also allow the user to specify the use of the CSV format we have studied earlier. In this case, we will print out a line of comma–separated names describing each field of every line, then show our results as before, but substituting a comma for the tab. We need a command line option for the CSV format. We cannot use -c because that already means use Connors' constant. For some strange reason, many web sites refer to CSV files as "Excel spreadsheet" (though the CSV format predates Excel). We will, therefore, use the -e switch to inform our software we want the output in the CSV format. We will start each line of the output with the focal length. This may sound repetitious at first, especially in the interactive mode: The user types in the focal length, and we are repeating it. But the user can type several focal lengths on one line. The input can also come in from a file or from the output of another program. In that case the user does not see the input at all. By the same token, the output can go to a file which we will want to examine later, or it could go to the printer, or become the input of another program. So, it makes perfect sense to start each line with the focal length as entered by the user. No, wait! Not as entered by the user. What if the user types in something like this: - -00000000150 - +00000000150 Clearly, we need to strip those leading zeros. So, we might consider reading the user input as is, converting it to binary inside the FPU, and printing it out from there. But... What if the user types something like this: - -17459765723452353453534535353530530534563507309676764423 - +17459765723452353453534535353530530534563507309676764423 Ha! The packed decimal FPU format lets us input 18–digit numbers. But the user has entered more than 18 digits. How do we handle that? Well, we could modify our code to read the first 18 digits, enter it to the FPU, then read more, multiply what we already have on the TOS by 10 raised to the number of additional digits, then add to it. Yes, we could do that. But in this program it would be ridiculous (in a different one it may be just the thing to do): Even the circumference of the Earth expressed in millimeters only takes 11 digits. Clearly, we cannot build a camera that large (not yet, anyway). So, if the user enters such a huge number, he is either bored, or testing us, or trying to break into the system, or playing games—doing anything but designing a pinhole camera. What will we do? We will slap him in the face, in a manner of speaking: - -17459765723452353453534535353530530534563507309676764423 ??? ??? ??? ??? ??? - +17459765723452353453534535353530530534563507309676764423 ??? ??? ??? ??? ??? To achieve that, we will simply ignore any leading zeros. Once we find a non–zero digit, we will initialize a counter to 0 and start taking three steps: Send the digit to the output. Append the digit to a buffer we will use later to produce the packed decimal we can send to the FPU. Increase the counter. Now, while we are taking these three steps, we also need to watch out for one of two conditions: If the counter grows above 18, we stop appending to the buffer. We continue reading the digits and sending them to the output. If, or rather when, the next input character is not a digit, we are done inputting for now. Incidentally, we can simply discard the non–digit, unless it is a #, which we must return to the input stream. It starts a comment, so we must see it after we are done producing output and start looking for more input. That still leaves one possibility uncovered: If all the user enters is a zero (or several zeros), we will never find a non–zero to display. We can determine this has happened whenever our counter stays at 0. In that case we need to send 0 to the output, and perform another "slap in the face": - -0 ??? ??? ??? ??? ??? - +0 ??? ??? ??? ??? ??? Once we have displayed the focal length and determined it is valid (greater than 0 but not exceeding 18 digits), we can calculate the pinhole diameter. It is not by coincidence that pinhole contains the word pin. Indeed, many a pinhole literally is a pin hole, a hole carefully punched with the tip of a pin. That is because a typical pinhole is very small. Our formula gets the result in millimeters. We will multiply it by 1000, so we can output the result in microns. At this point we have yet another trap to face: Too much precision. Yes, the FPU was designed for high precision mathematics. But we are not dealing with high precision mathematics. We are dealing with physics (optics, specifically). Suppose we want to convert a truck into a pinhole camera (we would not be the first ones to do that!). Suppose its box is 12 meters long, so we have the focal length of 12000. Well, using Bender's constant, it gives us square root of 12000 multiplied by 0.04, which is 4.381780460 millimeters, or 4381.780460 microns. Put either way, the result is absurdly precise. Our truck is not exactly 12000 millimeters long. We did not measure its length with such a precision, so stating we need a pinhole with the diameter of 4.381780460 millimeters is, well, deceiving. 4.4 millimeters would do just fine. I "only" used ten digits in the above example. Imagine the absurdity of going for all 18! We need to limit the number of significant digits of our result. One way of doing it is by using an integer representing microns. So, our truck would need a pinhole with the diameter of 4382 microns. Looking at that number, we still decide that 4400 microns, or 4.4 millimeters is close enough. Additionally, we can decide that no matter how big a result we get, we only want to display four siginificant digits (or any other number of them, of course). Alas, the FPU does not offer rounding to a specific number of digits (after all, it does not view the numbers as decimal but as binary). We, therefore, must devise an algorithm to reduce the number of significant digits. Here is mine (I think it is awkward—if you know a better one, please, let me know): Initialize a counter to 0. While the number is greater than or equal to 10000, divide it by 10 and increase the counter. Output the result. While the counter is greater than 0, output 0 and decrease the counter. The 10000 is only good if you want four significant digits. For any other number of significant digits, replace 10000 with 10 raised to the number of significant digits. We will, then, output the pinhole diameter in microns, rounded off to four significant digits. At this point, we know the focal length and the pinhole diameter. That means we have enough information to also calculate the f–number. We will display the f–number, rounded to four significant digits. Chances are the f–number will tell us very little. To make it more meaningful, we can find the nearest normalized f–number, i.e., the nearest power of the square root of 2. We do that by multiplying the actual f–number by itself, which, of course, will give us its square. We will then calculate its base–2 logarithm, which is much easier to do than calculating the base–square–root–of–2 logarithm! We will round the result to the nearest integer. Next, we will raise 2 to the result. Actually, the FPU gives us a good shortcut to do that: We can use the fscale op code to "scale" 1, which is analogous to shifting an integer left. Finally, we calculate the square root of it all, and we have the nearest normalized f–number. If all that sounds overwhelming—or too much work, perhaps—it may become much clearer if you see the code. It takes 9 op codes altogether: fmul st0, st0 fld1 fld st1 fyl2x frndint fld1 fscale fsqrt fstp st1 The first line, fmul st0, st0, squares the contents of the TOS (top of the stack, same as st, called st0 by nasm). The fld1 pushes 1 on the TOS. The next line, fld st1, pushes the square back to the TOS. At this point the square is both in st and st(2) (it will become clear why we leave a second copy on the stack in a moment). st(1) contains 1. Next, fyl2x calculates base–2 logarithm of st multiplied by st(1). That is why we placed 1 on st(1) before. At this point, st contains the logarithm we have just calculated, st(1) contains the square of the actual f–number we saved for later. frndint rounds the TOS to the nearest integer. fld1 pushes a 1. fscale shifts the 1 we have on the TOS by the value in st(1), effectively raising 2 to st(1). Finally, fsqrt calculates the square root of the result, i.e., the nearest normalized f–number. We now have the nearest normalized f–number on the TOS, the base–2 logarithm rounded to the nearest integer in st(1), and the square of the actual f–number in st(2). We are saving the value in st(2) for later. But we do not need the contents of st(1) anymore. The last line, fstp st1, places the contents of st to st(1), and pops. As a result, what was st(1) is now st, what was st(2) is now st(1), etc. The new st contains the normalized f–number. The new st(1) contains the square of the actual f–number we have stored there for posterity. At this point, we are ready to output the normalized f–number. Because it is normalized, we will not round it off to four significant digits, but will send it out in its full precision. The normalized f-number is useful as long as it is reasonably small and can be found on our light meter. Otherwise we need a different method of determining proper exposure. Earlier we have figured out the formula of calculating proper exposure at an arbitrary f–number from that measured at a different f–number. Every light meter I have ever seen can determine proper exposure at f5.6. We will, therefore, calculate an "f5.6 multiplier," i.e., by how much we need to multiply the exposure measured at f5.6 to determine the proper exposure for our pinhole camera. From the above formula we know this factor can be calculated by dividing our f–number (the actual one, not the normalized one) by 5.6, and squaring the result. Mathematically, dividing the square of our f–number by the square of 5.6 will give us the same result. Computationally, we do not want to square two numbers when we can only square one. So, the first solution seems better at first. But... 5.6 is a constant. We do not have to have our FPU waste precious cycles. We can just tell it to divide the square of the f–number by whatever 5.6² equals to. Or we can divide the f–number by 5.6, and then square the result. The two ways now seem equal. But, they are not! Having studied the principles of photography above, we remember that the 5.6 is actually square root of 2 raised to the fifth power. An irrational number. The square of this number is exactly 32. Not only is 32 an integer, it is a power of 2. We do not need to divide the square of the f–number by 32. We only need to use fscale to shift it right by five positions. In the FPU lingo it means we will fscale it with st(1) equal to -5. That is much faster than a division. So, now it has become clear why we have saved the square of the f–number on the top of the FPU stack. The calculation of the f5.6 multiplier is the easiest calculation of this entire program! We will output it rounded to four significant digits. There is one more useful number we can calculate: The number of stops our f–number is from f5.6. This may help us if our f–number is just outside the range of our light meter, but we have a shutter which lets us set various speeds, and this shutter uses stops. Say, our f–number is 5 stops from f5.6, and the light meter says we should use 1/1000 sec. Then we can set our shutter speed to 1/1000 first, then move the dial by 5 stops. This calculation is quite easy as well. All we have to do is to calculate the base-2 logarithm of the f5.6 multiplier we had just calculated (though we need its value from before we rounded it off). We then output the result rounded to the nearest integer. We do not need to worry about having more than four significant digits in this one: The result is most likely to have only one or two digits anyway. FPU Optimizations In assembly language we can optimize the FPU code in ways impossible in high languages, including C. Whenever a C function needs to calculate a floating–point value, it loads all necessary variables and constants into FPU registers. It then does whatever calculation is required to get the correct result. Good C compilers can optimize that part of the code really well. It "returns" the value by leaving the result on the TOS. However, before it returns, it cleans up. Any variables and constants it used in its calculation are now gone from the FPU. It cannot do what we just did above: We calculated the square of the f–number and kept it on the stack for later use by another function. We knew we would need that value later on. We also knew we had enough room on the stack (which only has room for 8 numbers) to store it there. A C compiler has no way of knowing that a value it has on the stack will be required again in the very near future. Of course, the C programmer may know it. But the only recourse he has is to store the value in a memory variable. That means, for one, the value will be changed from the 80-bit precision used internally by the FPU to a C double (64 bits) or even single (32 bits). That also means that the value must be moved from the TOS into the memory, and then back again. Alas, of all FPU operations, the ones that access the computer memory are the slowest. So, whenever programming the FPU in assembly language, look for the ways of keeping intermediate results on the FPU stack. We can take that idea even further! In our program we are using a constant (the one we named PC). It does not matter how many pinhole diameters we are calculating: 1, 10, 20, 1000, we are always using the same constant. Therefore, we can optimize our program by keeping the constant on the stack all the time. Early on in our program, we are calculating the value of the above constant. We need to divide our input by 10 for every digit in the constant. It is much faster to multiply than to divide. So, at the start of our program, we divide 10 into 1 to obtain 0.1, which we then keep on the stack: Instead of dividing the input by 10 for every digit, we multiply it by 0.1. By the way, we do not input 0.1 directly, even though we could. We have a reason for that: While 0.1 can be expressed with just one decimal place, we do not know how many binary places it takes. We, therefore, let the FPU calculate its binary value to its own high precision. We are using other constants: We multiply the pinhole diameter by 1000 to convert it from millimeters to microns. We compare numbers to 10000 when we are rounding them off to four significant digits. So, we keep both, 1000 and 10000, on the stack. And, of course, we reuse the 0.1 when rounding off numbers to four digits. Last but not least, we keep -5 on the stack. We need it to scale the square of the f–number, instead of dividing it by 32. It is not by coincidence we load this constant last. That makes it the top of the stack when only the constants are on it. So, when the square of the f–number is being scaled, the -5 is at st(1), precisely where fscale expects it to be. It is common to create certain constants from scratch instead of loading them from the memory. That is what we are doing with -5: fld1 ; TOS = 1 fadd st0, st0 ; TOS = 2 fadd st0, st0 ; TOS = 4 fld1 ; TOS = 1 faddp st1, st0 ; TOS = 5 fchs ; TOS = -5 We can generalize all these optimizations into one rule: Keep repeat values on the stack! PostScript is a stack–oriented programming language. There are many more books available about PostScript than about the FPU assembly language: Mastering PostScript will help you master the FPU. <application>pinhole</application>—The Code ;;;;;;; pinhole.asm ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; ; Find various parameters of a pinhole camera construction and use ; ; Started: 9-Jun-2001 ; Updated: 10-Jun-2001 ; ; Copyright (c) 2001 G. Adam Stanislav ; All rights reserved. ; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; %include 'system.inc' %define BUFSIZE 2048 section .data align 4 ten dd 10 thousand dd 1000 tthou dd 10000 fd.in dd stdin fd.out dd stdout envar db 'PINHOLE=' ; Exactly 8 bytes, or 2 dwords long pinhole db '04,', ; Bender's constant (0.04) connors db '037', 0Ah ; Connors' constant usg db 'Usage: pinhole [-b] [-c] [-e] [-p <value>] [-o <outfile>] [-i <infile>]', 0Ah usglen equ $-usg iemsg db "pinhole: Can't open input file", 0Ah iemlen equ $-iemsg oemsg db "pinhole: Can't create output file", 0Ah oemlen equ $-oemsg pinmsg db "pinhole: The PINHOLE constant must not be 0", 0Ah pinlen equ $-pinmsg toobig db "pinhole: The PINHOLE constant may not exceed 18 decimal places", 0Ah biglen equ $-toobig huhmsg db 9, '???' separ db 9, '???' sep2 db 9, '???' sep3 db 9, '???' sep4 db 9, '???', 0Ah huhlen equ $-huhmsg header db 'focal length in millimeters,pinhole diameter in microns,' db 'F-number,normalized F-number,F-5.6 multiplier,stops ' db 'from F-5.6', 0Ah headlen equ $-header section .bss ibuffer resb BUFSIZE obuffer resb BUFSIZE dbuffer resb 20 ; decimal input buffer bbuffer resb 10 ; BCD buffer section .text align 4 huh: call write push dword huhlen push dword huhmsg push dword [fd.out] sys.write add esp, byte 12 ret align 4 perr: push dword pinlen push dword pinmsg push dword stderr sys.write push dword 4 ; return failure sys.exit align 4 consttoobig: push dword biglen push dword toobig push dword stderr sys.write push dword 5 ; return failure sys.exit align 4 ierr: push dword iemlen push dword iemsg push dword stderr sys.write push dword 1 ; return failure sys.exit align 4 oerr: push dword oemlen push dword oemsg push dword stderr sys.write push dword 2 sys.exit align 4 usage: push dword usglen push dword usg push dword stderr sys.write push dword 3 sys.exit align 4 global _start _start: add esp, byte 8 ; discard argc and argv[0] sub esi, esi .arg: pop ecx or ecx, ecx je near .getenv ; no more arguments ; ECX contains the pointer to an argument cmp byte [ecx], '-' jne usage inc ecx mov ax, [ecx] inc ecx .o: cmp al, 'o' jne .i ; Make sure we are not asked for the output file twice cmp dword [fd.out], stdout jne usage ; Find the path to output file - it is either at [ECX+1], ; i.e., -ofile -- ; or in the next argument, ; i.e., -o file or ah, ah jne .openoutput pop ecx jecxz usage .openoutput: push dword 420 ; file mode (644 octal) push dword 0200h | 0400h | 01h ; O_CREAT | O_TRUNC | O_WRONLY push ecx sys.open jc near oerr add esp, byte 12 mov [fd.out], eax jmp short .arg .i: cmp al, 'i' jne .p ; Make sure we are not asked twice cmp dword [fd.in], stdin jne near usage ; Find the path to the input file or ah, ah jne .openinput pop ecx or ecx, ecx je near usage .openinput: push dword 0 ; O_RDONLY push ecx sys.open jc near ierr ; open failed add esp, byte 8 mov [fd.in], eax jmp .arg .p: cmp al, 'p' jne .c or ah, ah jne .pcheck pop ecx or ecx, ecx je near usage mov ah, [ecx] .pcheck: cmp ah, '0' jl near usage cmp ah, '9' ja near usage mov esi, ecx jmp .arg .c: cmp al, 'c' jne .b or ah, ah jne near usage mov esi, connors jmp .arg .b: cmp al, 'b' jne .e or ah, ah jne near usage mov esi, pinhole jmp .arg .e: cmp al, 'e' jne near usage or ah, ah jne near usage mov al, ',' mov [huhmsg], al mov [separ], al mov [sep2], al mov [sep3], al mov [sep4], al jmp .arg align 4 .getenv: ; If ESI = 0, we did not have a -p argument, ; and need to check the environment for "PINHOLE=" or esi, esi jne .init sub ecx, ecx .nextenv: pop esi or esi, esi je .default ; no PINHOLE envar found ; check if this envar starts with 'PINHOLE=' mov edi, envar mov cl, 2 ; 'PINHOLE=' is 2 dwords long rep cmpsd jne .nextenv ; Check if it is followed by a digit mov al, [esi] cmp al, '0' jl .default cmp al, '9' jbe .init ; fall through align 4 .default: ; We got here because we had no -p argument, ; and did not find the PINHOLE envar. mov esi, pinhole ; fall through align 4 .init: sub eax, eax sub ebx, ebx sub ecx, ecx sub edx, edx mov edi, dbuffer+1 mov byte [dbuffer], '0' ; Convert the pinhole constant to real .constloop: lodsb cmp al, '9' ja .setconst cmp al, '0' je .processconst jb .setconst inc dl .processconst: inc cl cmp cl, 18 ja near consttoobig stosb jmp short .constloop align 4 .setconst: or dl, dl je near perr finit fild dword [tthou] fld1 fild dword [ten] fdivp st1, st0 fild dword [thousand] mov edi, obuffer mov ebp, ecx call bcdload .constdiv: fmul st0, st2 loop .constdiv fld1 fadd st0, st0 fadd st0, st0 fld1 faddp st1, st0 fchs ; If we are creating a CSV file, ; print header cmp byte [separ], ',' jne .bigloop push dword headlen push dword header push dword [fd.out] sys.write .bigloop: call getchar jc near done ; Skip to the end of the line if you got '#' cmp al, '#' jne .num call skiptoeol jmp short .bigloop .num: ; See if you got a number cmp al, '0' jl .bigloop cmp al, '9' ja .bigloop ; Yes, we have a number sub ebp, ebp sub edx, edx .number: cmp al, '0' je .number0 mov dl, 1 .number0: or dl, dl ; Skip leading 0's je .nextnumber push eax call putchar pop eax inc ebp cmp ebp, 19 jae .nextnumber mov [dbuffer+ebp], al .nextnumber: call getchar jc .work cmp al, '#' je .ungetc cmp al, '0' jl .work cmp al, '9' ja .work jmp short .number .ungetc: dec esi inc ebx .work: ; Now, do all the work or dl, dl je near .work0 cmp ebp, 19 jae near .toobig call bcdload ; Calculate pinhole diameter fld st0 ; save it fsqrt fmul st0, st3 fld st0 fmul st5 sub ebp, ebp ; Round off to 4 significant digits .diameter: fcom st0, st7 fstsw ax sahf jb .printdiameter fmul st0, st6 inc ebp jmp short .diameter .printdiameter: call printnumber ; pinhole diameter ; Calculate F-number fdivp st1, st0 fld st0 sub ebp, ebp .fnumber: fcom st0, st6 fstsw ax sahf jb .printfnumber fmul st0, st5 inc ebp jmp short .fnumber .printfnumber: call printnumber ; F number ; Calculate normalized F-number fmul st0, st0 fld1 fld st1 fyl2x frndint fld1 fscale fsqrt fstp st1 sub ebp, ebp call printnumber ; Calculate time multiplier from F-5.6 fscale fld st0 ; Round off to 4 significant digits .fmul: fcom st0, st6 fstsw ax sahf jb .printfmul inc ebp fmul st0, st5 jmp short .fmul .printfmul: call printnumber ; F multiplier ; Calculate F-stops from 5.6 fld1 fxch st1 fyl2x sub ebp, ebp call printnumber mov al, 0Ah call putchar jmp .bigloop .work0: mov al, '0' call putchar align 4 .toobig: call huh jmp .bigloop align 4 done: call write ; flush output buffer ; close files push dword [fd.in] sys.close push dword [fd.out] sys.close finit ; return success push dword 0 sys.exit align 4 skiptoeol: ; Keep reading until you come to cr, lf, or eof call getchar jc done cmp al, 0Ah jne .cr ret .cr: cmp al, 0Dh jne skiptoeol ret align 4 getchar: or ebx, ebx jne .fetch call read .fetch: lodsb dec ebx clc ret read: jecxz .read call write .read: push dword BUFSIZE mov esi, ibuffer push esi push dword [fd.in] sys.read add esp, byte 12 mov ebx, eax or eax, eax je .empty sub eax, eax ret align 4 .empty: add esp, byte 4 stc ret align 4 putchar: stosb inc ecx cmp ecx, BUFSIZE je write ret align 4 write: jecxz .ret ; nothing to write sub edi, ecx ; start of buffer push ecx push edi push dword [fd.out] sys.write add esp, byte 12 sub eax, eax sub ecx, ecx ; buffer is empty now .ret: ret align 4 bcdload: ; EBP contains the number of chars in dbuffer push ecx push esi push edi lea ecx, [ebp+1] lea esi, [dbuffer+ebp-1] shr ecx, 1 std mov edi, bbuffer sub eax, eax mov [edi], eax mov [edi+4], eax mov [edi+2], ax .loop: lodsw sub ax, 3030h shl al, 4 or al, ah mov [edi], al inc edi loop .loop fbld [bbuffer] cld pop edi pop esi pop ecx sub eax, eax ret align 4 printnumber: push ebp mov al, [separ] call putchar ; Print the integer at the TOS mov ebp, bbuffer+9 fbstp [bbuffer] ; Check the sign mov al, [ebp] dec ebp or al, al jns .leading ; We got a negative number (should never happen) mov al, '-' call putchar .leading: ; Skip leading zeros mov al, [ebp] dec ebp or al, al jne .first cmp ebp, bbuffer jae .leading ; We are here because the result was 0. ; Print '0' and return mov al, '0' jmp putchar .first: ; We have found the first non-zero. ; But it is still packed test al, 0F0h jz .second push eax shr al, 4 add al, '0' call putchar pop eax and al, 0Fh .second: add al, '0' call putchar .next: cmp ebp, bbuffer jb .done mov al, [ebp] push eax shr al, 4 add al, '0' call putchar pop eax and al, 0Fh add al, '0' call putchar dec ebp jmp short .next .done: pop ebp or ebp, ebp je .ret .zeros: mov al, '0' call putchar dec ebp jne .zeros .ret: ret The code follows the same format as all the other filters we have seen before, with one subtle exception:
We are no longer assuming that the end of input implies the end of things to do, something we took for granted in the character–oriented filters. This filter does not process characters. It processes a language (albeit a very simple one, consisting only of numbers). When we have no more input, it can mean one of two things: We are done and can quit. This is the same as before. The last character we have read was a digit. We have stored it at the end of our ASCII–to–float conversion buffer. We now need to convert the contents of that buffer into a number and write the last line of our output. For that reason, we have modified our getchar and our read routines to return with the carry flag clear whenever we are fetching another character from the input, or the carry flag set whenever there is no more input. Of course, we are still using assembly language magic to do that! Take a good look at getchar. It always returns with the carry flag clear. Yet, our main code relies on the carry flag to tell it when to quit—and it works. The magic is in read. Whenever it receives more input from the system, it just returns to getchar, which fetches a character from the input buffer, clears the carry flag and returns. But when read receives no more input from the system, it does not return to getchar at all. Instead, the add esp, byte 4 op code adds 4 to ESP, sets the carry flag, and returns. So, where does it return to? Whenever a program uses the call op code, the microprocessor pushes the return address, i.e., it stores it on the top of the stack (not the FPU stack, the system stack, which is in the memory). When a program uses the ret op code, the microprocessor pops the return value from the stack, and jumps to the address that was stored there. But since we added 4 to ESP (which is the stack pointer register), we have effectively given the microprocessor a minor case of amnesia: It no longer remembers it was getchar that called read. And since getchar never pushed anything before calling read, the top of the stack now contains the return address to whatever or whoever called getchar. As far as that caller is concerned, he called getchar, which returned with the carry flag set!
Other than that, the bcdload routine is caught up in the middle of a Lilliputian conflict between the Big–Endians and the Little–Endians. It is converting the text representation of a number into that number: The text is stored in the big–endian order, but the packed decimal is little–endian. To solve the conflict, we use the std op code early on. We cancel it with cld later on: It is quite important we do not call anything that may depend on the default setting of the direction flag while std is active. Everything else in this code should be quite clear, providing you have read the entire chapter that precedes it. It is a classical example of the adage that programming requires a lot of thought and only a little coding. Once we have thought through every tiny detail, the code almost writes itself. Using <application>pinhole</application> Because we have decided to make the program ignore any input except for numbers (and even those inside a comment), we can actually perform textual queries. We do not have to, but we can. In my humble opinion, forming a textual query, instead of having to follow a very strict syntax, makes software much more user friendly. Suppose we want to build a pinhole camera to use the 4x5 inch film. The standard focal length for that film is about 150mm. We want to fine–tune our focal length so the pinhole diameter is as round a number as possible. Let us also suppose we are quite comfortable with cameras but somewhat intimidated by computers. Rather than just have to type in a bunch of numbers, we want to ask a couple of questions. Our session might look like this: - -&prompt.user; pinhole +&prompt.user; pinhole Computer, What size pinhole do I need for the focal length of 150? 150 490 306 362 2930 12 Hmmm... How about 160? 160 506 316 362 3125 12 Let's make it 155, please. 155 498 311 362 3027 12 Ah, let's try 157... 157 501 313 362 3066 12 156? 156 500 312 362 3047 12 That's it! Perfect! Thank you very much! -^D - +^D We have found that while for the focal length of 150, our pinhole diameter should be 490 microns, or 0.49 mm, if we go with the almost identical focal length of 156 mm, we can get away with a pinhole diameter of exactly one half of a millimeter. Scripting Because we have chosen the # character to denote the start of a comment, we can treat our pinhole software as a scripting language. You have probably seen shell scripts that start with: #! /bin/sh ...or... #!/bin/sh ...because the blank space after the #! is optional. Whenever Unix is asked to run an executable file which starts with the #!, it assumes the file is a script. It adds the command to the rest of the first line of the script, and tries to execute that. Suppose now that we have installed pinhole in /usr/local/bin/, we can now write a script to calculate various pinhole diameters suitable for various focal lengths commonly used with the 120 film. The script might look something like this: #! /usr/local/bin/pinhole -b -i # Find the best pinhole diameter # for the 120 film ### Standard 80 ### Wide angle 30, 40, 50, 60, 70 ### Telephoto 100, 120, 140 Because 120 is a medium size film, we may name this file medium. We can set its permissions to execute, and run it as if it were a program: - -&prompt.user; chmod 755 medium -&prompt.user; ./medium - +&prompt.user; chmod 755 medium +&prompt.user; ./medium Unix will interpret that last command as: - -&prompt.user; /usr/local/bin/pinhole -b -i ./medium - +&prompt.user; /usr/local/bin/pinhole -b -i ./medium It will run that command and display: - -80 358 224 256 1562 11 +80 358 224 256 1562 11 30 219 137 128 586 9 40 253 158 181 781 10 50 283 177 181 977 10 60 310 194 181 1172 10 70 335 209 181 1367 10 100 400 250 256 1953 11 120 438 274 256 2344 11 -140 473 296 256 2734 11 - +140 473 296 256 2734 11 Now, let us enter: -&prompt.user; ./medium -c - +&prompt.user; ./medium -c Unix will treat that as: - -&prompt.user; /usr/local/bin/pinhole -b -i ./medium -c - +&prompt.user; /usr/local/bin/pinhole -b -i ./medium -c That gives it two conflicting options: -b and -c (Use Bender's constant and use Connors' constant). We have programmed it so later options override early ones—our program will calculate everything using Connors' constant: - -80 331 242 256 1826 11 +80 331 242 256 1826 11 30 203 148 128 685 9 40 234 171 181 913 10 50 262 191 181 1141 10 60 287 209 181 1370 10 70 310 226 256 1598 11 100 370 270 256 2283 11 120 405 296 256 2739 11 -140 438 320 362 3196 12 - +140 438 320 362 3196 12 We decide we want to go with Bender's constant after all. We want to save its values as a comma–separated file: &prompt.user; ./medium -b -e > bender &prompt.user; cat bender focal length in millimeters,pinhole diameter in microns,F-number,normalized F-number,F-5.6 multiplier,stops from F-5.6 80,358,224,256,1562,11 30,219,137,128,586,9 40,253,158,181,781,10 50,283,177,181,977,10 60,310,194,181,1172,10 70,335,209,181,1367,10 100,400,250,256,1953,11 120,438,274,256,2344,11 140,473,296,256,2734,11 -&prompt.user; - +&prompt.user; Caveats Assembly language programmers who "grew up" under MS DOS and Windows often tend to take shortcuts. Reading the keyboard scan codes and writing directly to video memory are two classical examples of practices which, under MS DOS are not frowned upon but considered the right thing to do. The reason? Both the PC BIOS and MS DOS are notoriously slow when performing these operations. You may be tempted to continue similar practices in the Unix environment. For example, I have seen a web site which explains how to access the keyboard scan codes on a popular Unix clone. That is generally a very bad idea in Unix environment! Let me explain why. Unix Is Protected For one thing, it may simply not be possible. Unix runs in protected mode. Only the kernel and device drivers are allowed to access hardware directly. Perhaps a particular Unix clone will let you read the keyboard scan codes, but chances are a real Unix operating system will not. And even if one version may let you do it, the next one may not, so your carefully crafted software may become a dinosaur overnight. Unix Is an Abstraction But there is a much more important reason not to try accessing the hardware directly (unless, of course, you are writing a device driver), even on the Unix-like systems that let you do it: Unix is an abstraction! There is a major difference in the philosophy of design between MS DOS and Unix. MS DOS was designed as a single-user system. It is run on a computer with a keyboard and a video screen attached directly to that computer. User input is almost guaranteed to come from that keyboard. Your program's output virtually always ends up on that screen. This is NEVER guaranteed under Unix. It is quite common for a Unix user to pipe and redirect program input and output: -&prompt.user; program1 | program2 | program3 > file1 - +&prompt.user; program1 | program2 | program3 > file1 If you have written program2, your input does not come from the keyboard but from the output of program1. Similarly, your output does not go to the screen but becomes the input for program3 whose output, in turn, goes to file1. But there is more! Even if you made sure that your input comes from, and your output goes to, the terminal, there is no guarantee the terminal is a PC: It may not have its video memory where you expect it, nor may its keyboard be producing PC-style scan codes. It may be a Macintosh, or any other computer. Now you may be shaking your head: My software is in PC assembly language, how can it run on a Macintosh? But I did not say your software would be running on a Macintosh, only that its terminal may be a Macintosh. Under Unix, the terminal does not have to be directly attached to the computer that runs your software, it can even be on another continent, or, for that matter, on another planet. It is perfectly possible that a Macintosh user in Australia connects to a Unix system in North America (or anywhere else) via telnet. The software then runs on one computer, while the terminal is on a different computer: If you try to read the scan codes, you will get the wrong input! Same holds true about any other hardware: A file you are reading may be on a disk you have no direct access to. A camera you are reading images from may be on a space shuttle, connected to you via satellites. That is why under Unix you must never make any assumptions about where your data is coming from and going to. Always let the system handle the physical access to the hardware. These are caveats, not absolute rules. Exceptions are possible. For example, if a text editor has determined it is running on a local machine, it may want to read the scan codes directly for improved control. I am not mentioning these caveats to tell you what to do or what not to do, just to make you aware of certain pitfalls that await you if you have just arrived to Unix form MS DOS. Of course, creative people often break rules, and it is OK as long as they know they are breaking them and why. Acknowledgements This tutorial would never have been possible without the help of many experienced FreeBSD programmers from the FreeBSD hackers mailing list, many of whom have patiently answered my questions, and pointed me in the right direction in my attempts to explore the inner workings of Unix system programming in general and FreeBSD in particular. Thomas M. Sommers opened the door for me. His How do I write "Hello, world" in FreeBSD assembler? web page was my first encounter with an example of assembly language programming under FreeBSD. Jake Burkholder has kept the door open by willingly answering all of my questions and supplying me with example assembly language source code. Copyright © 2000-2001 G. Adam Stanislav. All rights reserved. diff --git a/en_US.ISO8859-1/books/faq/book.sgml b/en_US.ISO8859-1/books/faq/book.sgml index b9e7a4b580..5b6a27c7b8 100644 --- a/en_US.ISO8859-1/books/faq/book.sgml +++ b/en_US.ISO8859-1/books/faq/book.sgml @@ -1,12523 +1,12519 @@ %man; %authors; %bookinfo; %mailing-lists; ]> Frequently Asked Questions for FreeBSD 2.X, 3.X and 4.X The FreeBSD Documentation Project - $FreeBSD: doc/en_US.ISO8859-1/books/faq/book.sgml,v 1.241 2001/06/29 02:30:44 chris Exp $ + $FreeBSD: doc/en_US.ISO8859-1/books/faq/book.sgml,v 1.242 2001/06/29 03:51:48 chris Exp $ 1995 1996 1997 1998 1999 2000 2001 The FreeBSD Documentation Project &bookinfo.legalnotice; This is the FAQ for FreeBSD versions 2.X, 3.X, and 4.X. All entries are assumed to be relevant to FreeBSD 2.0.5 and later, unless otherwise noted. Any entries with a <XXX> are under construction. If you are interested in helping with this project, send email to the &a.doc;. The latest version of this document is always available from the FreeBSD World Wide Web server. It may also be downloaded as one large HTML file with HTTP or as plain text, postscript, PDF, etc. from the FreeBSD FTP server. You may also want to Search the FAQ. Introduction Welcome to the FreeBSD 2.X-4.X FAQ! As is usual with Usenet FAQs, this document aims to cover the most frequently asked questions concerning the FreeBSD operating system (and of course answer them!). Although originally intended to reduce bandwidth and avoid the same old questions being asked over and over again, FAQs have become recognized as valuable information resources. Every effort has been made to make this FAQ as informative as possible; if you have any suggestions as to how it may be improved, please feel free to mail them to the &a.faq;. What is FreeBSD? Briefly, FreeBSD is a UN*X-like operating system for the i386 and Alpha/AXP platforms based on U.C. Berkeley's 4.4BSD-Lite release, with some 4.4BSD-Lite2 enhancements. It is also based indirectly on William Jolitz's port of U.C. Berkeley's Net/2 to the i386, known as 386BSD, though very little of the 386BSD code remains. A fuller description of what FreeBSD is and how it can work for you may be found on the FreeBSD home page. FreeBSD is used by companies, Internet Service Providers, researchers, computer professionals, students and home users all over the world in their work, education and recreation. See some of them in the FreeBSD Gallery. For more detailed information on FreeBSD, please see the FreeBSD Handbook. What are the goals of FreeBSD? The goals of the FreeBSD Project are to provide software that may be used for any purpose and without strings attached. Many of us have a significant investment in the code (and project) and would certainly not mind a little financial compensation now and then, but we are definitely not prepared to insist on it. We believe that our first and foremost mission is to provide code to any and all comers, and for whatever purpose, so that the code gets the widest possible use and provides the widest possible benefit. This is, we believe, one of the most fundamental goals of Free Software and one that we enthusiastically support. That code in our source tree which falls under the GNU General Public License (GPL) or GNU Library General Public License (LGPL) comes with slightly more strings attached, though at least on the side of enforced access rather than the usual opposite. Due to the additional complexities that can evolve in the commercial use of GPL software, we do, however, endeavor to replace such software with submissions under the more relaxed FreeBSD copyright whenever possible. Why is it called FreeBSD? It may be used free of charge, even by commercial users. Full source for the operating system is freely available, and the minimum possible restrictions have been placed upon its use, distribution and incorporation into other work (commercial or non-commercial). Anyone who has an improvement and/or bug fix is free to submit their code and have it added to the source tree (subject to one or two obvious provisions). For those of our readers whose first language is not English, it may be worth pointing out that the word free is being used in two ways here, one meaning at no cost, the other meaning you can do whatever you like. Apart from one or two things you cannot do with the FreeBSD code, for example pretending you wrote it, you really can do whatever you like with it. What is the latest version of FreeBSD? Version 4.3 is the latest STABLE version; it was released in April, 2001. This is also the latest RELEASE version. Briefly explained, -STABLE is aimed at the ISP or other corporate user who wants stability and a low change count over the wizzy new features of the latest -CURRENT snapshot. Releases can come from either branch, but you should only use -CURRENT if you are sure that you are prepared for its increased volatility (relative to -STABLE, that is). Releases are only made every few months. While many people stay more up-to-date with the FreeBSD sources (see the questions on FreeBSD-CURRENT and FreeBSD-STABLE) than that, doing so is more of a commitment, as the sources are a moving target. What is FreeBSD-CURRENT? FreeBSD-CURRENT is the development version of the operating system, which will in due course become 5.0-RELEASE. As such, it is really only of interest to developers working on the system and die-hard hobbyists. See the relevant section in the handbook for details on running -CURRENT. If you are not familiar with the operating system or are not capable of identifying the difference between a real problem and a temporary problem, you should not use FreeBSD-CURRENT. This branch sometimes evolves quite quickly and can be un-buildable for a number of days at a time. People that use FreeBSD-CURRENT are expected to be able to analyze any problems and only report them if they are deemed to be mistakes rather than glitches. Questions such as make world produces some error about groups on the -CURRENT mailing list are sometimes treated with contempt. Every day, snapshot releases are made based on the current state of the -CURRENT and -STABLE branches. Nowadays, distributions of the occasional snapshot are now being made available. The goals behind each snapshot release are: To test the latest version of the installation software. To give people who would like to run -CURRENT or -STABLE but who do not have the time and/or bandwidth to follow it on a day-to-day basis an easy way of bootstrapping it onto their systems. To preserve a fixed reference point for the code in question, just in case we break something really badly later. (Although CVS normally prevents anything horrible like this happening :) To ensure that any new features in need of testing have the greatest possible number of potential testers. No claims are made that any -CURRENT snapshot can be considered production quality for any purpose. If you want to run a stable and fully tested system, you will have to stick to full releases, or use the -STABLE snaphosts. Snapshot releases are directly available from ftp://current.FreeBSD.org/pub/FreeBSD/ for 5.0-CURRENT and releng4.FreeBSD.org for 4-STABLE snapshots. 3-STABLE snapshots are not being produced at the time of this writing (May 2000). Snapshots are generated, on the average, once a day for all actively developed branches. What is the FreeBSD-STABLE concept? Back when FreeBSD 2.0.5 was released, we decided to branch FreeBSD development into two parts. One branch was named -STABLE, with the intention that only well-tested bug fixes and small incremental enhancements would be made to it (for Internet Service Providers and other commercial enterprises for whom sudden shifts or experimental features are quite undesirable). The other branch was -CURRENT, which essentially has been one unbroken line leading towards 5.0-RELEASE (and beyond) since 2.0 was released. If a little ASCII art would help, this is how it looks: 2.0 | | | [2.1-STABLE] *BRANCH* 2.0.5 -> 2.1 -> 2.1.5 -> 2.1.6 -> 2.1.7.1 [2.1-STABLE ends] | (Mar 1997) | | | [2.2-STABLE] *BRANCH* 2.2.1 -> 2.2.2-RELEASE -> 2.2.5 -> 2.2.6 -> 2.2.7 -> 2.2.8 [end] | (Mar 1997) (Oct 97) (Apr 98) (Jul 98) (Dec 98) | | 3.0-SNAPs (started Q1 1997) | | 3.0-RELEASE (Oct 1998) | | [3.0-STABLE] *BRANCH* 3.1-RELEASE (Feb 1999) -> 3.2 -> 3.3 -> 3.4 -> 3.5 -> 3.5.1 | (May 1999) (Sep 1999) (Dec 1999) (June 2000) (July 2000) | | [4.0-STABLE] *BRANCH* 4.0 (Mar 2000) -> 4.1 -> 4.1.1 -> 4.2 -> 4.3 -> ... future 4.x releases ... | | (July 2000) (Sep 2000) (Nov 2000) \|/ + [5.0-CURRENT continues] The 2.2-STABLE branch was retired with the release of 2.2.8. The 3-STABLE branch has ended with the release of 3.5.1, the final 3.X release. The only changes made to either of these branches will be, for the most part, security-related bug fixes. 4-STABLE is the actively developed -STABLE branch. The latest release on the 4-STABLE is 4.3-RELEASE, which was released in April 2001. The 5-CURRENT branch is slowly progressing toward 5.0-RELEASE and beyond. See What is FreeBSD-CURRENT? for more information on this branch. When are FreeBSD releases made? As a general principle, the FreeBSD core team only release a new version of FreeBSD when they believe that there are sufficient new features and/or bug fixes to justify one, and are satisfied that the changes made have settled down sufficiently to avoid compromising the stability of the release. Many users regard this caution as one of the best things about FreeBSD, although it can be a little frustrating when waiting for all the latest goodies to become available... Releases are made about every 4 months on average. For people needing (or wanting) a little more excitement, binary snapshots are made every day... see above. Who is responsible for FreeBSD? The key decisions concerning the FreeBSD project, such as the overall direction of the project and who is allowed to add code to the source tree, are made by a core team of 9 people. There is a much larger team of more than 200 committers who are authorized to make changes directly to the FreeBSD source tree. However, most non-trivial changes are discussed in advance in the mailing lists, and there are no restrictions on who may take part in the discussion. Where can I get FreeBSD? Every significant release of FreeBSD is available via anonymous ftp from the FreeBSD FTP site: For the current 3.X-STABLE release, 3.5.1-RELEASE, see the 3.5.1-RELEASE directory. The current 4-STABLE release, 4.3-RELEASE can be found in the 4.3-RELEASE directory. 4.X snapshots are usually made once a day. 5.0 Snapshot releases are made once a day for the -CURRENT branch, these being of service purely to bleeding-edge testers and developers. FreeBSD is also available via CDROM, from the following place(s):
Wind River Systems 4041 Pike Lane, Suite F Concord, CA 94520 USA Orders: +1 800 786-9907 Questions: +1 925 674-0783 Fax: +1 925 674-0821 Email: orders@osd.bsdi.com WWW: http://www.freebsdmall.com/
In Australia, you may find it at:
Advanced Multimedia Distributors Factory 1/1 Ovata Drive Tullamarine, Melbourne Victoria Australia Voice: +61 3 9338 6777
CDROM Support BBS 17 Irvine St Peppermint Grove, WA 6011 Voice: +61 9 385-3793 Fax: +61 9 385-2360
And in the UK:
The Public Domain & Shareware Library Winscombe House, Beacon Rd Crowborough Sussex. TN6 1UL Voice: +44 1892 663-298 Fax: +44 1892 667-473
Where do I find info on the FreeBSD mailing lists? You can find full information in the Handbook entry on mailing-lists. Where do I find the FreeBSD Y2K info? You can find full information in the FreeBSD Y2K page. What FreeBSD news groups are available? You can find full information in the Handbook entry on newsgroups. Are there FreeBSD IRC (Internet Relay Chat) channels? Yes, most major IRC networks host a FreeBSD chat channel: Channel #FreeBSD on EFNet is a FreeBSD forum, but do not go there for tech support or to try and get folks there to help you avoid the pain of reading man pages or doing your own research. It is a chat channel, first and foremost, and topics there are just as likely to involve sex, sports or nuclear weapons as they are FreeBSD. You Have Been Warned! Available at server irc.chat.org. Channel #FreeBSDhelp on EFNet is a channel dedicated to helping FreeBSD users. They are much more sympathetic to questions then #FreeBSD is. Channel #FreeBSD on DALNET is available at irc.dal.net in the US and irc.eu.dal.net in Europe. Channel #FreeBSD on UNDERNET is available at us.undernet.org in the US and eu.undernet.org in Europe. Since it is a help channel, be prepared to read the documents you are referred to. Channel #FreeBSD on HybNet. This channel is a help channel. A list of servers can be found on the HybNet web site. Each of these channels are distinct and are not connected to each other. Their chat styles also differ, so you may need to try each to find one suited to your chat style. As with *all* types of IRC traffic, if you are easily offended or cannot deal with lots of young people (and more than a few older ones) doing the verbal equivalent of jello wrestling, do not even bother with it. How do I access the Problem Report database? The Problem Report database of all user change requests may be queried (or submitted to) by using our web-based PR submission and query interfaces. The &man.send-pr.1; command can also be used to submit problem reports and change requests via electronic mail. Is the documentation available in other formats, such as plain text (ASCII), or Postscript? Yes. The documentation is available in a number of different formats and compression schemes on the FreeBSD FTP site, in the /pub/FreeBSD/doc/ directory. The documentation is categorised in a number of different ways. These include: The document's name, such as faq, or handbook. The document's language and encoding. These are based on the locale names you will find under /usr/share/locale on your FreeBSD system. The current languages and encodings that we have for documentation are as follows: Name Meaning en_US.ISO8859-1 US English de_DE.ISO8859-1 German es_ES.ISO8859-1 Spanish fr_FR.ISO8859-1 French ja_JP.eucJP Japanese (EUC encoding) ru_RU.KOI8-R Russian (KOI8-R encoding) zh_TW.Big5 Chinese (Big5 encoding) Some documents may not be available in all languages. The document's format. We produce the documentation in a number of different output formats to try and make it as flexible as possible. The current formats are; Format Meaning html-split A collection of small, linked, HTML files. html One large HTML file containing the entire document pdb Palm Pilot database format, for use with the iSilo reader. pdf Adobe's Portable Document Format ps Postscript rtf Microsoft's Rich Text Format Page numbers are not automatically updated when loading this format in to Word. Press CTRL+A, CTRL+END, F9 after loading the document, to update the page numbers. txt Plain text The compression and packaging scheme. There are three of these currently in use. Where the format is html-split, the files are bundled up using &man.tar.1;. The resulting .tar file is then compressed using the compression schemes detailed in the next point. All the other formats generate one file, called book.format (i.e., book.pdb, book.html, and so on). These files are then compressed using three compression schemes. Scheme Description zip The Zip format. If you want to uncompress this on FreeBSD you will need to install the archivers/unzip port first. gz The GNU Zip format. Use &man.gunzip.1; to uncompress these files, which is part of FreeBSD. bz2 The BZip2 format. Less widespread than the others, but generally gives smaller files. Install the archivers/bzip2 port to uncompress these files. So the Postscript version of the Handbook, compressed using BZip2 will be stored in a file called book.sgml.bz2 in the handbook/ directory. The formatted documentation is also available as a FreeBSD package, of which more later. After choosing the format and compression mechanism that you want to download, you must then decide whether or not you want to download the document as a FreeBSD package. The advantage of downloading and installing the package is that the documentation can then be managed using the normal FreeBSD package management comments, such as &man.pkg.add.1; and &man.pkg.delete.1;. If you decide to download and install the package then you must know the filename to download. The documentation-as-packages files are stored in a directory called packages. Each package file looks like document-name.lang.encoding.format.tgz. For example, the FAQ, in English, formatted as PDF, is in the package called faq.en_US.ISO8859-1.pdf.tgz. Knowing this, you can use the following command to install the English PDF FAQ package. &prompt.root; pkg_add ftp://ftp.FreeBSD.org/pub/FreeBSD/doc/packages/faq.en_US.ISO8859-1.pdf.tgz Having done that, you can use &man.pkg.info.1; to determine where the file has been installed. &prompt.root; pkg_info -f faq.en_US.ISO8859-1.pdf Information for faq.en_US.ISO8859-1.pdf: Packing list: Package name: faq.en_US.ISO8859-1.pdf CWD to /usr/share/doc/en_US.ISO8859-1/books/faq File: book.pdf CWD to . File: +COMMENT (ignored) File: +DESC (ignored) As you can see, book.pdf will have been installed in to /usr/share/doc/en_US.ISO8859-1/books/faq. If you do not want to use the packages then you will have to download the compressed files yourself, uncompress them, and then copy the appropriate documents in to place. For example, the split HTML version of the FAQ, compressed using &man.gzip.1;, can be found in the en_US.ISO8859-1/books/faq/book.html-split.tar.gz file. To download and uncompress that file you would have to do this. &prompt.root; fetch ftp://ftp.freebsd.org/pub/FreeBSD/doc/en_US.ISO8859-1/books/faq/book.html-split.tar.gz &prompt.root; gzip -d book.html-split.tar.gz &prompt.root; tar xvf book.html-split.tar You will be left with a collection of .html files. The main one is called index.html, which will contain the table of contents, introductory material, and links to the other parts of the document. You can then copy or move these to their final location as necessary. How do I become a FreeBSD Web mirror? Certainly! There are multiple ways to mirror the Web pages. Using CVSup: You can retrieve the formatted files using CVSup, and connecting to a CVSup server. To retrieve the webpages, please look at the example supfile, which can be found in /usr/share/examples/cvsup/www-supfile. Using ftp mirror: You can download the FTP server's copy of the web site sources using your favorite ftp mirror tool. Keep in mind that you have to build these sources before publishing them. Simply start at ftp://ftp.FreeBSD.org/pub/FreeBSD/FreeBSD-current/www. What other sources of information are there? The following newsgroups contain pertinent discussion for FreeBSD users: comp.unix.bsd.freebsd.announce (moderated) comp.unix.bsd.freebsd.misc comp.unix.bsd.misc Web resources: The FreeBSD Home Page. If you have a laptop, be sure and see Tatsumi Hosokawa's Mobile Computing page in Japan. For information on SMP (Symmetric MultiProcessing), please see the SMP support page. For information on FreeBSD multimedia applications, please see the multimedia page. If you are interested specifically in the Bt848 video capture chip, then follow that link. The FreeBSD handbook also has a fairly complete bibliography section which is worth reading if you are looking for actual books to buy. Installation Which file do I download to get FreeBSD? Prior to release 3.1, you only needed one floppy image to install FreeBSD, namely floppies/boot.flp. However, since release 3.1 the Project has added base support for a wide variety of hardware which needed more space, and thus for 3.x and 4.x we now use two floppy images, namely floppies/kernel.flp and floppies/mfsroot.flp. These images need to be copied onto floppies by tools like fdimage or &man.dd.1;. If you need to download the distributions yourself (for a DOS filesystem install, for instance), below are some recommendations for distributions to grab: bin/ manpages/ compat*/ doc/ src/ssys.* Full instructions on this procedure and a little bit more about installation issues in general can be found in the Handbook entry on installing FreeBSD. What do I do if the floppy images does not fit on a single floppy? A 3.5 inch (1.44MB) floppy can accomodate 1474560 bytes of data. The boot image is exactly 1474560 bytes in size. Common mistakes when preparing the boot floppy are: Not downloading the floppy image in binary mode when using FTP. Some FTP clients default their transfer mode to ascii and attempt to change any end-of-line characters received to match the conventions used by the client's system. This will almost invariably corrupt the boot image. Check the size of the downloaded boot image: if it is not exactly that on the server, then the download process is suspect. To workaround: type binary at the FTP command prompt after getting connected to the server and before starting the download of the image. Using the DOS copy command (or equivalent GUI tool) to transfer the boot image to floppy. Programs like copy will not work as the boot image has been created to be booted into directly. The image has the complete content of the floppy, track for track, and is not meant to be placed on the floppy as a regular file. You have to transfer it to the floppy raw, using the low-level tools (e.g. fdimage or rawrite) described in the installation guide to FreeBSD. Where are the instructions for installing FreeBSD? Installation instructions can be found in the Handbook entry on installing FreeBSD. What do I need in order to run FreeBSD? You will need a 386 or better PC, with 5 MB or more of RAM and at least 60 MB of hard disk space. It can run with a low end MDA graphics card but to run X11R6, a VGA or better video card is needed. See also the section on I have only 4 MB of RAM. Can I install FreeBSD? FreeBSD 2.1.7 was the last version of FreeBSD that could be installed on a 4MB system. Newer versions of FreeBSD, like 2.2, need at least 5MB to install on a new system. All versions of FreeBSD, including 3.0, will run in 4MB of RAM, they just can not run the installation program in 4MB. You can add extra memory for the install process, if you like, and then after the system is up and running, go back to 4MB. Or you could always just swap your disk into a system which has >4MB, install onto it and then swap it back. There are also situations in which FreeBSD 2.1.7 will not install in 4 MB. To be exact: it does not install with 640 kB base + 3 MB extended memory. If your motherboard can remap some of the lost memory out of the 640kB to 1MB region, then you may still be able to get FreeBSD 2.1.7 up. Try to go into your BIOS setup and look for a remap option. Enable it. You may also have to disable ROM shadowing. It may be easier to get 4 more MB just for the install. Build a custom kernel with only the options you need and then get the 4MB out again. You may also install 2.0.5 and then upgrade your system to 2.1.7 with the upgrade option of the 2.1.7 installation program. After the installation, if you build a custom kernel, it will run in 4 MB. Someone has even succeeded in booting with 2 MB (the system was almost unusable though :-)) How can I make my own custom install floppy? Currently there is no way to just make a custom install floppy. You have to cut a whole new release, which will include your install floppy. To make a custom release, follow the instructions here. Can I have more than one operating system on my PC? Have a look at The multi-OS page. Can Windows 95/98 co-exist with FreeBSD? Install Windows 95/98 first, after that FreeBSD. FreeBSD's boot manager will then manage to boot Win95/98 and FreeBSD. If you install Windows 95/98 second, it will boorishly overwrite your boot manager without even asking. If that happens, see the next section. Windows 95/98 killed my boot manager! How do I get it back? You can reinstall the boot manager FreeBSD comes with in one of three ways: Running DOS, go into the tools/ directory of your FreeBSD distribution and look for bootinst.exe. You run it like so: ...\TOOLS> bootinst.exe boot.bin and the boot manager will be reinstalled. Boot the FreeBSD boot floppy again and go to the Custom installation menu item. Choose Partition. Select the drive which used to contain your boot manager (likely the first one) and when you come to the partition editor for it, as the very first thing (e.g. do not make any changes) select (W)rite. This will ask for confirmation, say yes, and when you get the Boot Manager selection prompt, be sure to select Boot Manager. This will re-write the boot manager to disk. Now quit out of the installation menu and reboot off the hard disk as normal. Boot the FreeBSD boot floppy (or CD-ROM) and choose the Fixit menu item. Select either the Fixit floppy or CD-ROM #2 (the live file system option) as appropriate and enter the fixit shell. Then execute the following command: Fixit# fdisk -B -b /boot/boot0 bootdevice substituting bootdevice for your real boot device such as ad0 (first IDE disk), ad4 (first IDE disk on auxiliary controller), da0 (first SCSI disk), etc. My A, T, or X series IBM Thinkpad locks up when I first booted up my FreeBSD installation. How can I solve this? A bug in early revisions of IBM's BIOS on these machines mistakenly identifies the FreeBSD partition as a potential FAT suspend-to-disk partition. When the BIOS tries to parse the FreeBSD partition it hangs. According to IBMIn an e-mail from Keith Frechette kfrechet@us.ibm.com., the following model/BIOS release numbers incorporate the fix. Model BIOS revision T20 IYET49WW or later T21 KZET22WW or later A20p IVET62WW or later A20m IWET54WW or later A21p KYET27WW or later A21m KXET24WW or later A21e KUET30WW It has been reported that later IBM BIOS revisions may have reintroduced the bug. This message from Jacques Vidrine to the &a.mobile; describes a procedure which may work if your newer IBM laptop does not boot FreeBSD properly, and you can upgrade or downgrade the BIOS.. If you have an earlier BIOS, and upgrading is not an option a workaround is to install FreeBSD, change the partition ID FreeBSD uses, and install new boot blocks that can handle the different partition ID. First, you will need to restore the machine to a state where it can get through its self-test screen. Doing this requires powering up the machine without letting it find a FreeBSD partition on its primary disk. One way is to remove the hard disk and temporarily move it to an older ThinkPad (such as a ThinkPad 600) or a desktop PC with an appropriate conversion cable. Once it is there, you can delete the FreeBSD partition and move the hard disk back. The ThinkPad should now be in a bootable state again. With the machine functional again, you can use the workaround procedure described here to get a working FreeBSD installation. Download boot1 and boot2 from http://people.freebsd.org/~bmah/ThinkPad/. Put these files somewhere you will be able to retrieve them later. Install FreeBSD as normal on to the ThinkPad. Do not use Dangerously Dedicated mode. Do not reboot when the install has finished. Either switch to the Emergency Holographic Shell (ALT F4) or start a fixit shell. Use &man.fdisk.8; to change the FreeBSD partition ID from 165 to 166 (this is the type used by OpenBSD). Bring the boot1 and boot2 files to the local filesystem. Use &man.disklabel.8; to write boot1 and boot2 to your FreeBSD slice. &prompt.root; disklabel -B -b boot1 -s boot2 ad0sn n is the number of the slice where you installed FreeBSD. Reboot. At the boot prompt you will be given the option of booting OpenBSD. This will actually boot FreeBSD. Getting this to work in the case where you want to dual boot OpenBSD and FreeBSD on the same laptop is left as an exercise for the reader. Can I install on a disk with bad blocks? Prior to 3.0, FreeBSD included a utility known as bad144, which automatically remapped bad blocks. Because modern IDE drives perform this function themselves, bad144 has been removed from the FreeBSD source tree. If you wish to install FreeBSD 3.0 or later, we strongly suggest you purchase a newer disk drive. If you do not wish to do this, you must run FreeBSD 2.x. If you are seeing bad block errors with a modern IDE drive, chances are the drive is going to die very soon (the drive's internal remapping functions are no longer sufficient to fix the bad blocks, which means the disk is heavily corrupted); we suggest you by a new hard drive. If you have a SCSI drive with bad blocks, see this answer. Strange things happen when I boot the install floppy! What is happening? If you are seeing things like the machine grinding to a halt or spontaneously rebooting when you try to boot the install floppy, here are three questions to ask yourself:- Did you use a new, freshly-formatted, error-free floppy (preferably a brand-new one straight out of the box, as opposed to the magazine coverdisk that has been lying under the bed for the last three years)? Did you download the floppy image in binary (or image) mode? (do not be embarrassed, even the best of us have accidentally downloaded a binary file in ASCII mode at least once!) If you are using Windows95 or Win98 did you run fdimage or rawrite in pure DOS mode? These OS's can interfere with programs that write directly to hardware, which the disk creation program does; even running it inside a DOS shell in the GUI can cause this problem. There have also been reports of Netscape causing problems when downloading the boot floppy, so it is probably best to use a different FTP client if you can. I booted from my ATAPI CD-ROM, but the install program says no CD-ROM is found. Where did it go? The usual cause of this problem is a mis-configured CD-ROM drive. Many PCs now ship with the CD-ROM as the slave device on the secondary IDE controller, with no master device on that controller. This is illegal according to the ATAPI specification, but Windows plays fast and loose with the specification, and the BIOS ignores it when booting. This is why the BIOS was able to see the CD-ROM to boot from it, but why FreeBSD can not see it to complete the install. Reconfigure your system so that the CD-ROM is either the master device on the IDE controller it is attached to, or make sure that it is the slave on an IDE controller that also has a master device. Why can I not install from tape? If you are installing 2.1.7R from tape, you must create the tape using a tar blocksize of 10 (5120 bytes). The default tar blocksize is 20 (10240 bytes), and tapes created using this default size cannot be used to install 2.1.7R; with these tapes, you will get an error that complains about the record size being too big. Can I install on my laptop over PLIP (Parallel Line IP)? Connect the two computers using a Laplink parallel cable to use this feature: Wiring a parallel cable for networking A-name A-End B-End Descr. Post/Bit DATA0 -ERROR 2 15 15 2 Data 0/0x01 1/0x08 DATA1 +SLCT 3 13 13 3 Data 0/0x02 1/0x10 DATA2 +PE 4 12 12 4 Data 0/0x04 1/0x20 DATA3 -ACK 5 10 10 5 Strobe 0/0x08 1/0x40 DATA4 BUSY 6 11 11 6 Data 0/0x10 1/0x80 GND 18-25 18-25 GND -
See also this note on the Mobile Computing page. Which geometry should I use for a disk drive? By the geometry of a disk, we mean the number of cylinders, heads and sectors/track on a disk - I will refer to this as C/H/S for convenience. This is how the PC's BIOS works out which area on a disk to read/write from. This seems to cause a lot of confusion for some reason. First of all, the physical geometry of a SCSI drive is totally irrelevant, as FreeBSD works in term of disk blocks. In fact, there is no such thing as the physical geometry, as the sector density varies across the disk - what manufacturers claim is the quote physical geometry is usually the geometry that they have worked out results in the least wasted space. For IDE disks, FreeBSD does work in terms of C/H/S, but all modern drives will convert this into block references internally as well. All that matters is the logical geometry - the answer that the BIOS gets when it asks what is your geometry? and then uses to access the disk. As FreeBSD uses the BIOS when booting, it is very important to get this right. In particular, if you have more than one operating system on a disk, they must all agree on the geometry, otherwise you will have serious problems booting! For SCSI disks, the geometry to use depends on whether extended translation support is turned on in your controller (this is often referred to as support for DOS disks >1GB or something similar). If it is turned off, then use N cylinders, 64 heads and 32 sectors/track, where N is the capacity of the disk in MB. For example, a 2GB disk should pretend to have 2048 cylinders, 64 heads and 32 sectors/track. If it is turned on (it is often supplied this way to get around certain limitations in MSDOS) and the disk capacity is more than 1GB, use M cylinders, 63 sectors per track (*not* 64), and 255 heads, where 'M' is the disk capacity in MB divided by 7.844238 (!). So our example 2GB drive would have 261 cylinders, 63 sectors per track and 255 heads. If you are not sure about this, or FreeBSD fails to detect the geometry correctly during installation, the simplest way around this is usually to create a small DOS partition on the disk. The correct geometry should then be detected (and you can always remove the DOS partition in the partition editor if you do not want to keep it, or leave it around for programming network cards and the like). Alternatively, there is a freely available utility distributed with FreeBSD called pfdisk.exe (located in the tools subdirectory on the FreeBSD CDROM or on the various FreeBSD ftp sites) which can be used to work out what geometry the other operating systems on the disk are using. You can then enter this geometry in the partition editor. Are there any restrictions on how I divide the disk up? Yes. You must make sure that your root partition is below 1024 cylinders so the BIOS can boot the kernel from it. (Note that this is a limitation in the PC's BIOS, not FreeBSD). For a SCSI drive, this will normally imply that the root partition will be in the first 1024MB (or in the first 4096MB if extended translation is turned on - see previous question). For IDE, the corresponding figure is 504MB. Is FreeBSD compatible with any disk managers? FreeBSD recognizes the Ontrack Disk Manager and makes allowances for it. Other disk managers are not supported. If you just want to use the disk with FreeBSD you do not need a disk manager. Just configure the disk for as much space as the BIOS can deal with (usually 504 megabytes), and FreeBSD should figure out how much space you really have. If you are using an old disk with an MFM controller, you may need to explicitly tell FreeBSD how many cylinders to use. If you want to use the disk with FreeBSD and another operating system, you may be able to do without a disk manager: just make sure the FreeBSD boot partition and the slice for the other operating system are in the first 1024 cylinders. If you are reasonably careful, a 20 megabyte boot partition should be plenty. When I boot FreeBSD I get Missing Operating System. What is happening? This is classically a case of FreeBSD and DOS or some other OS conflicting over their ideas of disk geometry. You will have to reinstall FreeBSD, but obeying the instructions given above will almost always get you going. Why can I not get past the boot manager's F? prompt? This is another symptom of the problem described in the preceding question. Your BIOS geometry and FreeBSD geometry settings do not agree! If your controller or BIOS supports cylinder translation (often marked as >1GB drive support), try toggling its setting and reinstalling FreeBSD. Do I need to install the complete sources? In general, no. However, we would strongly recommend that you install, at a minimum, the base source kit, which includes several of the files mentioned here, and the sys (kernel) source kit, which includes sources for the kernel. There is nothing in the system which requires the presence of the sources to operate, however, except for the kernel-configuration program &man.config.8;. With the exception of the kernel sources, our build structure is set up so that you can read-only mount the sources from elsewhere via NFS and still be able to make new binaries. (Because of the kernel-source restriction, we recommend that you not mount this on /usr/src directly, but rather in some other location with appropriate symbolic links to duplicate the top-level structure of the source tree.) Having the sources on-line and knowing how to build a system with them will make it much easier for you to upgrade to future releases of FreeBSD. To actually select a subset of the sources, use the Custom menu item when you are in the Distributions menu of the system installation tool. Do I need to build a kernel? Building a new kernel was originally pretty much a required step in a FreeBSD installation, but more recent releases have benefited from the introduction of a much friendlier kernel configuration tool. When at the FreeBSD boot prompt (boot:), use the flag and you will be dropped into a visual configuration screen which allows you to configure the kernel's settings for most common ISA cards. It is still recommended that you eventually build a new kernel containing just the drivers that you need, just to save a bit of RAM, but it is no longer a strict requirement for most systems. Should I use DES passwords, or MD5, and how do I specify which form my users receive? The default password format on FreeBSD is to use MD5-based passwords. These are believed to be more secure than the traditional UNIX password format, which used a scheme based on the DES algorithm. DES passwords are still available if you need to share your password file with legacy operating systems which still use the less secure password format (they are available if you choose to install the crypto distribution in sysinstall, or by installing the crypto sources if building from source). Which password format to use for new passwords is controlled by the passwd_format login capability in /etc/login.conf, which takes values of either des (if available) or md5. See the &man.login.conf.5; manpage for more information about login capabilities. Why does the boot floppy start, but hang at the Probing Devices... screen? If you have a IDE Zip or Jaz drive installed, remove it and try again. The boot floppy can get confused by the drives. After the system is installed you can reconnect the drive. Hopefully this will be fixed in a later release. Why do I get a panic: can't mount root error when rebooting the system after installation? This error comes from confusion between the boot block's and the kernel's understanding of the disk devices. The error usually manifests on two-disk IDE systems, with the hard disks arranged as the master or single device on separate IDE controllers, with FreeBSD installed on the secondary IDE controller. The boot blocks think the system is installed on wd1 (the second BIOS disk) while the kernel assigns the first disk on the secondary controller device wd2. After the device probing, the kernel tries to mount what the boot blocks think is the boot disk, wd1, while it is really wd2, and fails. To fix the problem, do one of the following: For FreeBSD 3.3 and later, reboot the system and hit Enter at the Booting kernel in 10 seconds; hit [Enter] to interrupt prompt. This will drop you into the boot loader. Then type set root_disk_unit="disk_number" . disk_number will be 0 if FreeBSD is installed on the master drive on the first IDE controller, 1 if it is installed on the slave on the first IDE controller, 2 if it is installed on the master of the second IDE controller, and 3 if it is installed on the slave of the second IDE controller. Then type boot, and your system should boot correctly. To make this change permanent (ie so you do not have to do this every time you reboot or turn on your FreeBSD machine), put the line root_disk_unit="disk_number" in /boot/loader.conf.local . If using FreeBSD 3.2 or earlier, at the Boot: prompt, enter 1:wd(2,a)kernel and press Enter. If the system starts, then run the command echo "1:wd(2,a)kernel" > /boot.config to make it the default boot string. Move the FreeBSD disk onto the primary IDE controller, so the hard disks are consecutive. Rebuild your kernel, modify the wd configuration lines to read: controller wdc0 at isa? port "IO_WD1" bio irq 14 vector wdintr disk wd0 at wdc0 drive 0 # disk wd1 at wdc0 drive 1 # comment out this line controller wdc1 at isa? port "IO_WD2" bio irq 15 vector wdintr disk wd1 at wdc1 drive 0 # change from wd2 to wd1 disk wd2 at wdc1 drive 1 # change from wd3 to wd2 Install the new kernel. If you moved your disks and wish to restore the previous configuration, replace the disks in the desired configuration and reboot. Your system should boot successfully. What are the limits for memory? For memory, the limit is 4 gigabytes. This configuration has been tested, see wcarchive's configuration for more details. If you plan to install this much memory into a machine, you need to be careful. You will probably want to use ECC memory and to reduce capacitive loading use 9 chip memory modules vice 18 chip memory modules. What are the limits for ffs filesystems? For ffs filesystems, the maximum theoretical limit is 8 terabytes (2G blocks), or 16TB for the default block size of 8K. In practice, there is a soft limit of 1 terabyte, but with modifications filesystems with 4 terabytes are possible (and exist). The maximum size of a single ffs file is approximately 1G blocks (4TB) if the block size is 4K. Maximum file sizes fs block size 2.2.7-stable 3.0-current works should work 4K 4T-1 4T-1 4T-1 >4T 8K >32G 8T-1 >32G 32T-1 16K >128G 16T-1 >128G 32T-1 32K >512G 32T-1 >512G 64T-1 64K >2048G 64T-1 >2048G 128T-1
When the fs block size is 4K, triple indirect blocks work and everything should be limited by the maximum fs block number that can be represented using triple indirect blocks (approx. 1K^3 + 1K^2 + 1K), but everything is limited by a (wrong) limit of 1G-1 on fs block numbers. The limit on fs block numbers should be 2G-1. There are some bugs for fs block numbers near 2G-1, but such block numbers are unreachable when the fs block size is 4K. For block sizes of 8K and larger, everything should be limited by the 2G-1 limit on fs block numbers, but is actually limited by the 1G-1 limit on fs block numbers, except under -STABLE triple indirect blocks are unreachable, so the limit is the maxiumum fs block number that can be represented using double indirect blocks (approx. (blocksize/4)^2 + (blocksize/4)), and under -CURRENT exceeding this limit may cause problems. Using the correct limit of 2G-1 blocks does cause problems. How can I put 1TB files on my floppy? I keep several virtual ones on floppies :-). The maxiumum file size is not closely related to the maximum disk size. The maximum disk size is 1TB. It is a feature that the file size can be larger than the disk size. The following example creates a file of size 8T-1 using a whole 32K of disk space (3 indirect blocks and 1 data block) on a small root partition. The dd command requires a dd that works with large files. &prompt.user; cat foo df . dd if=/dev/zero of=z bs=1 seek=`echo 2^43 - 2 | bc` count=1 ls -l z du z df . &prompt.user; sh foo Filesystem 1024-blocks Used Avail Capacity Mounted on /dev/da0a 64479 27702 31619 47% / 1+0 records in 1+0 records out 1 bytes transferred in 0.000187 secs (5346 bytes/sec) -rw-r--r-- 1 bde bin 8796093022207 Sep 7 16:04 z 32 z Filesystem 1024-blocks Used Avail Capacity Mounted on /dev/da0a 64479 27734 31587 47% / Bruce Evans, September 1998 Why do I get an error message, archsw.readin.failed after compiling and booting a new kernel? You can boot by specifying the kernel directly at the second stage, pressing any key when the | shows up before loader is started. More specifically, you have upgraded the source for your kernel, and installed a new kernel builtin from them without making world. This is not supported. Make world. How do I upgrade from 3.X -> 4.X? We strongly recommend that you use binary snapshots to do this. 4-STABLE snapshots are available at releng4.FreeBSD.org. If you wish to upgrade using source, please see the FreeBSD Handbook for more information. Upgrading via source is never recommended for new users, and upgrading from 3.X to 4.X is even less so; make sure you have read the instructions carefully before attempting to upgrade via source. What are these security profiles? A security profile is a set of configuration options that attempts to achieve the desired ratio of security to convenience by enabling and disabling certain programs and other settings. The more severe the security profile, the less programs will be enabled by default; this is one of the basic principles of security: do not run anything except what you must. Please note that the security profile is just a default setting. All programs can be enabled and disabled after you have installed FreeBSD by editing or adding the appropriate line(s) to /etc/rc.conf. For more information on the latter, please see the &man.rc.conf.5; manual page. Following is a table that describes what each security profile does. The columns are the choices you have for a security profile, and the rows are the program or feature that is enabled or disabled. Possible security profiles Extreme High Moderate Low &man.inetd.8; NO NO YES YES &man.sendmail.8; NO YES YES YES &man.sshd.8; NO YES YES YES &man.portmap.8; NO NO MAYBE The portmapper is enabled if the machine has been configured as an NFS client or server earlier in the installation. YES NFS server NO NO YES YES &man.securelevel.8; YES (2) If you choose a security profile that sets the securelevel (Extreme or High), you must be aware of the implications. Please read the &man.init.8; manual page and pay particular attention to the meanings of the security levels, or you may have significant trouble later! YES (1) NO NO
The security profile is not a silver bullet! Setting it high does not mean you do not have to keep up with security issues by reading an appropriate mailing list, using good passwords and passphrases, and generally adhering to good security practices. It simply sets up the desired security to convenience ration out of the box. The security profile mechanism is meant to be used when you first install FreeBSD. If you already have FreeBSD installed, it would probably be more beneficial to simply enable or disable the desired functionality. If you really want to use a security profile, you can re-run &man.sysinstall.8; to set it. Hardware compatibility Does FreeBSD support architectures other than the x86? Yes. FreeBSD currently runs on both Intel x86 and DEC (now Compaq) Alpha architectures. Interest has also been expressed in a port of FreeBSD to the SPARC architecture, join the freebsd-sparc@FreeBSD.org mailing list if you are interested in joining that project. Most recent additions to the list of upcoming platforms are IA-64 and PowerPC, join the freebsd-ia64@FreeBSD.org and/or freebsd-ppc@FreeBSD.org mailing lists for more information. For general discussion on new architectures, join the freebsd-platforms@FreeBSD.org mailing list. If your machine has a different architecture and you need something right now, we suggest you look at NetBSD or OpenBSD. What kind of hard drives does FreeBSD support? FreeBSD supports EIDE and SCSI drives (with a compatible controller; see the next section), and all drives using the original Western Digital interface (MFM, RLL, ESDI, and of course IDE). A few ESDI controllers that use proprietary interfaces may not work: stick to WD1002/3/6/7 interfaces and clones. Which SCSI controllers are supported? See the complete list in the Handbook. Which CD-ROM drives are supported by FreeBSD? Any SCSI drive connected to a supported controller is supported. The following proprietary CD-ROM interfaces are also supported: Mitsumi LU002 (8bit), LU005 (16bit) and FX001D (16bit 2x Speed). Sony CDU 31/33A Sound Blaster Non-SCSI CD-ROM Matsushita/Panasonic CD-ROM ATAPI compatible IDE CD-ROMs All non-SCSI cards are known to be extremely slow compared to SCSI drives, and some ATAPI CDROMs may not work. As of 2.2 the FreeBSD CDROM from the FreeBSD Mall supports booting directly from the CD. Which CD-RW drives are supported by FreeBSD? FreeBSD supports any ATAPI-compatible IDE CD-R or CD-RW drive. For FreeBSD versions 4.0 and later, see the man page for &man.burncd.8;. For earlier FreeBSD versions, see the examples in /usr/share/examples/atapi. FreeBSD also supports any SCSI CD-R or CD-RW drives. Install and use the cdrecord command from the ports or packages system, and make sure that you have the pass device compiled in your kernel. Does FreeBSD support ZIP drives? FreeBSD supports the SCSI ZIP drive out of the box, of course. The ZIP drive can only be set to run at SCSI target IDs 5 or 6, but if your SCSI host adapter's BIOS supports it you can even boot from it. It is not clear which host adapters support booting from targets other than 0 or 1, so you will have to consult your adapter's documentation if you would like to use this feature. ATAPI (IDE) Zip drives are supported in FreeBSD 2.2.6 and later releases. FreeBSD has contained support for Parallel Port Zip Drives since version 3.0. If you are using a sufficiently up to date version, then you should check that your kernel contains the scbus0, da0, ppbus0, and vp0 drivers (the GENERIC kernel contains everything except vp0). With all these drivers present, the Parallel Port drive should be available as /dev/da0s4. Disks can be mounted using mount /dev/da0s4 /mnt OR (for dos disks) mount_msdos /dev/da0s4 /mnt as appropriate. Also check out this note on removable drives, and this note on formatting. Does FreeBSD support JAZ, EZ and other removable drives? Apart from the IDE version of the EZ drive, these are all SCSI devices, so the should all look like SCSI disks to FreeBSD, and the IDE EZ should look like an IDE drive. I am not sure how well FreeBSD supports changing the media out while running. You will of course need to dismount the drive before swapping media, and make sure that any external units are powered on when you boot the system so FreeBSD can see them. See this note on formatting. Which multi-port serial cards are supported by FreeBSD? There is a list of these in the Miscellaneous devices section of the handbook. Some unnamed clone cards have also been known to work, especially those that claim to be AST compatible. Check the &man.sio.4; man page to get more information on configuring such cards. Does FreeBSD support my USB keyboard? USB device support was added to FreeBSD 3.1. However, it is still in preliminary state and may not always work as of version 3.2. If you want to experiment with the USB keyboard support, follow the procedure described below. Use FreeBSD 3.2 or later. Add the following lines to your kernel configuration file, and rebuild the kernel. device uhci device ohci device usb device ukbd options KBD_INSTALL_CDEV In versions of FreeBSD before 4.0, use this instead: controller uhci0 controller ohci0 controller usb0 controller ukbd0 options KBD_INSTALL_CDEV Go to the /dev directory and create device nodes as follows: &prompt.root; cd /dev &prompt.root; ./MAKEDEV kbd0 kbd1 Edit /etc/rc.conf and add the following lines: usbd_enable="YES" usbd_flags="" After the system is rebooted, the AT keyboard becomes /dev/kbd0 and the USB keyboard becomes /dev/kbd1, if both are connected to the system. If there is the USB keyboard only, it will be /dev/ukbd0. If you want to use the USB keyboard in the console, you have to explicitly tell the console driver to use the existence of the USB keyboard. This can be done by running the following command as a part of system initialization. &prompt.root; kbdcontrol -k /dev/kbd1 < /dev/ttyv0 > /dev/null Note that if the USB keyboard is the only keyboard, it is accessed as /dev/kbd0, thus, the command should look like: &prompt.root; kbdcontrol -k /dev/kbd0 < /dev/ttyv0 > /dev/null /etc/rc.i386 is a good place to add the above command. Once this is done, the USB keyboard should work in the X environment as well without any special settings. Hot-plugging and unplugging of the USB keyboard may not work quite right yet. It is a good idea to connect the keyboard before you start the system and leave it connected until the system is shutdown to avoid troubles. See the &man.ukbd.4; man page for more information. I have an unusual bus mouse. How do I set it up? FreeBSD supports the bus mouse and the InPort bus mouse from such manufactures as Microsoft, Logitech and ATI. The bus device driver is compiled in the GENERIC kernel by default in FreeBSD versions 2.X, but not included in version 3.0 or later. If you are building a custom kernel with the bus mouse driver, make sure to add the following line to the kernel config file In FreeBSD 3.0 or before, add: device mse0 at isa? port 0x23c tty irq5 vector mseintr In FreeBSD 3.X, the line should be: device mse0 at isa? port 0x23c tty irq5 And in FreeBSD 4.X and later, the line should read: device mse0 at isa? port 0x23c irq5 Bus mice usually comes with dedicated interface cards. These cards may allow you to set the port address and the IRQ number other than shown above. Refer to the manual of your mouse and the &man.mse.4; man page for more information. How do I use my PS/2 (mouse port or keyboard) mouse? If you are running a post-2.2.5 version of FreeBSD, the necessary driver, psm, is included and enabled in the kernel. The kernel should detect your PS/2 mouse at boot time. If you are running a previous but relatively recent version of FreeBSD (2.1.x or better) then you can simply enable it in the kernel configuration menu at installation time, otherwise later with at the boot: prompt. It is disabled by default, so you will need to enable it explicitly. If you are running an older version of FreeBSD then you will have to add the following lines to your kernel configuration file and compile a new kernel. In FreeBSD 3.0 or earlier, the line should be: device psm0 at isa? port "IO_KBD" conflicts tty irq 12 vector psmintr In FreeBSD 3.1 or later, the line should be: device psm0 at isa? tty irq 12 In FreeBSD 4.0 or later, the line should be: device psm0 at atkbdc? irq 12 See the Handbook entry on configuring the kernel if you have no experience with building kernels. Once you have a kernel detecting psm0 correctly at boot time, make sure that an entry for psm0 exists in /dev. You can do this by typing: &prompt.root; cd /dev; sh MAKEDEV psm0 when logged in as root. Is it possible to make use of a mouse in any way outside the X Window system? If you are using the default console driver, syscons, you can use a mouse pointer in text consoles to cut & paste text. Run the mouse daemon, moused, and turn on the mouse pointer in the virtual console: &prompt.root; moused -p /dev/xxxx -t yyyy &prompt.root; vidcontrol -m on Where xxxx is the mouse device name and yyyy is a protocol type for the mouse. See the &man.moused.8; man page for supported protocol types. You may wish to run the mouse daemon automatically when the system starts. In version 2.2.1, set the following variables in /etc/sysconfig. mousedtype="yyyy" mousedport="xxxx" mousedflags="" In versions 2.2.2 to 3.0, set the following variables in /etc/rc.conf. moused_type="yyyy" moused_port="xxxx" moused_flags="" In 3.1 and later, assuming you have a PS/2 mouse, all you need to is add moused_enable="YES" to /etc/rc.conf. In addition, if you would like to be able to use the mouse daemon on all virtual terminals instead of just console at boot-time, add the following to /etc/rc.conf. allscreens_flags="-m on" Staring from FreeBSD 2.2.6, the mouse daemon is capable of determining the correct protocol type automatically unless the mouse is a relatively old serial mouse model. Specify auto the protocol to invoke automatic detection. When the mouse daemon is running, access to the mouse needs to be coordinated between the mouse daemon and other programs such as the X Window. Refer to another section on this issue. How do I cut and paste text with mouse in the text console? Once you get the mouse daemon running (see previous section), hold down the button 1 (left button) and move the mouse to select a region of text. Then, press the button 2 (middle button) or the button 3 (right button) to paste it at the text cursor. In versions 2.2.6 and later, pressing the button 2 will paste the text. Pressing the button 3 will extend the selected region of text. If your mouse does not have the middle button, you may wish to emulate it or remap buttons using moused options. See the &man.moused.8; man page for details. Does FreeBSD support any USB mice? USB device support was added to FreeBSD 3.1. However, it is still in a preliminary state and may not always work as of version 3.2. If you want to experiment with the USB mouse support, follow the procedure described below. Use FreeBSD 3.2 or later. Add the following lines to your kernel configuration file, and rebuild the kernel. device uhci device ohci device usb device ums In versions of FreeBSD before 4.0, use this instead: controller uhci0 controller ohci0 controller usb0 device ums0 Go to the /dev directory and create a device node as follows: &prompt.root; cd /dev &prompt.root; ./MAKEDEV ums0 Edit /etc/rc.conf and add the following lines: moused_enable="YES" moused_type="auto" moused_port="/dev/ums0" moused_flags="" usbd_enable="YES" usbd_flags="" See the previous section for more detailed discussion on moused. In order to use the USB mouse in the X session, edit XF86Config. If you are using XFree86 3.3.2 or later, be sure to have the following lines in the Pointer section: Device "/dev/sysmouse" Protocol "Auto" If you are using earlier versions of XFree86, be sure to have the following lines in the Pointer section: Device "/dev/sysmouse" Protocol "SysMouse" Refer to another section on the mouse support in the X environment. Hot-plugging and unplugging of the USB mouse may not work quite right yet. It is a good idea connect the mouse before you start the system and leave it connected until the system is shutdown to avoid trouble. My mouse has a fancy wheel and buttons. Can I use them in FreeBSD? The answer is, unfortunately, It depends. These mice with additional features require specialized driver in most cases. Unless the mouse device driver or the user program has specific support for the mouse, it will act just like a standard two, or three button mouse. For the possible usage of wheels in the X Window environment, refer to that section. Why does my wheel-equipped PS/2 mouse cause my mouse cursor to jump around the screen? The PS/2 mouse driver psm in FreeBSD versions 3.2 or earlier has difficulty with some wheel mice, including Logitech model M-S48 and its OEM siblings. Apply the following patch to /sys/i386/isa/psm.c and rebuild the kernel. Index: psm.c =================================================================== RCS file: /src/CVS/src/sys/i386/isa/Attic/psm.c,v retrieving revision 1.60.2.1 retrieving revision 1.60.2.2 diff -u -r1.60.2.1 -r1.60.2.2 --- psm.c 1999/06/03 12:41:13 1.60.2.1 +++ psm.c 1999/07/12 13:40:52 1.60.2.2 @@ -959,14 +959,28 @@ sc->mode.packetsize = vendortype[i].packetsize; /* set mouse parameters */ +#if 0 + /* + * A version of Logitech FirstMouse+ won't report wheel movement, + * if SET_DEFAULTS is sent... Don't use this command. + * This fix was found by Takashi Nishida. + */ i = send_aux_command(sc->kbdc, PSMC_SET_DEFAULTS); if (verbose >= 2) printf("psm%d: SET_DEFAULTS return code:%04x\n", unit, i); +#endif if (sc->config & PSM_CONFIG_RESOLUTION) { sc->mode.resolution = set_mouse_resolution(sc->kbdc, - (sc->config & PSM_CONFIG_RESOLUTION) - 1); + (sc->config & PSM_CONFIG_RESOLUTION) - 1); + } else if (sc->mode.resolution >= 0) { + sc->mode.resolution + = set_mouse_resolution(sc->kbdc, sc->dflt_mode.resolution); + } + if (sc->mode.rate > 0) { + sc->mode.rate = set_mouse_sampling_rate(sc->kbdc, sc->dflt_mode.rate); } + set_mouse_scaling(sc->kbdc, 1); /* request a data packet and extract sync. bits */ if (get_mouse_status(sc->kbdc, stat, 1, 3) < 3) { Versions later than 3.2 should be all right. How do I use the mouse/trackball/touchpad on my laptop? Please refer to the answer to the previous question. And check out this note on the Mobile Computing page. What types of tape drives are supported? FreeBSD supports SCSI and QIC-36 (with a QIC-02 interface). This includes 8-mm (aka Exabyte) and DAT drives. Some of the early 8-mm drives are not quite compatible with SCSI-2, and may not work well with FreeBSD. Does FreeBSD support tape changers? FreeBSD 2.2 supports SCSI changers using the &man.ch.4; device and the &man.chio.1; command. The details of how you actually control the changer can be found in the &man.chio.1; man page. If you are not using AMANDA or some other product that already understands changers, remember that they only know how to move a tape from one point to another, so you need to keep track of which slot a tape is in, and which slot the tape currently in the drive needs to go back to. Which sound cards are supported by FreeBSD? FreeBSD supports the SoundBlaster, SoundBlaster Pro, SoundBlaster 16, Pro Audio Spectrum 16, AdLib and Gravis UltraSound sound cards. There is also limited support for MPU-401 and compatible MIDI cards. Cards conforming to the Microsoft Sound System specification are also supported through the pcm driver. This is only for sound! This driver does not support CD-ROMs, SCSI or joysticks on these cards, except for the SoundBlaster. The SoundBlaster SCSI interface and some non-SCSI CDROMS are supported, but you cannot boot off this device. Workarounds for no sound from es1370 with pcm driver? You can run the following command every time the machine booted up: &prompt.root; mixer pcm 100 vol 100 cd 100 Which network cards does FreeBSD support? See the Ethernet cards section of the handbook for a more complete list. I do not have a math co-processor - is that bad? This will only affect 386/486SX/486SLC owners - other machines will have one built into the CPU. In general this will not cause any problems, but there are circumstances where you will take a hit, either in performance or accuracy of the math emulation code (see the section on FP emulation). In particular, drawing arcs in X will be VERY slow. It is highly recommended that you buy a math co-processor; it is well worth it. Some math co-processors are better than others. It pains us to say it, but nobody ever got fired for buying Intel. Unless you are sure it works with FreeBSD, beware of clones. What other devices does FreeBSD support? See the Handbook for the list of other devices supported. Does FreeBSD support power management on my laptop? FreeBSD supports APM on certain machines. Please look in the LINT kernel config file, searching for the APM keyword. Further information can be found in &man.apm.4;. Why does my Micron system hang at boot time? Certain Micron motherboards have a non-conforming PCI BIOS implementation that causes grief when FreeBSD boots because PCI devices do not get configured at their reported addresses. Disable the Plug and Play Operating System flag in the BIOS to work around this problem. More information can be found at http://cesdis.gsfc.nasa.gov/linux/drivers/vortex.html#micron Why does FreeBSD not recognize my Adaptec SCSI controller card? The newer AIC789x series Adaptec chips are supported under the CAM SCSI framework which made it's debut in 3.0. Patches against 2.2-STABLE are in ftp://ftp.FreeBSD.org/pub/FreeBSD/development/cam/. A CAM-enhanced boot floppy is available at http://people.FreeBSD.org/~abial/cam-boot/. In both cases read the README before beginning. How come FreeBSD cannot find my internal Plug & Play modem? You will need to add the modem's PnP ID to the PnP ID list in the serial driver. To enable Plug & Play support, compile a new kernel with controller pnp0 in the configuration file, then reboot the system. The kernel will print the PnP IDs of all the devices it finds. Copy the PnP ID from the modem to the table in /sys/i386/isa/sio.c, at about line 2777. Look for the string SUP1310 in the structure siopnp_ids[] to find the table. Build the kernel again, install, reboot, and your modem should be found. You may have to manually configure the PnP devices using the pnp command in the boot-time configuration with a command like pnp 1 0 enable os irq0 3 drq0 0 port0 0x2f8 to make the modem show. How do I get the boot: prompt to show on the serial console? Build a kernel with options COMCONSOLE. Create /boot.config and place as the only text in the file. Unplug the keyboard from the system. See /usr/src/sys/i386/boot/biosboot/README.serial for information. Why doesn't my 3Com PCI network card work with my Micron computer? Certain Micron motherboards have a non-conforming PCI BIOS implementation that does not configure PCI devices at the addresses reported. This causes grief when FreeBSD boots. To work around this problem, disable the Plug and Play Operating System flag in the BIOS. More information on this problem is available at URL: http://cesdis.gsfc.nasa.gov/linux/drivers/vortex.html#micron Does FreeBSD support Symmetric Multiprocessing (SMP)? SMP is supported in 3.0-STABLE and later releases only. SMP is not enabled in the GENERIC kernel, so you will have to recompile your kernel to enable SMP. Take a look at /sys/i386/conf/LINT to figure out what options to put in your kernel config file. The boot floppy hangs on a system with an ASUS K7V motherboard. How do I fix this? Go in to the BIOS setup and disable the boot virus protection. Troubleshooting What do I do when I have bad blocks on my hard drive? With SCSI drives, the drive should be capable of re-mapping these automatically. However, many drives are shipped with this feature disabled, for some mysterious reason... To enable this, you will need to edit the first device page mode, which can be done on FreeBSD by giving the command (as root) &prompt.root; scsi -f /dev/rsd0c -m 1 -e -P 3 and changing the values of AWRE and ARRE from 0 to 1:- AWRE (Auto Write Reallocation Enbld): 1 ARRE (Auto Read Reallocation Enbld): 1 The following paragraphs were submitted by Ted Mittelstaedt tedm@toybox.placo.com: For IDE drives, any bad block is usually a sign of potential trouble. All modern IDE drives come with internal bad-block remapping turned on. All IDE hard drive manufacturers today offer extensive warranties and will replace drives with bad blocks on them. If you still want to attempt to rescue an IDE drive with bad blocks, you can attempt to download the IDE drive manufacturer's IDE diagnostic program, and run this against the drive. Sometimes these programs can be set to force the drive electronics to rescan the drive for bad blocks and lock them out. For ESDI, RLL and MFM drives, bad blocks are a normal part of the drive and are no sign of trouble, generally. With a PC, the disk drive controller card and BIOS handle the task of locking out bad sectors. This is fine for operating systems like DOS that use BIOS code to access the disk. However, FreeBSD's disk driver does not go through BIOS, therefore a mechanism, bad144, exists that replaces this functionality. bad144 only works with the wd driver (which means it is not supported in FreeBSD 4.0), it is NOT able to be used with SCSI. bad144 works by entering all bad sectors found into a special file. One caveat with bad144 - the bad block special file is placed on the last track of the disk. As this file may possibly contain a listing for a bad sector that would occur near the beginning of the disk, where the /kernel file might be located, it therefore must be accessible to the bootstrap program that uses BIOS calls to read the kernel file. This means that the disk with bad144 used on it must not exceed 1024 cylinders, 16 heads, and 63 sectors. This places an effective limit of 500MB on a disk that is mapped with bad144. To use bad144, simply set the Bad Block scanning to ON in the FreeBSD fdisk screen during the initial install. This works up through FreeBSD 2.2.7. The disk must have less than 1024 cylinders. It is generally recommended that the disk drive has been in operation for at least 4 hours prior to this to allow for thermal expansion and track wandering. If the disk has more than 1024 cylinders (such as a large ESDI drive) the ESDI controller uses a special translation mode to make it work under DOS. The wd driver understands about these translation modes, IF you enter the translated geometry with the set geometry command in fdisk. You must also NOT use the dangerously dedicated mode of creating the FreeBSD partition, as this ignores the geometry. Also, even though fdisk will use your overridden geometry, it still knows the true size of the disk, and will attempt to create a too large FreeBSD partition. If the disk geometry is changed to the translated geometry, the partition MUST be manually created with the number of blocks. A quick trick to use is to set up the large ESDI disk with the ESDI controller, boot it with a DOS disk and format it with a DOS partition. Then, boot the FreeBSD install and in the fdisk screen, read off and write down the blocksize and block numbers for the DOS partition. Then, reset the geometry to the same that DOS uses, delete the DOS partition, and create a cooperative FreeBSD partition using the blocksize you recorded earlier. Then, set the partition bootable and turn on bad block scanning. During the actual install, bad144 will run first, before any filesystems are created. (you can view this with an Alt-F2) If it has any trouble creating the badsector file, you have set too large a disk geometry - reboot the system and start all over again (including repartitioning and reformatting with DOS). If remapping is enabled and you are seeing bad blocks, consider replacing the drive. The bad blocks will only get worse as time goes on. How come FreeBSD does not recognize my Bustek 742a EISA SCSI controller? This info is specific to the 742a but may also cover other Buslogic cards. (Bustek = Buslogic) There are 2 general versions of the 742a card. They are hardware revisions A-G, and revisions H - onwards. The revision letter is located after the Assembly number on the edge of the card. The 742a has 2 ROM chips on it, one is the BIOS chip and the other is the Firmware chip. FreeBSD does not care what version of BIOS chip you have but it does care about what version of firmware chip. Buslogic will send upgrade ROMS out if you call their tech support dept. The BIOS and Firmware chips are shipped as a matched pair. You must have the most current Firmware ROM in your adapter card for your hardware revision. The REV A-G cards can only accept BIOS/Firmware sets up to 2.41/2.21. The REV H- up cards can accept the most current BIOS/Firmware sets of 4.70/3.37. The difference between the firmware sets is that the 3.37 firmware supports round robin The Buslogic cards also have a serial number on them. If you have a old hardware revision card you can call the Buslogic RMA department and give them the serial number and attempt to exchange the card for a newer hardware revision. If the card is young enough they will do so. FreeBSD 2.1 only supports Firmware revisions 2.21 onward. If you have a Firmware revision older than this your card will not be recognized as a Buslogic card. It may be recognized as an Adaptec 1540, however. The early Buslogic firmware contains an AHA1540 emulation mode. This is not a good thing for an EISA card, however. If you have an old hardware revision card and you obtain the 2.21 firmware for it, you will need to check the position of jumper W1 to B-C, the default is A-B. How come FreeBSD does not detect my HP Netserver's SCSI controller? This is basically a known problem. The EISA on-board SCSI controller in the HP Netserver machines occupies EISA slot number 11, so all the true EISA slots are in front of it. Alas, the address space for EISA slots >= 10 collides with the address space assigned to PCI, and FreeBSD's auto-configuration currently cannot handle this situation very well. So now, the best you can do is to pretend there is no address range clash :), by bumping the kernel option EISA_SLOTS to a value of 12. Configure and compile a kernel, as described in the Handbook entry on configuring the kernel. Of course, this does present you with a chicken-and-egg problem when installing on such a machine. In order to work around this problem, a special hack is available inside UserConfig. Do not use the visual interface, but the plain command-line interface there. Simply type eisa 12 quit at the prompt, and install your system as usual. While it is recommended you compile and install a custom kernel anyway. Hopefully, future versions will have a proper fix for this problem. You can not use a dangerously dedicated disk with an HP Netserver. See this note for more info. What is going on with my CMD640 IDE controller? It is broken. It cannot handle commands on both channels simultaneously. There's a workaround available now and it is enabled automatically if your system uses this chip. For the details refer to the manual page of the disk driver (man 4 wd). If you are already running FreeBSD 2.2.1 or 2.2.2 with a CMD640 IDE controller and you want to use the second channel, build a new kernel with options "CMD640" enabled. This is the default for 2.2.5 and later. I keep seeing messages like ed1: timeout. What do these messages mean? This is usually caused by an interrupt conflict (e.g., two boards using the same IRQ). FreeBSD prior to 2.0.5R used to be tolerant of this, and the network driver would still function in the presence of IRQ conflicts. However, with 2.0.5R and later, IRQ conflicts are no longer tolerated. Boot with the -c option and change the ed0/de0/... entry to match your board. If you are using the BNC connector on your network card, you may also see device timeouts because of bad termination. To check this, attach a terminator directly to the NIC (with no cable) and see if the error messages go away. Some NE2000 compatible cards will give this error if there is no link on the UTP port or if the cable is disconnected. Why do I get Incorrect super block when mounting a CDROM? You have to tell &man.mount.8; the type of the device that you want to mount. By default, &man.mount.8; will assume the filesystem is of type ufs. You want to mount a CDROM filesystem, and you do this by specifying the option to &man.mount.8;. This does, of course, assume that the CDROM contains an ISO 9660 filesystem, which is what most CDROMs have. As of 1.1R, FreeBSD automatically understands the Rock Ridge (long filename) extensions as well. As an example, if you want to mount the CDROM device, /dev/cd0c, under /mnt, you would execute: &prompt.root; mount -t cd9660 /dev/cd0c /mnt Note that your device name (/dev/cd0c in this example) could be different, depending on the CDROM interface. Note that the option just causes the &man.mount.cd9660.8; command to be executed, and so the above example could be shortened to: &prompt.root; mount_cd9660 /dev/cd0c /mnt Why do I get Device not configured when mounting a CDROM? This generally means that there is no CDROM in the CDROM drive, or the drive is not visible on the bus. Feed the drive something, and/or check its master/slave status if it is IDE (ATAPI). It can take a couple of seconds for a CDROM drive to notice that it has been fed, so be patient. Sometimes a SCSI CD-ROM may be missed because it had not enough time to answer the bus reset. If you have a SCSI CD-ROM please try to add the following symbol into your kernel configuration file and recompile. options "SCSI_DELAY=15" Why do all non-English characters in filenames show up as ? on my CDs when mounted in FreeBSD? Most likely your CDROM uses the Joliet extension for storing information about files and directories. This extension specifies that all filenames are stored using Unicode two-byte characters. Currently, efforts are under way to introduce a generic Unicode interface into the FreeBSD kernel, but since that is not ready yet, the CD9660 driver does not have the ability to decode the characters in the filenames. As a temporary solution, starting with FreeBSD 4.3, a special hook has been added into the CD9660 driver to allow the user to load an appropriate conversion table on the fly. Modules for some of the common encodings are available via the sysutils/cd9660_unicode port. My printer is ridiculously slow. What can I do? If it is parallel, and the only problem is that it is terribly slow, try setting your printer port into polled mode: &prompt.root; lptcontrol -p Some newer HP printers are claimed not to work correctly in interrupt mode, apparently due to some (not yet exactly understood) timing problem. Why do my programs occasionally die with Signal 11 errors? Signal 11 errors are caused when your process has attempted to access memory which the operating system has not granted it access to. If something like this is happening at seemingly random intervals then you need to start investigating things very carefully. These problems can usually be attributed to either: If the problem is occurring only in a specific application that you are developing yourself it is probably a bug in your code. If it is a problem with part of the base FreeBSD system, it may also be buggy code, but more often than not these problems are found and fixed long before us general FAQ readers get to use these bits of code (that is what -current is for). In particular, a dead giveaway that this is *not* a FreeBSD bug is if you see the problem when you are compiling a program, but the activity that the compiler is carrying out changes each time. For example, suppose you are running make buildworld, and the compile fails while trying to compile ls.c in to ls.o. If you next run make buildworld again, and the compile fails in the same place then this is a broken build -- try updating your sources and try again. If the compile fails elsewhere then this is almost certainly hardware. What you should do: In the first case you can use a debugger e.g. gdb to find the point in the program which is attempting to access a bogus address and then fix it. In the second case you need to verify that it is not your hardware at fault. Common causes of this include: Your hard disks might be overheating: Check the fans in your case are still working, as your disk (and perhaps other hardware might be overheating). The processor running is overheating: This might be because the processor has been overclocked, or the fan on the processor might have died. In either case you need to ensure that you have hardware running at what it is specified to run at, at least while trying to solve this problem. i.e. Clock it back to the default settings. If you are overclocking then note that it is far cheaper to have a slow system than a fried system that needs replacing! Also the wider community is not often sympathetic to problems on overclocked systems, whether you believe it is safe or not. Dodgy memory: If you have multiple memory SIMMS/DIMMS installed then pull them all out and try running the machine with each SIMM or DIMM individually and narrow the problem down to either the problematic DIMM/SIMM or perhaps even a combination. Over-optimistic Motherboard settings: In your BIOS settings, and some motherboard jumpers you have options to set various timings, mostly the defaults will be sufficient, but sometimes, setting the wait states on RAM too low, or setting the RAM Speed: Turbo option, or similar in the BIOS will cause strange behaviour. A possible idea is to set to BIOS defaults, but it might be worth noting down your settings first! Unclean or insufficient power to the motherboard. If you have any unused I/O boards, hard disks, or CDROMs in your system, try temporarily removing them or disconnecting the power cable from them, to see if your power supply can manage a smaller load. Or try another power supply, preferably one with a little more power (for instance, if your current power supply is rated at 250 Watts try one rated at 300 Watts). You should also read the SIG11 FAQ (listed below) which has excellent explanations of all these problems, albeit from a Linux viewpoint. It also discusses how memory testing software or hardware can still pass faulty memory. Finally, if none of this has helped it is possible that you have just found a bug in FreeBSD, and you should follow the instructions to send a problem report. There is an extensive FAQ on this at the SIG11 problem FAQ Why does the screen go black and lose sync when I boot? This is a known problem with the ATI Mach 64 video card. The problem is that this card uses address 2e8, and the fourth serial port does too. Due to a bug (feature?) in the &man.sio.4; driver it will touch this port even if you do not have the fourth serial port, and even if you disable sio3 (the fourth port) which normally uses this address. Until the bug has been fixed, you can use this workaround: Enter at the boot prompt. (This will put the kernel into configuration mode). Disable sio0, sio1, sio2 and sio3 (all of them). This way the sio driver does not get activated -> no problems. Type exit to continue booting. If you want to be able to use your serial ports, you will have to build a new kernel with the following modification: in /usr/src/sys/i386/isa/sio.c find the one occurrence of the string 0x2e8 and remove that string and the preceding comma (keep the trailing comma). Now follow the normal procedure of building a new kernel. Even after applying these workarounds, you may still find that the X Window System does not work properly. If this is the case, make sure that the XFree86 version you are using is at least XFree86 3.3.3 or higher. This version and upwards has built-in support for the Mach64 cards and even a dedicated X server for those cards. How come FreeBSD uses only 64 MB of RAM when my system has 128 MB of RAM installed? Due to the manner in which FreeBSD gets the memory size from the BIOS, it can only detect 16 bits worth of Kbytes in size (65535 Kbytes = 64MB) (or less... some BIOSes peg the memory size to 16M). If you have more than 64MB, FreeBSD will attempt to detect it; however, the attempt may fail. To work around this problem, you need to use the kernel option specified below. There is a way to get complete memory information from the BIOS, but we do not have room in the bootblocks to do it. Someday when lack of room in the bootblocks is fixed, we will use the extended BIOS functions to get the full memory information...but for now we are stuck with the kernel option. options "MAXMEM=n" Where n is your memory in Kilobytes. For a 128 MB machine, you would want to use 131072. Why does FreeBSD 2.0 panic with kmem_map too small!? The message may also be mb_map too small! The panic indicates that the system ran out of virtual memory for network buffers (specifically, mbuf clusters). You can increase the amount of VM available for mbuf clusters by adding: options "NMBCLUSTERS=n" to your kernel config file, where n is a number in the range 512-4096, depending on the number of concurrent TCP connections you need to support. I would recommend trying 2048 - this should get rid of the panic completely. You can monitor the number of mbuf clusters allocated/in use on the system with netstat -m (see &man.netstat.1;). The default value for NMBCLUSTERS is 512 + MAXUSERS * 16. Why do I get an error reading CMAP busy when rebooting with a new kernel? The logic that attempts to detect an out of date /var/db/kvm_*.db files sometimes fails and using a mismatched file can sometimes lead to panics. If this happens, reboot single-user and do: &prompt.root; rm /var/db/kvm_*.db What does the message ahc0: brkadrint, Illegal Host Access at seqaddr 0x0 mean? This is a conflict with an Ultrastor SCSI Host Adapter. During the boot process enter the kernel configuration menu and disable uha0, which is causing the problem. Why does Sendmail give me an error reading mail loops back to myself? This is answered in the sendmail FAQ as follows:- * I'm getting "Local configuration error" messages, such as: 553 relay.domain.net config error: mail loops back to myself 554 <user@domain.net>... Local configuration error How can I solve this problem? You have asked mail to the domain (e.g., domain.net) to be forwarded to a specific host (in this case, relay.domain.net) by using an MX record, but the relay machine doesn't recognize itself as domain.net. Add domain.net to /etc/sendmail.cw (if you are using FEATURE(use_cw_file)) or add "Cw domain.net" to /etc/sendmail.cf. The current version of the sendmail FAQ is no longer maintained with the sendmail release. It is however regularly posted to comp.mail.sendmail, comp.mail.misc, comp.mail.smail, comp.answers, and news.answers. You can also receive a copy via email by sending a message to mail-server@rtfm.mit.edu with the command send usenet/news.answers/mail/sendmail-faq as the body of the message. Why do full screen applications on remote machines misbehave? The remote machine may be setting your terminal type to something other than the cons25 terminal type required by the FreeBSD console. There are a number of possible work-arounds for this problem: After logging on to the remote machine, set your TERM shell variable to ansi or sco if the remote machine knows about these terminal types. Use a VT100 emulator like screen at the FreeBSD console. screen offers you the ability to run multiple concurrent sessions from one terminal, and is a neat program in its own right. Each screen window behaves like a VT100 terminal, so the TERM variable at the remote end should be set to vt100. Install the cons25 terminal database entry on the remote machine. The way to do this depends on the operating system on the remote machine. The system administration manuals for the remote system should be able to help you here. Fire up an X server at the FreeBSD end and login to the remote machine using an X based terminal emulator such as xterm or rxvt. The TERM variable at the remote host should be set to xterm or vt100. Why does my machine print calcru: negative time...? This can be caused by various hardware and/or software ailments relating to interrupts. It may be due to bugs but can also happen by nature of certain devices. Running TCP/IP over the parallel port using a large MTU is one good way to provoke this problem. Graphics accelerators can also get you here, in which case you should check the interrupt setting of the card first. A side effect of this problem are dying processes with the message SIGXCPU exceeded cpu time limit. For FreeBSD 3.0 and later from Nov 29, 1998 forward: If the problem cannot be fixed otherwise the solution is to set this sysctl variable: &prompt.root; sysctl -w kern.timecounter.method=1 This means a performance impact, but considering the cause of this problem, you probably will not notice. If the problem persists, keep the sysctl set to one and set the NTIMECOUNTER option in your kernel to increasingly large values. If by the time you have reached NTIMECOUNTER=20 the problem is not solved, interrupts are too hosed on your machine for reliable timekeeping. I see pcm0 not found or my sound card is found as pcm1 but I have device pcm0 in my kernel config file. What is going on? This occurs in FreeBSD 3.x with PCI sound cards. The pcm0 device is reserved exclusively for ISA-based cards so, if you have a PCI card, then you will see this error, and your card will appear as pcm1. You cannot remove the warning by simply changing the line in the kernel config file to device pcm1 as this will result in pcm1 being reserved for ISA cards and your PCI card being found as pcm2 (along with the warning pcm1 not found). If you have a PCI sound card you will also have to make the snd1 device rather than snd0: &prompt.root; cd /dev &prompt.root; ./MAKEDEV snd1 This situation does not arise in FreeBSD 4.x as has a lot of work has been done to make the it more PnP-centric and the pcm0 device is no longer reserved exclusively for ISA cards Why is my PnP card no longer found (or found as unknown) since upgrading to FreeBSD 4.x? FreeBSD 4.x is now much more PnP-centric and this has had the side effect of some PnP devices (e.g. sound cards and internal modems) not working even though they worked under FreeBSD 3.x. The reasons for this behaviour are explained by the following e-mail, posted to the freebsd-questions mailing list by Peter Wemm, in answer to a question about an internal modem that was no longer found after an upgrade to FreeBSD 4.x (the comments in [] have been added to clarify the context.
The PNP bios preconfigured it [the modem] and left it laying around in port space, so [in 3.x] the old-style ISA probes found it there. Under 4.0, the ISA code is much more PnP-centric. It was possible [in 3.x] for an ISA probe to find a stray device and then for the PNP device id to match and then fail due to resource conflicts. So, it disables the programmable cards first so this double probing cannot happen. It also means that it needs to know the PnP id's for supported PnP hardware. Making this more user tweakable is on the TODO list.
To get the device working again requires finding its PnP id and adding it to the list that the ISA probes use to identify PnP devices. This is obtained using &man.pnpinfo.8; to probe the device, for example this is the output from &man.pnpinfo.8; for an internal modem: &prompt.root; pnpinfo Checking for Plug-n-Play devices... Card assigned CSN #1 Vendor ID PMC2430 (0x3024a341), Serial Number 0xffffffff PnP Version 1.0, Vendor Version 0 Device Description: Pace 56 Voice Internal Plug & Play Modem Logical Device ID: PMC2430 0x3024a341 #0 Device supports I/O Range Check TAG Start DF I/O Range 0x3f8 .. 0x3f8, alignment 0x8, len 0x8 [16-bit addr] IRQ: 4 - only one type (true/edge) [more TAG lines elided] - -TAG End DF + TAG End DF End Tag Successfully got 31 resources, 1 logical fdevs -- card select # 0x0001 CSN PMC2430 (0x3024a341), Serial Number 0xffffffff Logical device #0 IO: 0x03e8 0x03e8 0x03e8 0x03e8 0x03e8 0x03e8 0x03e8 0x03e8 IRQ 5 0 DMA 4 0 IO range check 0x00 activate 0x01 The information you require is in the Vendor ID line at the start of the output. The hexadecimal number in parentheses (0x3024a341 in this example) is the PnP id and the string immediately before this (PMC2430) is a unique ASCII id. This information needs adding to the file /usr/src/sys/isa/sio.c. You should first make a backup of sio.c just in case things go wrong. You will also need it to make the patch to submit with your PR (you are going to submit a PR, aren't you?) then edit sio.c and search for the line static struct isa_pnp_id sio_ids[] = { then scroll down to find the correct place to add the entry for your device. The entries look like this, and are sorted on the ASCII Vendor ID string which should be included in the comment to the right of the line of code along with all (if it will fit) or part of the Device Description from the output of &man.pnpinfo.8;: {0x0f804f3f, NULL}, /* OZO800f - Zoom 2812 (56k Modem) */ {0x39804f3f, NULL}, /* OZO8039 - Zoom 56k flex */ {0x3024a341, NULL}, /* PMC2430 - Pace 56 Voice Internal Modem */ {0x1000eb49, NULL}, /* ROK0010 - Rockwell ? */ {0x5002734a, NULL}, /* RSS0250 - 5614Jx3(G) Internal Modem */ Add the hexadecimal Vendor ID for your device in the correct place, save the file, rebuild your kernel, and reboot. Your device should now be found as an sio device as it was under FreeBSD 3.x Why do I get the error nlist failed when running, for example, top or systat? The problem is that the application you are trying to run is looking for a specific kernel symbol, but, for whatever reason, cannot find it; this error stems from one of two problems: Your kernel and userland are not synchronized (i.e., you built a new kernel but did not do an installworld, or vice versa), and thus the symbol table is different from what the user application thinks it is. If this is the case, simply complete the upgrade process (see /usr/src/UPDATING for the correct sequence). You are not using /boot/loader to load your kernel, but doing it directly from boot2 (see &man.boot.8;). While there is nothing wrong with bypassing /boot/loader, it generally does a better job of making the kernel symbols available to user applications. Why does it take so long to connect to my computer via ssh or telnet? The symptom: there is a long delay between the time the TCP connection is established and the time when the client software asks for a password (or, in &man.telnet.1;'s case, when a login prompt appears). The problem: more likely than not, the delay is caused by the server software trying to resolve the client's IP address into a hostname. Many servers, including the Telnet and SSH servers that come with FreeBSD, do this in order to, among other things, store the hostname in a log file for future reference by the administrator. The remedy: if the problem occurs whenever you connect from your computer (the client) to any server, the problem is with the client; likewise, if the problem only occurs when someone connects to your computer (the server) the problem is with the server. If the problem is with the client, the only remedy is to fix the DNS so the server can resolve it. If this is on a local network, consider it a server problem and keep reading; conversely, if this is on the global Internet, you will most likely need to contact your ISP and ask them to fix it for you. If the problem is with the server, and this is on a local network, you need to configure the server to be able to resolve address-to-hostname queries for your local address range. See the &man.hosts.5; and &man.named.8; manual pages for more information. If this is on the global Internet, the problem may be that your server's resolver is not functioning correctly. To check, try to look up another host--say, www.yahoo.com. If it does not work, that is your problem. Why does file: table is full show up repeatedly in dmesg? This error is caused when you have exhausted the number of available file descriptors on your system. The file table in memory is full. The solution: Manually adjust the kern.maxfiles kernel limit setting. &prompt.root; sysctl -w kern.maxfiles=n Adjust n according to your system needs. Each open file, socket, or fifo uses one file descriptor. A large-scale server may easily require tens of thousands of file descriptors (10,000+), depending on the kind and number of services running concurrently. The number of default file descriptors set in the kernel is dictated by the maxusers 32 maxusers line in your kernel config file. Increasing this will proportionally increase kern.maxfiles. You can see what kern.maxfiles is currently set to by: &prompt.root; sysctl kern.maxfiles -kern.maxfiles: 1064 - +kern.maxfiles: 1064 Why does the clock on my laptop keep incorrect time? Your laptop has two or more clocks, and FreeBSD has chosen to use the wrong one. Run &man.dmesg.8;, and check for lines that contain Timecounter. The last line printed is the one that FreeBSD chose, and will almost certainly be TSC. &prompt.root; dmesg | grep Timecounter Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 595573479 Hz You can confirm this by checking the kern.timecounter.hardware &man.sysctl.3;. &prompt.root; sysctl kern.timecounter.hardware kern.timecounter.hardware: TSC The BIOS may modify the TSC clock—perhaps to change the speed of the processor when running from batteries, or going in to a power saving mode, but FreeBSD is unaware of these adjustments, and appears to gain or lose time. In this example, the i8254 clock is also available, and can be selected by writing its name to the kern.timecounter.hardware &man.sysctl.3;. &prompt.root; sysctl -w kern.timecounter.hardware=i8254 kern.timecounter.hardware: TSC -> i8254 Your laptop should now start keeping more accurate time. To have this change automatically run at boot time, add the following line to /etc/sysctl.conf. kern.timecounter.hardware=i8254 Why does FreeBSD's boot loader display Read error and stop after the BIOS screen? FreeBSD's boot loader is incorrectly recognizing the hard drive's geometry. This must be manually set within fdisk when creating or modifying FreeBSD's slice. The correct drive geometry values can be found within the machine's BIOS. Look for the number of cylinders, heads and sectors for the particular drive. Within &man.sysinstall.8;'s fdisk, hit G to set the drive geometry. A dialog will pop up requesting the number of cylinders, heads and sectors. Type the numbers found from the BIOS separates by forward slashes. 5000 cylinders, 250 sectors and 60 sectors would be entered as 5000/250/60 Press enter to set the values, and hit W to write the new partition table to the drive. Another operating system destroyed my Boot Manager. How do I get it back? Enter &man.sysinstall.8; and choose Configure, then Fdisk. Select the disk the Boot Manager resided on with the space key. Press W to write changes to the drive. A prompt will appear asking which boot loader to install. Select this, and it will be restored. Commercial Applications This section is still very sparse, though we are hoping, of course, that companies will add to it! :) The FreeBSD group has no financial interest in any of the companies listed here but simply lists them as a public service (and feels that commercial interest in FreeBSD can have very positive effects on FreeBSD's long-term viability). We encourage commercial software vendors to send their entries here for inclusion. See the Vendors page for a longer list. Where can I get an Office Suite for FreeBSD? The FreeBSD Mall offers a FreeBSD native version of VistaSource ApplixWare 5. ApplixWare is a rich full-featured, commercial Office Suite for FreeBSD containing a word processor, spreadsheet, presentation program, vector drawing package, and other applications. You can purchase ApplixWare for FreeBSD here. The Linux version of StarOffice works flawlessly on FreeBSD. The easiest way to install the Linux version of StarOffice is through the FreeBSD Ports collection. Future versions of the open-source OpenOffice suite should work as well. Where can I get Motif for FreeBSD? The Open Group has released the source code to Motif 2.1.30. You can install the open-motif package, or compile it from ports. Refer to the ports section of the Handbook for more information on how to do this. The Open Motif distribution only allows redistribution if it is running on an open source operating system. In addition, there are commercial distributions of the Motif software available. These, however, are not for free, but their license allows them to be used in closed-source software. Contact Apps2go for the least expensive ELF Motif 2.1.20 distribution for FreeBSD (either i386 or Alpha). There are two distributions, the developement edition and the runtime edition (for much less). These distributions includes: OSF/Motif manager, xmbind, panner, wsm. Development kit with uil, mrm, xm, xmcxx, include and Imake files. Static and dynamic ELF libraries (for use with FreeBSD 3.0 and above). Demonstration applets. Be sure to specify that you want the FreeBSD version of Motif when ordering (do not forget to mention the architecture you want too)! Versions for NetBSD and OpenBSD are also sold by Apps2go. This is currently a FTP only download. More info Apps2go WWW page or sales@apps2go.com or support@apps2go.com or phone (817) 431 8775 or +1 817 431-8775 Contact Metro Link for an either ELF or a.out Motif 2.1 distribution for FreeBSD. This distribution includes: OSF/Motif manager, xmbind, panner, wsm. Development kit with uil, mrm, xm, xmcxx, include and Imake files. Static and dynamic libraries (specify ELF for use with FreeBSD 3.0 and later; or a.out for use with FreeBSD 2.2.8 and earlier). Demonstration applets. Preformatted man pages. Be sure to specify that you want the FreeBSD version of Motif when ordering! Versions for Linux are also sold by Metro Link. This is available on either a CDROM or for FTP download. Contact Xi Graphics for an a.out Motif 2.0 distribution for FreeBSD. This distribution includes: OSF/Motif manager, xmbind, panner, wsm. Development kit with uil, mrm, xm, xmcxx, include and Imake files. Static and dynamic libraries (for use with FreeBSD 2.2.8 and earlier). Demonstration applets. Preformatted man pages. Be sure to specify that you want the FreeBSD version of Motif when ordering! Versions for BSDI and Linux are also sold by Xi Graphics. This is currently a 4 diskette set... in the future this will change to a unified CD distribution like their CDE. Where can I get CDE for FreeBSD? Xi Graphics used to sell CDE for FreeBSD, but no longer do. KDE is an open source X11 desktop which is similar to CDE in many respects. You might also like the look and feel of xfce. KDE and xfce are both in the ports system. Are there any commercial high-performance X servers? Yes, Xi Graphics and Metro Link sells Accelerated-X product for FreeBSD and other Intel based systems. The Metro Link offering is a high performance X Server that offers easy configuration using the FreeBSD Package suite of tools, support for multiple concurrent video boards and is distributed in binary form only, in a convenient FTP download. Not to mention the Metro Link offering is available at the very reasonable price of $39. Metro Link also sells both ELF and a.out Motif for FreeBSD (see above). More info Metro Link WWW page or sales@metrolink.com or tech@metrolink.com or phone (954) 938-0283 or +1 954 938-0283 The Xi Graphics offering is a high performance X Server that offers easy configuration, support for multiple concurrent video boards and is distributed in binary form only, in a unified diskette distribution for FreeBSD and Linux. Xi Graphics also offers a high performance X Server tailored for laptop support. There is a free compatibility demo of version 5.0 available. Xi Graphics also sells Motif and CDE for FreeBSD (see above). More info Xi Graphics WWW page or sales@xig.com or support@xig.com or phone (800) 946 7433 or +1 303 298-7478. Are there any Database systems for FreeBSD? Yes! See the Commercial Vendors section of FreeBSD's Web site. Also see the Databases section of the Ports collection. Can I run Oracle on FreeBSD? Yes. The following pages tell you exactly how to setup Linux-Oracle on FreeBSD: http://www.scc.nl/~marcel/howto-oracle.html http://www.lf.net/lf/pi/oracle/install-linux-oracle-on-freebsd User Applications So, where are all the user applications? Please take a look at the ports page for info on software packages ported to FreeBSD. The list currently tops 3400 and is growing daily, so come back to check often or subscribe to the freebsd-announce mailing list for periodic updates on new entries. Most ports should be available for the 2.2, 3.x and 4.x branches, and many of them should work on 2.1.x systems as well. Each time a FreeBSD release is made, a snapshot of the ports tree at the time of release in also included in the ports/ directory. We also support the concept of a package, essentially no more than a gzipped binary distribution with a little extra intelligence embedded in it for doing whatever custom installation work is required. A package can be installed and uninstalled again easily without having to know the gory details of which files it includes. Use the package installation menu in /stand/sysinstall (under the post-configuration menu item) or invoke the &man.pkg.add.1; command on the specific package files you are interested in installing. Package files can usually be identified by their .tgz suffix and CDROM distribution people will have a packages/All directory on their CD which contains such files. They can also be downloaded over the net for various versions of FreeBSD at the following locations: for 2.2.8-RELEASE/2.2.8-STABLE ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/i386/packages-2.2.8/ for 3.X-RELEASE/3.X-STABLE ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/i386/packages-3-stable/ for 4.X-RELEASE/4-STABLE ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/i386/packages-4-stable/ for 5.X-CURRENT ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/i386/packages-5-current or your nearest local mirror site. Note that all ports may not be available as packages since new ones are constantly being added. It is always a good idea to check back periodically to see which packages are available at the ftp.FreeBSD.org master site. Why is /bin/sh so minimal? Why doesn't FreeBSD use bash or another shell? Because POSIX says that there shall be such a shell. The more complicated answer: many people need to write shell scripts which will be portable across many systems. That is why POSIX specifies the shell and utility commands in great detail. Most scripts are written in Bourne shell, and because several important programming interfaces (&man.make.1;, &man.system.3;, &man.popen.3;, and analogues in higher-level scripting languages like Perl and Tcl) are specified to use the Bourne shell to interpret commands. Because the Bourne shell is so often and widely used, it is important for it to be quick to start, be deterministic in its behavior, and have a small memory footprint. The existing implementation is our best effort at meeting as many of these requirements simultaneously as we can. In order to keep /bin/sh small, we have not provided many of the convenience features that other shells have. That is why the Ports Collection includes more featureful shells like bash, scsh, tcsh, and zsh. (You can compare for yourself the memory utilization of all these shells by looking at the VSZ and RSS columns in a ps -u listing.) Where do I find libc.so.3.0? You are trying to run a package built on 2.2 and later on a 2.1.x system. Please take a look at the previous section and get the correct port/package for your system. Why do I get a message reading Error: can't find libc.so.4.0? You accidently downloaded packages meant for 4.X and 5.X systems and attempted to install them on your 2.X or 3.X FreeBSD system. Please download the correct version of the packages. Why does ghostscript give lots of errors with my 386/486SX? You do not have a math co-processor, right? You will need to add the alternative math emulator to your kernel; you do this by adding the following to your kernel config file and it will be compiled in. options GPL_MATH_EMULATE You will need to remove the MATH_EMULATE option when you do this. Why do SCO/iBCS2 applications bomb on socksys? (FreeBSD 3.0 and older only). You first need to edit the /etc/sysconfig (or /etc/rc.conf, see &man.rc.conf.5;) file in the last section to change the following variable to YES: # Set to YES if you want ibcs2 (SCO) emulation loaded at startup ibcs2=NO It will load the ibcs2 kernel module at startup. You will then need to set up /compat/ibcs2/dev to look like: lrwxr-xr-x 1 root wheel 9 Oct 15 22:20 X0R@ -> /dev/null lrwxr-xr-x 1 root wheel 7 Oct 15 22:20 nfsd@ -> socksys -rw-rw-r-- 1 root wheel 0 Oct 28 12:02 null lrwxr-xr-x 1 root wheel 9 Oct 15 22:20 socksys@ -> /dev/null crw-rw-rw- 1 root wheel 41, 1 Oct 15 22:14 spx You just need socksys to go to /dev/null (see &man.null.4;) to fake the open & close. The code in -CURRENT will handle the rest. This is much cleaner than the way it was done before. If you want the spx driver for a local socket X connection, define SPX_HACK when you compile the system. How do I configure INN (Internet News) for my machine? After installing the inn package or port, an excellent place to start is Dave Barr's INN Page where you will find the INN FAQ. What version of Microsoft FrontPage should I get? Use the Port, Luke! A pre-patched version of Apache is available in the ports tree. Does FreeBSD support Java? Yes. Please see http://www.FreeBSD.org/java/. Why can't I build this port on my 3.X-STABLE machine? If you are running a FreeBSD version that lags significantly behind -CURRENT or -STABLE, you may need a ports upgrade kit from http://www.FreeBSD.org/ports/. If you are up to date, then someone might have committed a change to the port which works for -CURRENT but which broke the port for -STABLE. Please submit a bug report on this with the &man.send-pr.1; command, since the ports collection is supposed to work for both the -CURRENT and -STABLE branches. Where do I find ld.so? If you want to run some aout applications like Netscape Navigator on an Elf'ened machine such as 3.1-R or later, it would need /usr/libexec/ld.so and some aout libs. They are included in the compat22 distribution. Use /stand/sysinstall or install.sh in the compat22 subdirectory and install it. Also read ERRATAs for 3.1-R and 3.2-R. I updated the sources, now how do I update my installed ports? Unfortunately, there is no easy way to update installed ports. The &man.pkg.version.1; command can be used to generate a script that will update the installed ports with a newer version in the ports tree: &prompt.root; pkg_version > /tmp/myscript The output script must be edited by hand before you use it. Current versions of &man.pkg.version.1; force this by inserting an &man.exit.1; at the beginning of the script. You should save the output of the script, as it will note packages that depend on the one that has been updated. These may or may not need to be updated as well. The usual case where they need to be updated is that a shared library has changed version numbers, so the ports that used that library need to be rebuilt to use the new version. If your system is up full time, the &man.periodic.8 system can be used to generate a weekly list of ports that might need updating by setting weekly_status_pkg_enable="YES" in /etc/periodic.conf. Kernel Configuration I would like to customize my kernel. Is it difficult? Not at all! Check out the kernel config section of the Handbook. It is recommended that you make a dated snapshot of your kernel in kernel.YYMMDD after you get it all working, that way if you do something dire the next time you play with your configuration you can boot that kernel instead of having to go all the way back to kernel.GENERIC. This is particularly important if you are now booting off a controller that is not supported in the GENERIC kernel. My kernel compiles fail because _hw_float is missing. How do I solve this problem? Let me guess. You removed npx0 (see &man.npx.4;) from your kernel configuration file because you do not have a math co-processor, right? Wrong! :-) The npx0 is MANDATORY. Even if you do not have a mathematic co-processor, you must include the npx0 device. Why is my kernel so big (over 10MB)? Chances are, you compiled your kernel in debug mode. Kernels built in debug mode contain many symbols that are used for debugging, thus greatly increasing the size of the kernel. Note that if you running a FreeBSD 3.0 or later system, there will be little or no performance decrease from running a debug kernel, and it is useful to keep one around in case of a system panic. However, if you are running low on disk space, or you simply do not want to run a debug kernel, make sure that both of the following are true: You do not have a line in your kernel configuration file that reads: makeoptions DEBUG=-g You are not running &man.config.8; with the option. Both of the above situations will cause your kernel to be built in debug mode. As long as you make sure you follow the steps above, you can build your kernel normally, and you should notice a fairly large size decrease; most kernels tend to be around 1.5MB to 2MB. Why do I get interrupt conflicts with multi-port serial code? When I compile a kernel with multi-port serial code, it tells me that only the first port is probed and the rest skipped due to interrupt conflicts. How do I fix this? The problem here is that FreeBSD has code built-in to keep the kernel from getting trashed due to hardware or software conflicts. The way to fix this is to leave out the IRQ settings on all but one port. Here is a example: # # Multiport high-speed serial line - 16550 UARTS # device sio2 at isa? port 0x2a0 tty irq 5 flags 0x501 vector siointr device sio3 at isa? port 0x2a8 tty flags 0x501 vector siointr device sio4 at isa? port 0x2b0 tty flags 0x501 vector siointr device sio5 at isa? port 0x2b8 tty flags 0x501 vector siointr Why does every kernel I try to build fail to compile, even GENERIC? There are a number of possible causes for this problem. They are, in no particular order: You are not using the new make buildkernel and make installkernel targets, and your source tree is different from the one used to build the currently running system (e.g., you are compiling 4.3-RELEASE on a 4.0-RELEASE system). If you are attempting an upgrade, please read the /usr/src/UPDATING file, paying particular attention to the COMMON ITEMS section at the end. You are using the new make buildkernel and make installkernel targets, but you failed to assert the completion of the make buildworld target. The make buildkernel target relies on files generated by the make buildworld target to complete its job correctly. Even if you are trying to build FreeBSD-STABLE, it is possible that you fetched the source tree at a time when it was either being modified, or broken for other reasons; only releases are absolutely guaranteed to be buildable, although FreeBSD-STABLE builds fine the majority of the time. If you have not already done so, try re-fetching the source tree and see if the problem goes away. Try using a different server in case the one you are using is having problems. System Administration Where are the system start-up configuration files? From 2.0.5R to 2.2.1R, the primary configuration file is /etc/sysconfig. All the options are to be specified in this file and other files such as /etc/rc (see &man.rc.8;) and /etc/netstart just include it. Look in the /etc/sysconfig file and change the value to match your system. This file is filled with comments to show what to put in there. In post-2.2.1 and 3.0, /etc/sysconfig was renamed to a more self-describing &man.rc.conf.5; file and the syntax cleaned up a bit in the process. /etc/netstart was also renamed to /etc/rc.network so that all files could be copied with a cp /usr/src/etc/rc* /etc command. And, in 3.1 and later, /etc/rc.conf has been moved to /etc/defaults/rc.conf. Do not edit this file! Instead, if there is any entry in /etc/defaults/rc.conf that you want to change, you should copy the line into /etc/rc.conf and change it there. For example, if you wish to start named, the DNS server included with FreeBSD in FreeBSD 3.1 or later, all you need to do is: &prompt.root; echo named_enable="YES" >> /etc/rc.conf To start up local services in FreeBSD 3.1 or later, place shell scripts in the /usr/local/etc/rc.d directory. These shell scripts should be set executable, and end with a .sh. In FreeBSD 3.0 and earlier releases, you should edit the /etc/rc.local file. The /etc/rc.serial is for serial port initialization (e.g. locking the port characteristics, and so on.). The /etc/rc.i386 is for Intel-specifics settings, such as iBCS2 emulation or the PC system console configuration. How do I add a user easily? Use the &man.adduser.8; command. For more complicated usage, the &man.pw.8; command. To remove the user again, use the &man.rmuser.8; command. Once again, &man.pw.8; will work as well. How can I add my new hard disk to my FreeBSD system? See the Disk Formatting Tutorial at www.FreeBSD.org. I have a new removable drive, how do I use it? Whether it is a removable drive like a ZIP or an EZ drive (or even a floppy, if you want to use it that way), or a new hard disk, once it is installed and recognized by the system, and you have your cartridge/floppy/whatever slotted in, things are pretty much the same for all devices. (this section is based on Mark Mayo's ZIP FAQ) If it is a ZIP drive or a floppy , you have already got a DOS filesystem on it, you can use a command like this: &prompt.root; mount -t msdos /dev/fd0c /floppy if it is a floppy, or this: &prompt.root; mount -t msdos /dev/da2s4 /zip for a ZIP disk with the factory configuration. For other disks, see how they are laid out using &man.fdisk.8; or &man.sysinstall.8;. The rest of the examples will be for a ZIP drive on da2, the third SCSI disk. Unless it is a floppy, or a removable you plan on sharing with other people, it is probably a better idea to stick a BSD file system on it. You will get long filename support, at least a 2X improvement in performance, and a lot more stability. First, you need to redo the DOS-level partitions/filesystems. You can either use &man.fdisk.8; or /stand/sysinstall, or for a small drive that you do not want to bother with multiple operating system support on, just blow away the whole FAT partition table (slices) and just use the BSD partitioning: &prompt.root; dd if=/dev/zero of=/dev/rda2 count=2 &prompt.root; disklabel -Brw da2 auto You can use disklabel or /stand/sysinstall to create multiple BSD partitions. You will certainly want to do this if you are adding swap space on a fixed disk, but it is probably irrelevant on a removable drive like a ZIP. Finally, create a new file system, this one is on our ZIP drive using the whole disk: &prompt.root; newfs /dev/rda2c and mount it: &prompt.root; mount /dev/da2c /zip and it is probably a good idea to add a line like this to /etc/fstab (see &man.fstab.5;) so you can just type mount /zip in the future: /dev/da2c /zip ffs rw,noauto 0 0 Why do I keep getting messages like root: not found after editing my crontab file? This is normally caused by editing the system crontab (/etc/crontab) and then using &man.crontab.1; to install it: &prompt.root; crontab /etc/crontab This is not the correct way to do things. The system crontab has a different format to the per-user crontabs which &man.crontab.1; updates (the &man.crontab.5; manual page explains the differences in more detail). If this is what you did, the extra crontab is simply a copy of /etc/crontab in the wrong format it. Delete it with the command: &prompt.root; crontab -r Next time, when you edit /etc/crontab, you should not do anything to inform &man.cron.8; of the changes, since it will notice them automatically. If you want something to be run once per day, week, or month, it is probably better to add shell scripts /usr/local/etc/periodic, and let the &man.periodic.8; command run from the system cron schedule it with the other periodic system tasks. The actual reason for the error is that the system crontab has an extra field, specifying which user to run the command as. In the default system crontab provided with FreeBSD, this is root for all entries. When this crontab is used as the root user's crontab (which is not the same as the system crontab), &man.cron.8; assumes the string root is the first word of the command to execute, but no such command exists. Why do I get the error, you are not in the correct group to su root when I try to su to root? This is a security feature. In order to su to root (or any other account with superuser privileges), you must be in the wheel group. If this feature were not there, anybody with an account on a system who also found out root's password would be able to gain superuser level access to the system. With this feature, this is not strictly true; &man.su.1; will prevent them from even trying to enter the password if they are not in wheel. To allow someone to su to root, simply put them in the wheel group. I made a mistake in rc.conf, or another startup file, and now I cannot edit it because the filesystem is read-only. What should I do? When you get the prompt to enter the shell pathname, simply press ENTER, and run mount / to re-mount the root filesystem in read/write mode. You may also need to run mount -a -t ufs to mount the filesystem where your favourite editor is defined. If your favourite editor is on a network filesystem, you will need to either configure the network manually before you can mount network filesystems, or use an editor which resides on a local filesystem, such as &man.ed.1;. If you intend to use a full screen editor such as &man.vi.1; or &man.emacs.1;, you may also need to run export TERM=cons25 so that these editors can load the correct data from the &man.termcap.5; database. Once you have performed these steps, you can edit /etc/rc.conf as you usually would to fix the syntax error. The error message displayed immediately after the kernel boot messages should tell you the number of the line in the file which is at fault. How do I mount a secondary DOS partition? The secondary DOS partitions are found after ALL the primary partitions. For example, if you have an E partition as the second DOS partition on the second SCSI drive, you need to create the special files for slice 5 in /dev, then mount /dev/da1s5: &prompt.root; cd /dev &prompt.root; sh MAKEDEV da1s5 &prompt.root; mount -t msdos /dev/da1s5 /dos/e Can I mount other foreign filesystems under FreeBSD? Digital UNIX UFS CDROMs can be mounted directly on FreeBSD. Mounting disk partitions from Digital UNIX and other systems that support UFS may be more complex, depending on the details of the disk partitioning for the operating system in question. Linux As of 2.2, FreeBSD supports ext2fs partitions. See &man.mount.ext2fs.8; for more information. NT A read-only NTFS driver exists for FreeBSD. For more information, see this tutorial by Mark Ovens at http://ukug.uk.freebsd.org/~mark/ntfs_install.html. Any other information on this subject would be appreciated. How can I use the NT loader to boot FreeBSD? This procedure is slightly different for 2.2.x and 3.x (with the 3-stage boot) systems. The general idea is that you copy the first sector of your native root FreeBSD partition into a file in the DOS/NT partition. Assuming you name that file something like c:\bootsect.bsd (inspired by c:\bootsect.dos), you can then edit the c:\boot.ini file to come up with something like this: [boot loader] timeout=30 default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS [operating systems] multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows NT" C:\BOOTSECT.BSD="FreeBSD" C:\="DOS" For 2.2.x systems this procedure assumes that DOS, NT, FreeBSD, or whatever have been installed into their respective fdisk partitions on the same disk. This example was tested on a system where DOS & NT were on the first fdisk partition, and FreeBSD on the second. FreeBSD was also set up to boot from its native partition, not the disk's MBR. Mount a DOS-formatted floppy (if you have converted to NTFS) or the FAT partition, under, say, /mnt. &prompt.root; dd if=/dev/rda0a of=/mnt/bootsect.bsd bs=512 count=1 Reboot into DOS or NT. NTFS users copy the bootsect.bsd and/or the bootsect.lnx file from the floppy to C:\. Modify the attributes (permissions) on boot.ini with: C:\> attrib -s -r c:\boot.ini Edit to add the appropriate entries from the example boot.ini above, and restore the attributes: C:\> attrib +s +r c:\boot.ini If FreeBSD is booting from the MBR, restore it with the DOS fdisk command after you reconfigure them to boot from their native partitions. For FreeBSD 3.x systems the procedure is somewhat simpler. If FreeBSD is installed on the same disk as the NT boot partition simply copy /boot/boot1 to C:\BOOTSECT.BSD However, if FreeBSD is installed on a different disk /boot/boot1 will not work, /boot/boot0 is needed. DO NOT SIMPLY COPY /boot/boot0 INSTEAD OF /boot/boot1, YOU WILL OVERWRITE YOUR PARTITION TABLE AND RENDER YOUR COMPUTER UN-BOOTABLE! /boot/boot0 needs to be installed using sysinstall by selecting the FreeBSD boot manager on the screen which asks if you wish to use a boot manager. This is because /boot/boot0 has the partition table area filled with NULL characters but sysinstall copies the partition table before copying /boot/boot0 to the MBR. When the FreeBSD boot manager runs it records the last OS booted by setting the active flag on the partition table entry for that OS and then writes the whole 512-bytes of itself back to the MBR so if you just copy /boot/boot0 to C:\BOOTSECT.BSD then it writes an empty partition table, with the active flag set on one entry, to the MBR. How do I boot FreeBSD and Linux from LILO? If you have FreeBSD and Linux on the same disk, just follow LILO's installation instructions for booting a non-Linux operating system. Very briefly, these are: Boot Linux, and add the following lines to /etc/lilo.conf: other=/dev/hda2 table=/dev/hda label=FreeBSD (the above assumes that your FreeBSD slice is known to Linux as /dev/hda2; tailor to suit your setup). Then, run lilo as root and you should be done. If FreeBSD resides on another disk, you need to add loader=/boot/chain.b to the LILO entry. For example: other=/dev/dab4 table=/dev/dab loader=/boot/chain.b label=FreeBSD In some cases you may need to specify the BIOS drive number to the FreeBSD boot loader to successfully boot off the second disk. For example, if your FreeBSD SCSI disk is probed by BIOS as BIOS disk 1, at the FreeBSD boot loader prompt you need to specify: Boot: 1:da(0,a)/kernel On FreeBSD 2.2.5 and later, you can configure &man.boot.8; to automatically do this for you at boot time. The Linux+FreeBSD mini-HOWTO is a good reference for FreeBSD and Linux interoperability issues. How do I boot FreeBSD and Linux using BootEasy? Install LILO at the start of your Linux boot partition instead of in the Master Boot Record. You can then boot LILO from BootEasy. If you are running Windows-95 and Linux this is recommended anyway, to make it simpler to get Linux booting again if you should need to reinstall Windows95 (which is a Jealous Operating System, and will bear no other Operating Systems in the Master Boot Record). Will a dangerously dedicated disk endanger my health? The installation procedure allows you to chose two different methods in partitioning your harddisk(s). The default way makes it compatible with other operating systems on the same machine, by using fdisk table entries (called slices in FreeBSD), with a FreeBSD slice that employs partitions of its own. Optionally, one can chose to install a boot-selector to switch between the possible operating systems on the disk(s). The alternative uses the entire disk for FreeBSD, and makes no attempt to be compatible with other operating systems. So why it is called dangerous? A disk in this mode does not contain what normal PC utilities would consider a valid fdisk table. Depending on how well they have been designed, they might complain at you once they are getting in contact with such a disk, or even worse, they might damage the BSD bootstrap without even asking or notifying you. In addition, the dangerously dedicated disk's layout is known to confuse many BIOSsen, including those from AWARD (eg. as found in HP Netserver and Micronics systems as well as many others) and Symbios/NCR (for the popular 53C8xx range of SCSI controllers). This is not a complete list, there are more. Symptoms of this confusion include the read error message printed by the FreeBSD bootstrap when it cannot find itself, as well as system lockups when booting. Why have this mode at all then? It only saves a few kbytes of disk space, and it can cause real problems for a new installation. Dangerously dedicated mode's origins lie in a desire to avoid one of the most common problems plaguing new FreeBSD installers - matching the BIOS geometry numbers for a disk to the disk itself. Geometry is an outdated concept, but one still at the heart of the PC's BIOS and its interaction with disks. When the FreeBSD installer creates slices, it has to record the location of these slices on the disk in a fashion that corresponds with the way the BIOS expects to find them. If it gets it wrong, you will not be able to boot. Dangerously dedicated mode tries to work around this by making the problem simpler. In some cases, it gets it right. But it is meant to be used as a last-ditch alternative - there are better ways to solve the problem 99 times out of 100. So, how do you avoid the need for DD mode when you are installing? Start by making a note of the geometry that your BIOS claims to be using for your disks. You can arrange to have the kernel print this as it boots by specifying at the boot: prompt, or using boot -v in the loader. Just before the installer starts, the kernel will print a list of BIOS geometries. Do not panic - wait for the installer to start and then use scrollback to read the numbers. Typically the BIOS disk units will be in the same order that FreeBSD lists your disks, first IDE, then SCSI. When you are slicing up your disk, check that the disk geometry displayed in the FDISK screen is correct (ie. it matches the BIOS numbers); if it is wrong, use the g key to fix it. You may have to do this if there is absolutely nothing on the disk, or if the disk has been moved from another system. Note that this is only an issue with the disk that you are going to boot from; FreeBSD will sort itself out just fine with any other disks you may have. Once you have got the BIOS and FreeBSD agreeing about the geometry of the disk, your problems are almost guaranteed to be over, and with no need for DD mode at all. If, however, you are still greeted with the dreaded read error message when you try to boot, it is time to cross your fingers and go for it - there's nothing left to lose. To return a dangerously dedicated disk for normal PC use, there are basically two options. The first is, you write enough NULL bytes over the MBR to make any subsequent installation believe this to be a blank disk. You can do this for example with &prompt.root; dd if=/dev/zero of=/dev/rda0 count=15 Alternatively, the undocumented DOS feature C:\> fdisk /mbr will to install a new master boot record as well, thus clobbering the BSD bootstrap. How can I add more swap space? The best way is to increase the size of your swap partition, or take advantage of this convenient excuse to add another disk. The general rule of thumb is to have around 2x the swap space as you have main memory. However, if you have a very small amount of main memory you may want to configure swap beyond that. It is also a good idea to configure sufficient swap relative to anticipated future memory upgrades so you do not have to futz with your swap configuration later. Adding swap onto a separate disk makes things faster than simply adding swap onto the same disk. As an example, if you are compiling source located on one disk, and the swap is on another disk, this is much faster than both swap and compile on the same disk. This is true for SCSI disks specifically. When you have several disks, configuring a swap partition on each one is usually beneficial, even if you wind up putting swap on a work disk. Typically, each fast disk in your system should have some swap configured. FreeBSD supports up to 4 interleaved swap devices by default. When configuring multiple swap partitions you generally want to make them all about the same size, but people sometimes make their primary swap partition larger in order to accomodate a kernel core dump. Your primary swap partition must be at least as large as main memory in order to be able to accomodate a kernel core. IDE drives are not able to allow access to both drives on the same channel at the same time (FreeBSD does not support mode 4, so all IDE disk I/O is programmed). It is still suggested that you put your swap partition on a separate driver, however: the drives are so cheap, it is not worth worrying about. Swapping over NFS is only recommended if you do not have a local disk to swap to. Swapping over NFS is slow and inefficient in FreeBSD releases prior to 4.x, but reasonably fast in releases greater or equal to 4.0. Even so, it will be limited to the network bandwidth available and puts an additional burden on the NFS server. Here is an example for 64Mb vn-swap (/usr/swap0, though of course you can use any name that you want). Make sure your kernel was built with the line pseudo-device vn 1 #Vnode driver (turns a file into a device) in your config-file. The GENERIC kernel already contains this. create a vn-device &prompt.root; cd /dev &prompt.root; sh MAKEDEV vn0 create a swapfile (/usr/swap0) &prompt.root; dd if=/dev/zero of=/usr/swap0 bs=1024k count=64 set proper permissions on (/usr/swap0) &prompt.root; chmod 0600 /usr/swap0 enable the swap file in /etc/rc.conf swapfile="/usr/swap0" # Set to name of swapfile if aux swapfile desired. reboot the machine To enable the swap file immediately, type &prompt.root; vnconfig -e /dev/vn0b /usr/swap0 swap Why am I having trouble setting up my printer? Please have a look at the Handbook entry on printing. It should cover most of your problem. See the Handbook entry on printing. Some printers require a host-based driver to do any kind of printing. These so-called WinPrinters are not natively supported by FreeBSD. If your printer does not work in DOS or Windows NT 4.0, it is probably a WinPrinter. Your only hope of getting one of these to work is to check if the ports/print/pnm2ppa port supports it. From its package description:
This software creates output using the PPA (printer performance architecture) protocol. This protocol is used by some HP "Windows-only" printers, including the HP Deskjet 820C series, the HP DeskJet 720 series, and the HP DeskJet 1000 series. [...] WWW: http://pnm2ppa.sourceforge.net/
How can I correct the keyboard mappings for my system? The kbdcontrol program has an option to load a keyboard map file. Under /usr/share/syscons/keymaps are a number of map files. Choose the one relevant to your system and load it. &prompt.root; kbdcontrol -l uk.iso Both the /usr/share/syscons/keymaps and the .kbd extension are assumed by &man.kbdcontrol.1;. This can be configured in /etc/sysconfig (or &man.rc.conf.5;). See the appropriate comments in this file. In 2.0.5R and later, everything related to text fonts, keyboard mapping is in /usr/share/examples/syscons. The following mappings are currently supported: Belgian ISO-8859-1 Brazilian 275 keyboard Codepage 850 Brazilian 275 keyboard ISO-8859-1 Danish Codepage 865 Danish ISO-8859-1 French ISO-8859-1 German Codepage 850 German ISO-8859-1 Italian ISO-8859-1 Japanese 106 Japanese 106x Latin American Norwegian ISO-8859-1 Polish ISO-8859-2 (programmer's) Russian Codepage 866 (alternative) Russian koi8-r (shift) Russian koi8-r Spanish ISO-8859-1 Swedish Codepage 850 Swedish ISO-8859-1 Swiss-German ISO-8859-1 United Kingdom Codepage 850 United Kingdom ISO-8859-1 United States of America ISO-8859-1 United States of America dvorak United States of America dvorakx Why do I get messages like: unknown: <PNP0303> can't assign resources on boot? The following is an excerpt from a post to the freebsd-current mailing list.
&a.wollman;, 24 April 2001 The can't assign resources messages indicate that the devices are legacy ISA devices for which a non-PnP-aware driver is compiled into the kernel. These include devices such as keyboard controllers, the programmable interrupt controller chip, and several other bits of standard infrastructure. The resources cannot be assigned because there is already a driver using those addresses.
How come I cannot get user quotas to work properly? Do not turn on quotas on /, Put the quota file on the file system that the quotas are to be enforced on. ie: Filesystem Quota file /usr /usr/admin/quotas /home /home/admin/quotas … … What is inappropriate about my ccd? The symptom of this is: &prompt.root; ccdconfig -C ccdconfig: ioctl (CCDIOCSET): /dev/ccd0c: Inappropriate file type or format This usually happens when you are trying to concatenate the c partitions, which default to type unused. The ccd driver requires the underlying partition type to be FS_BSDFFS. Edit the disklabel of the disks you are trying to concatenate and change the types of partitions to 4.2BSD. Why can't I edit the disklabel on my ccd? The symptom of this is: &prompt.root; disklabel ccd0 (it prints something sensible here, so let's try to edit it) &prompt.root; disklabel -e ccd0 (edit, save, quit) disklabel: ioctl DIOCWDINFO: No disk label on disk; use "disklabel -r" to install initial label This is because the disklabel returned by ccd is actually a fake one that is not really on the disk. You can solve this problem by writing it back explicitly, as in: &prompt.root; disklabel ccd0 > /tmp/disklabel.tmp &prompt.root; disklabel -Rr ccd0 /tmp/disklabel.tmp &prompt.root; disklabel -e ccd0 (this will work now) Does FreeBSD support System V IPC primitives? Yes, FreeBSD supports System V-style IPC. This includes shared memory, messages and semaphores. You need to add the following lines to your kernel config to enable them. options SYSVSHM options SYSVSHM # enable shared memory options SYSVSEM # enable for semaphores options SYSVMSG # enable for messaging In FreeBSD 3.2 and later, these options are already part of the GENERIC kernel, which means they should already be compiled into your system. Recompile and install your kernel. How do I use sendmail for mail delivery with UUCP? The sendmail configuration that ships with FreeBSD is suited for sites that connect directly to the Internet. Sites that wish to exchange their mail via UUCP must install another sendmail configuration file. Tweaking /etc/sendmail.cf manually is considered something for purists. Sendmail version 8 comes with a new approach of generating config files via some &man.m4.1; preprocessing, where the actual hand-crafted configuration is on a higher abstraction level. You should use the configuration files under /usr/src/usr.sbin/sendmail/cf If you did not install your system with full sources, the sendmail config stuff has been broken out into a separate source distribution tarball just for you. Assuming you have got your CD-ROM mounted, do: &prompt.root; cd /cdrom/src &prompt.root; cat scontrib.?? | tar xzf - -C /usr/src contrib/sendmail Do not panic, this is only a few hundred kilobytes in size. The file README in the cf directory can serve as a basic introduction to m4 configuration. For UUCP delivery, you are best advised to use the mailertable feature. This constitutes a database that sendmail can use to base its routing decision upon. First, you have to create your .mc file. The directory /usr/src/usr.sbin/sendmail/cf/cf is the home of these files. Look around, there are already a few examples. Assuming you have named your file foo.mc, all you need to do in order to convert it into a valid sendmail.cf is: - -&prompt.root; cd /usr/src/usr.sbin/sendmail/cf/cf + &prompt.root; cd /usr/src/usr.sbin/sendmail/cf/cf &prompt.root; make foo.cf &prompt.root; cp foo.cf /etc/sendmail.cf A typical .mc file might look like: include(`../m4/cf.m4') VERSIONID(`Your version number') OSTYPE(bsd4.4) FEATURE(nodns) FEATURE(nocanonify) FEATURE(mailertable) define(`UUCP_RELAY', your.uucp.relay) define(`UUCP_MAX_SIZE', 200000) MAILER(local) MAILER(smtp) MAILER(uucp) Cw your.alias.host.name Cw youruucpnodename.UUCP The nodns and nocanonify features will prevent any usage of the DNS during mail delivery. The UUCP_RELAY clause is needed for bizarre reasons, do not ask. Simply put an Internet hostname there that is able to handle .UUCP pseudo-domain addresses; most likely, you will enter the mail relay of your ISP there. Once you have got this, you need this file called /etc/mailertable. A typical example of this gender again: # # makemap hash /etc/mailertable.db < /etc/mailertable # horus.interface-business.de uucp-dom:horus .interface-business.de uucp-dom:if-bus interface-business.de uucp-dom:if-bus .heep.sax.de smtp8:%1 horus.UUCP uucp-dom:horus if-bus.UUCP uucp-dom:if-bus . uucp-dom: As you can see, this is part of a real-life file. The first three lines handle special cases where domain-addressed mail should not be sent out to the default route, but instead to some UUCP neighbor in order to shortcut the delivery path. The next line handles mail to the local Ethernet domain that can be delivered using SMTP. Finally, the UUCP neighbors are mentioned in the .UUCP pseudo-domain notation, to allow for a uucp-neighbor !recipient override of the default rules. The last line is always a single dot, matching everything else, with UUCP delivery to a UUCP neighbor that serves as your universal mail gateway to the world. All of the node names behind the uucp-dom: keyword must be valid UUCP neighbors, as you can verify using the command uuname. As a reminder that this file needs to be converted into a DBM database file before being usable, the command line to accomplish this is best placed as a comment at the top of the mailertable. You always have to execute this command each time you change your mailertable. Final hint: if you are uncertain whether some particular mail routing would work, remember the option to sendmail. It starts sendmail in address test mode; simply enter 0, followed by the address you wish to test for the mail routing. The last line tells you the used internal mail agent, the destination host this agent will be called with, and the (possibly translated) address. Leave this mode by typing Control-D. &prompt.user; sendmail -bt ADDRESS TEST MODE (ruleset 3 NOT automatically invoked) Enter <ruleset> <address> > 0 foo@interface-business.de rewrite: ruleset 0 input: foo @ interface-business . de ... rewrite: ruleset 0 returns: $# uucp-dom $@ if-bus $: foo \ < @ interface-business . de > > ^D How do I set up mail with a dialup connection to the 'net? If you have got a statically assigned IP number, you should not need to adjust anything from the default. Set your host name up as your assigned internet name and sendmail will do the rest. If you have got a dynamically assigned IP number and use a dialup ppp connection to the internet, you will probably be given a mailbox on your ISPs mail server. Lets assume your ISPs domain is myISP.com, and that your user name is user. Lets also assume you have called your machine bsd.home and that your ISP has told you that you may use relay.myISP.com as a mail relay. In order to retrieve mail from your mailbox, you will need to install a retrieval agent. Fetchmail is a good choice as it supports many different protocols. Usually, POP3 will be provided by your ISP. If you have chosen to use user-ppp, you can automatically fetch your mail when a connection to the 'net is established with the following entry in /etc/ppp/ppp.linkup: MYADDR: !bg su user -c fetchmail If you are using sendmail (as shown below) to deliver mail to non-local accounts, put the command !bg su user -c "sendmail -q" after the above shown entry. This forces sendmail to process your mailqueue as soon as the connection to the 'net is established. I am assuming that you have an account for user on bsd.home. In the home directory of user on bsd.home, create a .fetchmailrc file: poll myISP.com protocol pop3 fetchall pass MySecret Needless to say, this file should not be readable by anyone except user as it contains the password MySecret. In order to send mail with the correct from: header, you must tell sendmail to use user@myISP.com rather than user@bsd.home. You may also wish to tell sendmail to send all mail via relay.myISP.com, allowing quicker mail transmission. The following .mc file should suffice: VERSIONID(`bsd.home.mc version 1.0') OSTYPE(bsd4.4)dnl FEATURE(nouucp)dnl MAILER(local)dnl MAILER(smtp)dnl Cwlocalhost Cwbsd.home MASQUERADE_AS(`myISP.com')dnl FEATURE(allmasquerade)dnl FEATURE(masquerade_envelope)dnl FEATURE(nocanonify)dnl FEATURE(nodns)dnl define(`SMART_HOST', `relay.myISP.com') Dmbsd.home define(`confDOMAIN_NAME',`bsd.home')dnl define(`confDELIVERY_MODE',`deferred')dnl Refer to the previous section for details of how to turn this .mc file into a sendmail.cf file. Also, don't forget to restart sendmail after updating sendmail.cf. What is this UID 0 toor account? Have I been compromised? Do not worry. toor is an alternative superuser account (toor is root spelt backwards). Previously it was created when the &man.bash.1; shell was installed but now it is created by default. It is intended to be used with a non-standard shell so you do not have to change root's default shell. This is important as shells which are not part of the base distribution (for example a shell installed from ports or packages) are likely be to be installed in /usr/local/bin which, by default, resides on a different filesystem. If root's shell is located in /usr/local/bin and /usr (or whatever filesystem contains /usr/local/bin) is not mounted for some reason, root will not be able to log in to fix a problem (although if you reboot into single user mode you will be prompted for the path to a shell). Some people use toor for day-to-day root tasks with a non-standard shell, leaving root, with a standard shell, for single user mode or emergencies. By default you cannot log in using toor as it does not have a password, so log in as root and set a password for toor if you want to use it. I have forgotten the root password! What do I do? Do not Panic! Simply restart the system, type boot -s at the Boot: prompt (just -s for FreeBSD releases before 3.2) to enter Single User mode. At the question about the shell to use, hit ENTER. You will be dropped to a &prompt.root; prompt. Enter mount -u / to remount your root filesystem read/write, then run mount -a to remount all the filesystems. Run passwd root to change the root password then run &man.exit.1; to continue booting. How do I keep Control-Alt-Delete from rebooting the system? If you are using syscons (the default console driver) in FreeBSD 2.2.7-RELEASE or later, build and install a new kernel with the line options SC_DISABLE_REBOOT in the configuration file. If you use the PCVT console driver in FreeBSD 2.2.5-RELEASE or later, use the following kernel configuration line instead: options PCVT_CTRL_ALT_DEL For older versions of FreeBSD, edit the keymap you are using for the console and replace the boot keywords with nop. The default keymap is /usr/share/syscons/keymaps/us.iso.kbd. You may have to instruct /etc/rc.conf to load this keymap explicitly for the change to take effect. Of course if you are using an alternate keymap for your country, you should edit that one instead. How do I reformat DOS text files to UNIX ones? Simply use this perl command: &prompt.user; perl -i.bak -npe 's/\r\n/\n/g' file ... file is the file(s) to process. The modification is done in-place, with the original file stored with a .bak extension. Alternatively you can use the &man.tr.1; command: &prompt.user; tr -d '\r' < dos-text-file > unix-file dos-text-file is the file containing DOS text while unix-file will contain the converted output. This can be quite a bit faster than using perl. How do I kill processes by name? Use &man.killall.1;. Why is su bugging me about not being in root's ACL? The error comes from the Kerberos distributed authentication system. The problem is not fatal but annoying. You can either run su with the -K option, or uninstall Kerberos as described in the next question. How do I uninstall Kerberos? To remove Kerberos from the system, reinstall the bin distribution for the release you are running. If you have the CDROM, you can mount the cd (we will assume on /cdrom) and run &prompt.root; cd /cdrom/bin &prompt.root; ./install.sh Alternately, you can remove all "MAKE_KERBEROS" options from /etc/make.conf and rebuild world. How do I add pseudoterminals to the system? If you have lots of telnet, ssh, X, or screen users, you will probably run out of pseudoterminals. Here is how to add more: Build and install a new kernel with the line pseudo-device pty 256 in the configuration file. Run the commands &prompt.root; cd /dev &prompt.root; sh MAKEDEV pty{1,2,3,4,5,6,7} to make 256 device nodes for the new terminals. Edit /etc/ttys and add lines for each of the 256 terminals. They should match the form of the existing entries, i.e. they look like ttyqc none network The order of the letter designations is tty[pqrsPQRS][0-9a-v], using a regular expression. Reboot the system with the new kernel and you are ready to go. How come I cannot create the snd0 device? There is no snd device. The name is used as a shorthand for the various devices that make up the FreeBSD sound driver, such as mixer, sequencer, and dsp. To create these devices you should &prompt.root; cd /dev &prompt.root; sh MAKEDEV snd0 How do I re-read /etc/rc.conf and re-start /etc/rc without a reboot? Go into single user mode and than back to multi user mode. On the console do: &prompt.root; shutdown now (Note: without -r or -h) &prompt.root; return &prompt.root; exit What is a sandbox? Sandbox is a security term. It can mean two things: A process which is placed inside a set of virtual walls that are designed to prevent someone who breaks into the process from being able to break into the wider system. The process is said to be able to play inside the walls. That is, nothing the process does in regards to executing code is supposed to be able to breech the walls so you do not have to do a detailed audit of its code to be able to say certain things about its security. The walls might be a userid, for example. This is the definition used in the security and named man pages. Take the ntalk service, for example (see /etc/inetd.conf). This service used to run as userid root. Now it runs as userid tty. The tty user is a sandbox designed to make it more difficult for someone who has successfully hacked into the system via ntalk from being able to hack beyond that user id. A process which is placed inside a simulation of the machine. This is more hard-core. Basically it means that someone who is able to break into the process may believe that he can break into the wider machine but is, in fact, only breaking into a simulation of that machine and not modifying any real data. The most common way to accomplish this is to build a simulated environment in a subdirectory and then run the processes in that directory chroot'd (i.e. / for that process is this directory, not the real / of the system). Another common use is to mount an underlying filesystem read-only and then create a filesystem layer on top of it that gives a process a seemingly writeable view into that filesystem. The process may believe it is able to write to those files, but only the process sees the effects - other processes in the system do not, necessarily. An attempt is made to make this sort of sandbox so transparent that the user (or hacker) does not realize that he is sitting in it. UNIX implements two core sandboxes. One is at the process level, and one is at the userid level. Every UNIX process is completely firewalled off from every other UNIX process. One process can not modify the address space of another. This is unlike Windows where a process can easily overwrite the address space of any other, leading to a crash. A UNIX process is owned by a particular userid. If the userid is not the root user, it serves to firewall the process off from processes owned by other users. The userid is also used to firewall off on-disk data. What is securelevel? The securelevel is a security mechanism implemented in the kernel. Basically, when the securelevel is positive, the kernel restricts certain tasks; not even the superuser (i.e., root) is allowed to do them. At the time of this writing, the securelevel mechanism is capable of, among other things, limiting the ability to, unset certain file flags, such as schg (the system immutable flag), write to kernel memory via /dev/mem and /dev/kmem, load kernel modules, and alter &man.ipfirewall.4; rules. To check the status of the securelevel on a running system, simply execute the following command: &prompt.root; sysctl kern.securelevel The output will contain the name of the &man.sysctl.8; variable (in this case, kern.securelevel) and a number. The latter is the current value of the securelevel. If it is positive (i.e., greater than 0), at least some of the securelevel's protections are enabled. You cannot lower the securelevel of a running system; being able to do that would defeat its purpose. If you need to do a task that requires that the securelevel be non-positive (e.g., an installworld or changing the date), you will have to change the securelevel setting in /etc/rc.conf (you want to look for the kern_securelevel and kern_securelevel_enable variables) and reboot. For more information on securelevel and the specific things all the levels do, please consult the &man.init.8; manual page. Securelevel is not a silver bullet; it has many known deficiencies. More often than not, it provides a false sense of security. One of its biggest problems is that in order for it to be at all effective, all files used in the boot process up until the securelevel is set must be protected. If an attacker can get the system to execute their code prior to the securelevel being set (which happens quite late in the boot process since some things the system must do at start-up cannot be done at an elevated securelevel), its protections are invalidated. While this task of protecting all files used in the boot process is not technically impossible, if it is achieved, system maintenance will become a nightmare since one would have to take the system down, at least to single-user mode, to modify a configuration file. This point and others are often discussed on the mailing lists, particularly freebsd-security. Please search the archives here for an extensive discussion. Some people are hopeful that securelevel will soon go away in favor of a more fine-grained mechanism, but things are still hazy in this respect. Consider yourself warned. How do I let ordinary users mount floppies, CDROMs and other removable media? Ordinary users can be permitted to mount devices. Here is how: As root set the sysctl variable vfs.usermount to 1. &prompt.root; sysctl -w vfs.usermount=1 As root assign the appropriate permissions to the block device associated with the removable media. For example, to allow users to mount the first floppy drive, use: &prompt.root; chmod 666 /dev/fd0 To allow users in the group operator to mount the cdrom drive, use: &prompt.root; chgrp operator /dev/cd0c &prompt.root; chmod 640 /dev/cd0c Finally, add the line vfs.usermount=1 to the file /etc/sysctl.conf so that it is reset at system boot time. All users can now mount the floppy /dev/fd0 onto a directory that they own: &prompt.user; mkdir ~/my-mount-point &prompt.user; mount -t msdos /dev/fd0 ~/my-mount-point Users in group operator can now mount the cdrom /dev/cd0c onto a directory that they own: &prompt.user; mkdir ~/my-mount-point &prompt.user; mount -t msdos /dev/cd0c ~/my-mount-point Unmounting the device is simple: &prompt.user; umount ~/my-mount-point Enabling vfs.usermount, however, has negative security implications. A better way to access MSDOS formatted media is to use the mtools package in the ports collection. How do I move my system over to my huge new disk? The best way is to reinstall the OS on the new disk, then move the user data over. This is highly recommended if you have been tracking -stable for more than one release, or have updated a release instead of installing a new one. You can install booteasy on both disks with &man.boot0cfg.8;, and dual boot them until you are happy with the new configuration. Skip the next paragraph to find out how to move the data after doing this. Should you decide not to do a fresh install, you need to partition and label the new disk with either /stand/sysinstall, or &man.fdisk.8; and &man.disklabel.8;. You should also install booteasy on both disks with &man.boot0cfg.8;, so that you can dual boot to the old or new system after the copying is done. See the formatting-media tutorial for details on this process. Now you have got the new disk set up, and are ready to move the data. Unfortunately, you cannot just blindly copy the data. Things like device files (in /dev) and symbolic links tend to screw that up. You need to use tools that understand these things, which means &man.dump.8; and &man.tar.1;. Although it is suggested that you move the data in single user mode, it is not required. You should never use anything but &man.dump.8; and &man.restore.8; to move the root file system. The &man.tar.1; command may work - then again, it may not. You should also use &man.dump.8; and &man.restore.8; if you are moving a single partition to another empty partition. The sequence of steps to use dump to move a partitions data to a new partition is: newfs the new partition. mount it on a temporary mount point. cd to that directory. dump the old partition, piping output to the new one. For example, if you are going to move root to /dev/ad1s1a, with /mnt as the temporary mount point, it is: &prompt.root; newfs /dev/ad1s1a &prompt.root; mount /dev/ad1s1a &prompt.root; cd /mnt &prompt.root; dump 0uaf - / | restore xf - If you are going to rearrange your partitions - say, splitting one into two, or combing two into one, you may find yourself needing to move everything under a subdirectory to a new location. Since &man.dump.8; works with file systems, it cannot do this. So you use &man.tar.1;. The general command to move /old to /new for &man.tar.1; is: &prompt.root; (cd /old; tar cf - .) | (cd /new; tar xpf -) If /old has file systems mounted on that, and you do not want to move that data or unmount them, you just add the 'l' flag to the first &man.tar.1;: &prompt.root; (cd /old; tar clf - .) | (cd /new; tar xpf -). You might prefer &man.cpio.1;, &man.pax.1;, or cpdup (in ports/sysutils/cpdup) to &man.tar.1;. I tried to update my system to the latest -STABLE, but got -RC or -BETA! What is going on? Short answer: it is just a name. RC stands for Release Candidate. It signifies that a release is imminent. In FreeBSD, -BETA is typically synonymous with the code freeze before a release. Long answer: FreeBSD derives its releases from one of two places. Major, dot-zero, releases, such as 3.0-RELEASE and 4.0-RELEASE, are branched from the head of the development stream, commonly referred to as -CURRENT. Minor releases, such as 3.1-RELEASE or 4.2-RELEASE, have been snapshots of the active -STABLE branch. Starting with 4.3-RELEASE, each release also now has its own branch which can be tracked by people requiring an extremely conservative rate of development (typically only security advisories). When a release is about to be made, the branch from which it will be derived from has to undergo a certain process. Part of this process is a code freeze. When a code freeze is initiated, the name of the branch is changed to reflect that it is about to become a release. For example, if the branch used to be called 4.0-STABLE, its name will be changed to 4.1-BETA to signify the code freeze and signify that extra pre-release testing should be happening. Bug fixes can still be committed to be part of the release. When the source code is in shape for the release the name will be changed to 4.1-RC to signify that a release is about to be made from it. Once in the RC stage, only the most critical bugs found can be fixed. Once the release, 4.1-RELEASE in this example, has been made, the branch will be renamed to 4.1-STABLE. I tried to install a new kernel, and the chflags failed. How do I get around this? Short answer: You are probably at security level greater than 0. Reboot directly to single user mode to install the kernel. Long answer: FreeBSD disallows changing system flags at security levels greater than 0. You can check your security level with the command: &prompt.root; sysctl kern.securelevel You cannot lower the security level; you have to boot to single mode to install the kernel, or change the security in /etc/rc.conf then reboot. See the &man.init.8; man page for details on securelevel, and see /etc/defaults/rc.conf and the &man.rc.conf.5; man page for more information on rc.conf. I cannot change the time on my system by more than one second! How do I get around this? Short answer: You are probably at security level greater than 1. Reboot directly to single user mode to change the date. Long answer: FreeBSD disallows changing the time by more that one second at security levels greater than 1. You can check your security level with the command: &prompt.root; sysctl kern.securelevel You cannot lower the security level; you have to boot to single mode to change the date, or change the security level in /etc/rc.conf then reboot. See the &man.init.8; man page for details on securelevel, and see /etc/defaults/rc.conf and the &man.rc.conf.5; man page for more information on rc.conf. Why is rpc.statd using 256 megabytes of memory? No, there is no memory leak, and it is not using 256 Mbytes of memory. It simply likes to (i.e., always does) map an obscene amount of memory into its address space for convenience. There is nothing terribly wrong with this from a technical standpoint; it just throws off things like &man.top.1; and &man.ps.1;. &man.rpc.statd.8; maps its status file (resident on /var) into its address space; to save worrying about remapping it later when it needs to grow, it maps it with a generous size. This is very evident from the source code, where one can see that the length argument to &man.mmap.2; is 0x10000000, or one sixteenth of the address space on an IA32, or exactly 256MB. Why can't I unset the schg file flag? You are running at an elevated (i.e., greater than 0) securelevel. Lower the securelevel and try again. For more information, see the FAQ entry on securelevel and the &man.init.8; manual page. Why doesn't SSH authentication through .shosts work by default in recent versions of FreeBSD? The reason why .shosts authentication does not work by default in more recent versions of FreeBSD is because &man.ssh.1; is not installed suid root by default. To fix this, you can do one of the following: As a permanent fix, set ENABLE_SUID_SSH to true in /etc/make.conf and rebuild ssh (or run make world). As a temporary fix, change the mode on /usr/bin/ssh to 4555 by running chmod 4755 /usr/bin/ssh as root. Then add ENABLE_SUID_SSH= true to /etc/make.conf so the change takes effect the next time make world is run. The X Window System and Virtual Consoles I want to run X, how do I go about it? The easiest way is to simply specify that you want to run X during the installation process. Then read and follow the documentation on the xf86config tool, which assists you in configuring XFree86(tm) for your particular graphics card/mouse/etc. You may also wish to investigate the Xaccel server. See the section on Xi Graphics or Metro Link for more details. I tried to run X, but I get an KDENABIO failed (Operation not permitted) error when I type startx. What do I do now? Your system is running at a raised securelevel, is not it? It is, indeed, impossible to start X at a raised securelevel. To see why, look at the &man.init.8; man page. So the question is what else you should do instead, and you basically have two choices: set your securelevel back down to zero (usually from /etc/rc.conf), or run &man.xdm.1; at boot time (before the securelevel is raised). See for more information about running &man.xdm.1; at boot time. Why doesn't my mouse work with X? If you are using syscons (the default console driver), you can configure FreeBSD to support a mouse pointer on each virtual screen. In order to avoid conflicting with X, syscons supports a virtual device called /dev/sysmouse. All mouse events received from the real mouse device are written to the sysmouse device via moused. If you wish to use your mouse on one or more virtual consoles, and use X, see and set up moused. Then edit /etc/XF86Config and make sure you have the following lines. Section Pointer Protocol "SysMouse" Device "/dev/sysmouse" ..... The above example is for XFree86 3.3.2 or later. For earlier versions, the Protocol should be MouseSystems. Some people prefer to use /dev/mouse under X. To make this work, /dev/mouse should be linked to /dev/sysmouse (see &man.sysmouse.4;): &prompt.root; cd /dev &prompt.root; rm -f mouse &prompt.root; ln -s sysmouse mouse My mouse has a fancy wheel. Can I use it in X? Yes. But you need to customize X client programs. See Colas Nahaboo's web page (http://www.inria.fr/koala/colas/mouse-wheel-scroll/) . If you want to use the imwheel program, just follow these simple steps. Translate the Wheel Events The imwheel program works by translating mouse button 4 and mouse button 5 events into key events. Thus, you have to get the mouse driver to translate mouse wheel events to button 4 and 5 events. There are two ways of doing this, the first way is to have &man.moused.8; do the translation. The second way is for the X server itself to do the event translation. Using &man.moused.8; to Translate Wheel Events To have &man.moused.8; perform the event translations, simply add to the command line used to start &man.moused.8;. For example, if you normally start &man.moused.8; via moused -p /dev/psm0 you would start it by entering moused -p /dev/psm0 -z 4 instead. If you start &man.moused.8; automatically during bootup via /etc/rc.conf, you can simply add to the moused_flags variable in /etc/rc.conf. You now need to tell X that you have a 5 button mouse. To do this, simply add the line Buttons 5 to the Pointer section of /etc/XF86Config. For example, you might have the following Pointer section in /etc/XF86Config. <quote>Pointer</quote> Section for Wheeled Mouse in XFree86 3.3.x series XF86Config with moused Translation Section "Pointer" Protocol "SysMouse" Device "/dev/sysmouse" Buttons 5 EndSection <quote>InputDevice</quote> Section for Wheeled Mouse in XFree86 4.x series XF86Config with automatic protocol recognition and button mapping Translation Section "InputDevice" Identifier "Mouse1" Driver "mouse" Option "Protocol" "auto" Option "Device" "/dev/psm0" Option "Buttons" "5" Option "ZAxisMapping" "4 5" EndSection <quote>.emacs</quote> example for naive page scrolling with Wheeled Mouse ;; wheel mouse (global-set-key [mouse-4] 'scroll-down) (global-set-key [mouse-5] 'scroll-up) Using Your X Server to Translate the Wheel Events If you are not running &man.moused.8;, or if you do not want &man.moused.8; to translate your wheel events, you can have the X server do the event translation instead. This requires a couple of modifications to your /etc/XF86Config file. First, you need to choose the proper protocol for your mouse. Most wheeled mice use the IntelliMouse protocol. However, XFree86 does support other protocols, such as MouseManPlusPS/2 for the Logitech MouseMan+ mice. Once you have chosen the protocol you will use, you need to add a Protocol line to the Pointer section. Secondly, you need to tell the X server to remap wheel scroll events to mouse buttons 4 and 5. This is done with the ZAxisMapping option. For example, if you are not using &man.moused.8;, and you have an IntelliMouse attached to the PS/2 mouse port you would use the following in /etc/XF86Config. <quote>Pointer</quote> Section for Wheeled Mouse in <filename>XF86Config</filename> with X Server Translation Section "Pointer" Protocol "IntelliMouse" Device "/dev/psm0" ZAxisMapping 4 5 EndSection Install imwheel Next, install imwheel from the Ports collection. It can be found in the x11 category. This program will map the wheel events from your mouse into keyboard events. For example, it might send Page Up to a program when you scroll the wheel forwards. Imwheel uses a configuration file to map the wheel events to keypresses so that it can send different keys to different applications. The default imwheel configuration file is installed in /usr/X11R6/etc/imwheelrc. You can copy it to ~/.imwheelrc and then edit it if you wish to customize imwheel's configuration. The format of the configuration file is documented in &man.imwheel.1;. Configure Emacs to Work with Imwheel (optional) If you use emacs or Xemacs, then you need to add a small section to your ~/.emacs file. For emacs, add the following: <application>Emacs</application> Configuration for <application>Imwheel</application> ;;; For imwheel (setq imwheel-scroll-interval 3) (defun imwheel-scroll-down-some-lines () (interactive) (scroll-down imwheel-scroll-interval)) (defun imwheel-scroll-up-some-lines () (interactive) (scroll-up imwheel-scroll-interval)) (global-set-key [?\M-\C-\)] 'imwheel-scroll-up-some-lines) (global-set-key [?\M-\C-\(] 'imwheel-scroll-down-some-lines) ;;; end imwheel section For Xemacs, add the following to your ~/.emacs file instead: <application>Xemacs</application> Configuration for <application>Imwheel</application> ;;; For imwheel (setq imwheel-scroll-interval 3) (defun imwheel-scroll-down-some-lines () (interactive) (scroll-down imwheel-scroll-interval)) (defun imwheel-scroll-up-some-lines () (interactive) (scroll-up imwheel-scroll-interval)) (define-key global-map [(control meta \))] 'imwheel-scroll-up-some-lines) (define-key global-map [(control meta \()] 'imwheel-scroll-down-some-lines) ;;; end imwheel section Run Imwheel You can just type imwheel in an xterm to start it up once it is installed. It will background itself and take effect immediately. If you want to always use imwheel, simply add it to your .xinitrc or .xsession file. You can safely ignore any warnings imwheel displays about PID files. Those warnings only apply to the Linux version of imwheel. Why do X Window menus and dialog boxes not work right? Try turning off the Num Lock key. If your Num Lock key is on by default at boot-time, you may add the following line in the Keyboard section of the XF86Config file. # Let the server do the NumLock processing. This should only be # required when using pre-R6 clients ServerNumLock What is a virtual console and how do I make more? Virtual consoles, put simply, enable you to have several simultaneous sessions on the same machine without doing anything complicated like setting up a network or running X. When the system starts, it will display a login prompt on the monitor after displaying all the boot messages. You can then type in your login name and password and start working (or playing!) on the first virtual console. At some point, you will probably wish to start another session, perhaps to look at documentation for a program you are running or to read your mail while waiting for an FTP transfer to finish. Just do Alt-F2 (hold down the Alt key and press the F2 key), and you will find a login prompt waiting for you on the second virtual console! When you want to go back to the original session, do Alt-F1. The default FreeBSD installation has three virtual consoles enabled (8 starting with 3.3-RELEASE), and Alt-F1, Alt-F2, and Alt-F3 will switch between these virtual consoles. To enable more of them, edit /etc/ttys (see &man.ttys.5;) and add entries for ttyv4 to ttyvc after the comment on Virtual terminals: # Edit the existing entry for ttyv3 in /etc/ttys and change # "off" to "on". ttyv3 "/usr/libexec/getty Pc" cons25 on secure ttyv4 "/usr/libexec/getty Pc" cons25 on secure ttyv5 "/usr/libexec/getty Pc" cons25 on secure ttyv6 "/usr/libexec/getty Pc" cons25 on secure ttyv7 "/usr/libexec/getty Pc" cons25 on secure ttyv8 "/usr/libexec/getty Pc" cons25 on secure ttyv9 "/usr/libexec/getty Pc" cons25 on secure ttyva "/usr/libexec/getty Pc" cons25 on secure ttyvb "/usr/libexec/getty Pc" cons25 on secure Use as many or as few as you want. The more virtual terminals you have, the more resources that are used; this can be important if you have 8MB RAM or less. You may also want to change the secure to insecure. If you want to run an X server you must leave at least one virtual terminal unused (or turned off) for it to use. That is to say that if you want to have a login prompt pop up for all twelve of your Alt-function keys, you are out of luck - you can only do this for eleven of them if you also want to run an X server on the same machine. The easiest way to disable a console is by turning it off. For example, if you had the full 12 terminal allocation mentioned above and you wanted to run X, you would change settings for virtual terminal 12 from: ttyvb "/usr/libexec/getty Pc" cons25 on secure to: ttyvb "/usr/libexec/getty Pc" cons25 off secure If your keyboard has only ten function keys, you would end up with: ttyv9 "/usr/libexec/getty Pc" cons25 off secure ttyva "/usr/libexec/getty Pc" cons25 off secure ttyvb "/usr/libexec/getty Pc" cons25 off secure (You could also just delete these lines.) Once you have edited /etc/ttys, the next step is to make sure that you have enough virtualterminal devices. The easiest way to do this is: &prompt.root; cd /dev &prompt.root; sh MAKEDEV vty12 Next, the easiest (and cleanest) way to activate the virtual consoles is to reboot. However, if you really do not want to reboot, you can just shut down the X Window system and execute (as root): &prompt.root; kill -HUP 1 It is imperative that you completely shut down X Window if it is running, before running this command. If you don't, your system will probably appear to hang/lock up after executing the kill command. How do I access the virtual consoles from X? Use Ctrl Alt Fn to switch back to a virtual console. Ctrl Alt F1 would return you to the first virtual console. Once you are back to a text console, you can then use Alt Fn as normal to move between them. To return to the X session, you must switch to the virtual console running X. If you invoked X from the command line, (e.g., using startx) then the X session will attach to the next unused virtual console, not the text console from which it was invoked. If you have eight active virtual terminals then X will be running on the ninth, and you would use Alt F9 to return. How do I start XDM on boot? There are two schools of thought on how to start xdm. One school starts xdm from /etc/ttys (see &man.ttys.5;) using the supplied example, while the other simply runs xdm from rc.local (see &man.rc.8;) or from a X.sh script in /usr/local/etc/rc.d. Both are equally valid, and one may work in situations where the other does not. In both cases the result is the same: X will popup a graphical login: prompt. The ttys method has the advantage of documenting which vty X will start on and passing the responsibility of restarting the X server on logout to init. The rc.local method makes it easy to kill xdm if there is a problem starting the X server. If loaded from rc.local, xdm should be started without any arguments (i.e., as a daemon). xdm must start AFTER getty runs, or else getty and xdm will conflict, locking out the console. The best way around this is to have the script sleep 10 seconds or so then launch xdm. If you are to start xdm from /etc/ttys, there still is a chance of conflict between xdm and &man.getty.8;. One way to avoid this is to add the vt number in the /usr/X11R6/lib/X11/xdm/Xservers file. :0 local /usr/X11R6/bin/X vt4 The above example will direct the X server to run in /dev/ttyv3. Note the number is offset by one. The X server counts the vty from one, whereas the FreeBSD kernel numbers the vty from zero. Why do I get Couldn't open console when I run xconsole? If you start X with startx, the permissions on /dev/console will not get changed, resulting in things like xterm -C and xconsole not working. This is because of the way console permissions are set by default. On a multi-user system, one does not necessarily want just any user to be able to write on the system console. For users who are logging directly onto a machine with a VTY, the &man.fbtab.5; file exists to solve such problems. In a nutshell, make sure an uncommented line of the form /dev/ttyv0 0600 /dev/console is in /etc/fbtab (see &man.fbtab.5;) and it will ensure that whomever logs in on /dev/ttyv0 will own the console. Before, I was able to run XFree86 as a regular user. Why does it now say that I must be root? All X servers need to be run as root in order to get direct access to your video hardware. Older versions of XFree86 (<= 3.3.6) installed all bundled servers to be automatically run as root (setuid to root). This is obviously a security hazard because X servers are large, complicated programs. Newer versions of XFree86 do not install the servers setuid to root for just this reason. Obviously, running an X server as the root user is not acceptable, nor a good idea security-wise. There are two ways to be able to use X as a regular user. The first is to use xdm or another display manager (e.g., kdm); the second is to use the Xwrapper. xdm is a daemon that handles graphical logins. It is usually started at boot time, and is responsible for authenticating users and starting their sessions; it is essentially the graphical counterpart of &man.getty.8; and &man.login.1;. For more information on xdm see the XFree86 documentation, and the the FAQ entry on it. Xwrapper is the X server wrapper; it is a small utility to enable one to manually run an X server while maintaining reasonable safety. It performs some sanity checks on the command line arguments given, and if they pass, runs the appropriate X server. If you do not want to run a display manger for whatever reason, this is for you. If you have installed the complete ports collection, you can find the port in /usr/ports/x11/wrapper. Why does my PS/2 mouse misbehave under X? Your mouse and the mouse driver may have somewhat become out of synchronization. In versions 2.2.5 and earlier, switching away from X to a virtual terminal and getting back to X again may make them re-synchronized. If the problem occurs often, you may add the following option in your kernel configuration file and recompile it. options PSM_CHECKSYNC See the section on building a kernel if you have no experience with building kernels. With this option, there should be less chance of synchronization problem between the mouse and the driver. If, however, you still see the problem, click any mouse button while holding the mouse still to re-synchronize the mouse and the driver. Note that unfortunately this option may not work with all the systems and voids the tap feature of the ALPS GlidePoint device attached to the PS/2 mouse port. In versions 2.2.6 and later, synchronization check is done in a slightly better way and is standard in the PS/2 mouse driver. It should even work with GlidePoint. (As the check code has become a standard feature, PSM_CHECKSYNC option is not available in these versions.) However, in rare case the driver may erroneously report synchronization problem and you may see the kernel message: psmintr: out of sync (xxxx != yyyy) and find your mouse does not seem to work properly. If this happens, disable the synchronization check code by setting the driver flags for the PS/2 mouse driver to 0x100. Enter UserConfig by giving the option at the boot prompt: boot: -c Then, in the UserConfig command line, type: UserConfig> flags psm0 0x100 UserConfig> quit How come my PS/2 mouse from MouseSystems does not seem to work? There have been some reports that certain model of PS/2 mouse from MouseSystems works only if it is put into the high resolution mode. Otherwise, the mouse cursor may jump to the upper-left corner of the screen every so often. Unfortunately there is no workaround for versions 2.0.X and 2.1.X. In versions 2.2 through 2.2.5, apply the following patch to /sys/i386/isa/psm.c and rebuild the kernel. See the section on building a kernel if you have no experience with building kernels. @@ -766,6 +766,8 @@ if (verbose >= 2) log(LOG_DEBUG, "psm%d: SET_DEFAULTS return code:%04x\n", unit, i); + set_mouse_resolution(sc->kbdc, PSMD_RES_HIGH); + #if 0 set_mouse_scaling(sc->kbdc); /* 1:1 scaling */ set_mouse_mode(sc->kbdc); /* stream mode */ In versions 2.2.6 or later, specify the flags 0x04 to the PS/2 mouse driver to put the mouse into the high resolution mode. Enter UserConfig by giving the option at the boot prompt: boot: -c Then, in the UserConfig command line, type: UserConfig> flags psm0 0x04 UserConfig> quit See the previous section for another possible cause of mouse problems. When building an X app, imake cannot find Imake.tmpl. Where is it? Imake.tmpl is part of the Imake package, a standard X application building tool. Imake.tmpl, as well as several header files that are required to build X apps, is contained in the X prog distribution. You can install this from sysinstall or manually from the X distribution files. How do I reverse the mouse buttons? Run the command xmodmap -e "pointer = 3 2 1" from your .xinitrc or .xsession. How do I install a splash screen and where do I find them? Just prior to the release of FreeBSD 3.1, a new feature was added to allow the display of splash screens during the boot messages. The splash screens currently must be a 256 color bitmap (*.BMP) or ZSoft PCX (*.PCX) file. In addition, they must have a resolution of 320x200 or less to work on standard VGA adapters. If you compile VESA support into your kernel, then you can use larger bitmaps up to 1024x768. Note that VESA support requires the VM86 kernel option to be compiled into the kernel. The actual VESA support can either be compiled directly into the kernel with the VESA kernel config option or by loading the VESA kld module during bootup. To use a splash screen, you need to modify the startup files that control the boot process for FreeBSD. The files for this changed prior to the release of FreeBSD 3.2, so there are now two ways of loading a splash screen: FreeBSD 3.1 The first step is to find a bitmap version of your splash screen. Release 3.1 only supports Windows bitmap splash screens. Once you have found your splash screen of choice copy it to /boot/splash.bmp. Next, you need to have a /boot/loader.rc file that contains the following lines: load kernel load -t splash_image_data /boot/splash.bmp load splash_bmp autoboot FreeBSD 3.2+ In addition to adding support for PCX splash screens, FreeBSD 3.2 includes a nicer way of configuring the boot process. If you wish, you can use the method listed above for FreeBSD 3.1. If you do and you want to use PCX, replace splash_bmp with splash_pcx. If, on the other hand, you want to use the newer boot configuration, you need to create a /boot/loader.rc file that contains the following lines: include /boot/loader.4th start and a /boot/loader.conf that contains the following: splash_bmp_load="YES" bitmap_load="YES" This assumes you are using /boot/splash.bmp for your splash screen. If you would rather use a PCX file, copy it to /boot/splash.pcx, create a /boot/loader.rc as instructed above, and create a /boot/loader.conf that contains: splash_pcx_load="YES" bitmap_load="YES" bitmap_name="/boot/splash.pcx" Now all you need is a splash screen. For that you can surf on over to the gallery at http://www.baldwin.cx/splash/. Can I use the Windows(tm) keys on my keyboard in X? Yes. All you need to do is use &man.xmodmap.1; to define what function you wish them to perform. Assuming all Windows(tm) keyboards are standard then the keycodes for the 3 keys are 115 - Windows(tm) key, between the left-hand Ctrl and Alt keys 116 - Windows(tm) key, to the right of the Alt-Gr key 117 - Menu key, to the left of the right-hand Ctrl key To have the left Windows(tm) key print a comma, try this. &prompt.root; xmodmap -e "keycode 115 = comma" You will probably have to re-start your window manager to see the result. To have the Windows(tm) key-mappings enabled automatically every time you start X either put the xmodmap commands in your ~/.xinitrc file or, preferably, create a file ~/.xmodmaprc and include the xmodmap options, one per line, then add the line xmodmap $HOME/.xmodmaprc to your ~/.xinitrc. For example, you could map the 3 keys top be F13, F14, and F15, respectively. This would make it easy to map them to useful functions within applications or your window manager, as demonstrated further down. To do this put the following in ~/.xmodmaprc. keycode 115 = F13 keycode 116 = F14 keycode 117 = F15 If you use fvwm2, for example, you could map the keys so that F13 iconifies (or de-iconifies) the window the cursor is in, F14 brings the window the cursor is in to the front or, if it is already at the front, pushes it to the back, and F15 pops up the main Workplace (application) menu even if the cursor is not on the desktop, which is useful if you do not have any part of the desktop visible (and the logo on the key matches its functionality). The following entries in ~/.fvwmrc implement the aforementioned setup: Key F13 FTIWS A Iconify Key F14 FTIWS A RaiseLower Key F15 A A Menu Workplace Nop Networking Where can I get information on diskless booting? Diskless booting means that the FreeBSD box is booted over a network, and reads the necessary files from a server instead of its hard disk. For full details, please read the Handbook entry on diskless booting Can a FreeBSD box be used as a dedicated network router? Internet standards and good engineering practice prohibit us from providing packet forwarding by default in FreeBSD. You can however enable this feature by changing the following variable to YES in &man.rc.conf.5;: gateway_enable=YES # Set to YES if this host will be a gateway This option will put the &man.sysctl.8; variable net.inet.ip.forwarding to 1. In most cases, you will also need to run a routing process to tell other systems on your network about your router; FreeBSD comes with the standard BSD routing daemon &man.routed.8; or for more complex situations you may want to try GaTeD (available from http://www.gated.org/) which supports FreeBSD as of 3_5Alpha7. It is our duty to warn you that, even when FreeBSD is configured in this way, it does not completely comply with the Internet standard requirements for routers; however, it comes close enough for ordinary usage. Can I connect my Win95 box to the Internet via FreeBSD? Typically, people who ask this question have two PC's at home, one with FreeBSD and one with Win95; the idea is to use the FreeBSD box to connect to the Internet and then be able to access the Internet from the Windows95 box through the FreeBSD box. This is really just a special case of the previous question. ... and the answer is yes! In FreeBSD 3.x, user-mode ppp contains a option. If you run ppp with the , set gateway_enable to YES in /etc/rc.conf, and configure your Windows machine correctly, this should work fine. More detailed information about setting this up can be found in the Pedantic PPP Primer by Steve Sims. If you are using kernel-mode ppp, or have an Ethernet connection to the Internet, you will have to use &man.natd.8;. Please look at the natd section of this FAQ. Why does recompiling the latest BIND from ISC fail? There is a conflict between the cdefs.h file in the distribution and the one shipped with FreeBSD. Just remove compat/include/sys/cdefs.h. Does FreeBSD support SLIP and PPP? Yes. See the manual pages for &man.slattach.8;, &man.sliplogin.8;, &man.ppp.8;, and &man.pppd.8;. &man.ppp.8; and &man.pppd.8; provide support for both incoming and outgoing connections, while &man.sliplogin.8; deals exclusively with incoming connections, and &man.slattach.8; deals exclusively with outgoing connections. For more information on how to use these, please see the Handbook chapter on PPP and SLIP. If you only have access to the Internet through a shell account, you may want to have a look at the slirp package. It can provide you with (limited) access to services such as ftp and http direct from your local machine. Does FreeBSD support NAT or Masquerading? If you have a local subnet (one or more local machines), but have been allocated only a single IP number from your Internet provider (or even if you receive a dynamic IP number), you may want to look at the &man.natd.8; program. &man.natd.8; allows you to connect an entire subnet to the internet using only a single IP number. The &man.ppp.8; program has similar functionality built in via the switch. The alias library (&man.libalias.3;) is used in both cases. How do I connect two FreeBSD systems over a parallel line using PLIP? Get a laplink cable. Make sure both computer have a kernel with lpt driver support. &prompt.root; dmesg | grep lp lpt0 at 0x378-0x37f irq 7 on isa lpt0: Interrupt-driven lp0: TCP/IP capable interface Plug in the laplink cable into the parallel interface. Configure the network interface parameters for lp0 on both sites as root. For example, if you want connect the host max with moritz max <-----> moritz IP Address 10.0.0.1 10.0.0.2 on max start &prompt.root; ifconfig lp0 10.0.0.1 10.0.0.2 on moritz start &prompt.root; ifconfig lp0 10.0.0.2 10.0.0.1 Thats all! Please read also the manpages &man.lp.4; and &man.lpt.4; . You should also add the hosts to /etc/hosts. 127.0.0.1 localhost.my.domain localhost 10.0.0.1 max.my.domain max 10.0.0.2 moritz.my.domain To check if it works do: on max: &prompt.root; ifconfig lp0 lp0: flags=8851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet 10.0.0.1 --> 10.0.0.2 netmask 0xff000000 &prompt.root; netstat -r Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire moritz max UH 4 127592 lp0 &prompt.root; ping -c 4 moritz PING moritz (10.0.0.2): 56 data bytes 64 bytes from 10.0.0.2: icmp_seq=0 ttl=255 time=2.774 ms 64 bytes from 10.0.0.2: icmp_seq=1 ttl=255 time=2.530 ms 64 bytes from 10.0.0.2: icmp_seq=2 ttl=255 time=2.556 ms 64 bytes from 10.0.0.2: icmp_seq=3 ttl=255 time=2.714 ms --- moritz ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max/stddev = 2.530/2.643/2.774/0.103 ms How come I cannot create a /dev/ed0 device? In the Berkeley networking framework, network interfaces are only directly accessible by kernel code. Please see the /etc/rc.network file and the manual pages for the various network programs mentioned there for more information. If this leaves you totally confused, then you should pick up a book describing network administration on another BSD-related operating system; with few significant exceptions, administering networking on FreeBSD is basically the same as on SunOS 4.0 or Ultrix. How can I setup Ethernet aliases? Add netmask 0xffffffff to your &man.ifconfig.8; command-line like the following: &prompt.root; ifconfig ed0 alias 204.141.95.2 netmask 0xffffffff How do I get my 3C503 to use the other network port? If you want to use the other ports, you will have to specify an additional parameter on the &man.ifconfig.8; command line. The default port is link0. To use the AUI port instead of the BNC one, use link2. These flags should be specified using the ifconfig_* variables in /etc/rc.conf (see &man.rc.conf.5;). Why am I having trouble with NFS and FreeBSD? Certain PC network cards are better than others (to put it mildly) and can sometimes cause problems with network intensive applications like NFS. See the Handbook entry on NFS for more information on this topic. Why can't I NFS-mount from a Linux box? Some versions of the Linux NFS code only accept mount requests from a privileged port; try &prompt.root; mount -o -P linuxbox:/blah /mnt Why can't I NFS-mount from a Sun box? Sun workstations running SunOS 4.X only accept mount requests from a privileged port; try &prompt.root; mount -o -P sunbox:/blah /mnt Why does mountd keep telling me it can't change attributes and that I have a bad exports list on my FreeBSD NFS server? The most frequent problem is not understanding this passage from the &man.exports.5; manual page correctly:
Each line in the file (other than comment lines that begin with a #) specifies the mount point(s) and export flags within one local server filesystem for one or more hosts. A host may be specified only once for each local filesystem on the server and there may be only one default entry for each server filesystem that applies to all other hosts.
This is made more clear by an example of a common mistake. If everything above /usr is part of one filesystem (there are no mounts above /usr) the following exports list is not valid: /usr/src client /usr/ports client There are two lines specifying properties for one filesystem, /usr, exported to the same host, client. The correct format is: /usr/src /usr/ports client To rephrase the passage from the manual page, the properties of one filesystem exported to a given host (world-wide exports are treated like another unique host) must all occur on one line. And yes, this does cause limitation in how you can export filesystems without ugly workarounds, but for most people, this is not an issue. The following is an example of a valid export list, where /usr and /exports are local filesystems: # Export src and ports to client01 and client02, but only # client01 has root privileges on it /usr/src /usr/ports -maproot=0 client01 /usr/src /usr/ports client02 # The "client" machines have root and can mount anywhere # up /exports. The world can mount /exports/obj read-only /exports -alldirs -maproot=0 client01 client02 /exports/obj -ro Why am I having problems talking PPP to NeXTStep machines? Try disabling the TCP extensions in /etc/rc.conf (see &man.rc.conf.5;) by changing the following variable to NO: tcp_extensions=NO Xylogic's Annex boxes are also broken in this regard and you must use the above change to connect thru them. How do I enable IP multicast support? Multicast host operations are fully supported in FreeBSD 2.0 and later by default. If you want your box to run as a multicast router, you will need to recompile your kernel with the MROUTING option and run &man.mrouted.8;. FreeBSD 2.2 and later will start &man.mrouted.8; at boot time if the flag mrouted_enable is set to "YES" in /etc/rc.conf. MBONE tools are available in their own ports category, mbone. If you are looking for the conference tools vic and vat, look there! For more information, see the Mbone Information Web. Which network cards are based on the DEC PCI chipset? Here is a list compiled by Glen Foster gfoster@driver.nsta.org, with some more modern additions: Network cards based on the DEC PCI chipset Vendor Model ASUS PCI-L101-TB Accton ENI1203 Cogent EM960PCI Compex ENET32-PCI D-Link DE-530 Dayna DP1203, DP2100 DEC DE435, DE450 Danpex EN-9400P3 JCIS Condor JC1260 Linksys EtherPCI Mylex LNP101 SMC EtherPower 10/100 (Model 9332) SMC EtherPower (Model 8432) TopWare TE-3500P Znyx (2.2.x) ZX312, ZX314, ZX342, ZX345, ZX346, ZX348 Znyx (3.x) ZX345Q, ZX346Q, ZX348Q, ZX412Q, ZX414, ZX442, ZX444, ZX474, ZX478, ZX212, ZX214 (10mbps/hd)
Why do I have to use the FQDN for hosts on my site? You will probably find that the host is actually in a different domain; for example, if you are in foo.bar.edu and you wish to reach a host called mumble in the bar.edu domain, you will have to refer to it by the fully-qualified domain name, mumble.bar.edu, instead of just mumble. Traditionally, this was allowed by BSD BIND resolvers. However the current version of bind (see &man.named.8;) that ships with FreeBSD no longer provides default abbreviations for non-fully qualified domain names other than the domain you are in. So an unqualified host mumble must either be found as mumble.foo.bar.edu, or it will be searched for in the root domain. This is different from the previous behavior, where the search continued across mumble.bar.edu, and mumble.edu. Have a look at RFC 1535 for why this was considered bad practice, or even a security hole. As a good workaround, you can place the line search foo.bar.edu bar.edu instead of the previous domain foo.bar.edu into your /etc/resolv.conf file (see &man.resolv.conf.5;). However, make sure that the search order does not go beyond the boundary between local and public administration, as RFC 1535 calls it. Why do I get an error, Permission denied, for all networking operations? If you have compiled your kernel with the IPFIREWALL option, you need to be aware that the default policy as of 2.1.7R (this actually changed during 2.1-STABLE development) is to deny all packets that are not explicitly allowed. If you had unintentionally misconfigured your system for firewalling, you can restore network operability by typing the following while logged in as root: &prompt.root; ipfw add 65534 allow all from any to any You can also set firewall_type="open" in /etc/rc.conf. For further information on configuring a FreeBSD firewall, see the Handbook section. How much overhead does IPFW incur? The answer to this depends mostly on your rule set and processor speed. For most applications dealing with ethernet and small rule sets, the answer is, negligible. For those of you that need actual measurements to satisfy your curiosity, read on. The following measurements were made using 2.2.5-STABLE on a 486-66. IPFW was modified to measure the time spent within the ip_fw_chk routine, displaying the results to the console every 1000 packets. Two rule sets, each with 1000 rules were tested. The first set was designed to demonstrate a worst case scenario by repeating the rule: &prompt.root; ipfw add deny tcp from any to any 55555 This demonstrates worst case by causing most of IPFW's packet check routine to be executed before finally deciding that the packet does not match the rule (by virtue of the port number). Following the 999th iteration of this rule was an allow ip from any to any. The second set of rules were designed to abort the rule check quickly: &prompt.root; ipfw add deny ip from 1.2.3.4 to 1.2.3.4 The nonmatching source IP address for the above rule causes these rules to be skipped very quickly. As before, the 1000th rule was an allow ip from any to any. The per-packet processing overhead in the former case was approximately 2.703ms/packet, or roughly 2.7 microseconds per rule. Thus the theoretical packet processing limit with these rules is around 370 packets per second. Assuming 10Mbps ethernet and a ~1500 byte packet size, we would only be able to achieve a 55.5% bandwidth utilization. For the latter case each packet was processed in approximately 1.172ms, or roughly 1.2 microseconds per rule. The theoretical packet processing limit here would be about 853 packets per second, which could consume 10Mbps ethernet bandwidth. The excessive number of rules tested and the nature of those rules do not provide a real-world scenario -- they were used only to generate the timing information presented here. Here are a few things to keep in mind when building an efficient rule set: Place an established rule early on to handle the majority of TCP traffic. Do not put any allow tcp statements before this rule. Place heavily triggered rules earlier in the rule set than those rarely used (without changing the permissiveness of the firewall, of course). You can see which rules are used most often by examining the packet counting statistics with ipfw -a l. Why is my ipfw fwd rule to redirect a service to another machine not working? Possibly because you want to do network address translation (NAT) and not just forward packets. A fwd rule does exactly what it says; it forwards packets. It does not actually change the data inside the packet. Say we have a rule like: 01000 fwd 10.0.0.1 from any to foo 21 When a packet with a destination address of foo arrives at the machine with this rule, the packet is forwarded to 10.0.0.1, but it still has the destination address of foo! The destination address of the packet is not changed to 10.0.0.1. Most machines would probably drop a packet that they receive with a destination address that is not their own. Therefore, using a fwd rule does not often work the way the user expects. This behavior is a feature and not a bug. See the FAQ about redirecting services, the &man.natd.8; manual, or one of the several port redirecting utilities in the ports collection for a correct way to do this. How can I redirect service requests from one machine to another? You can redirect FTP (and other service) request with the socket package, available in the ports tree in category sysutils. Simply replace the service's commandline to call socket instead, like so: ftp stream tcp nowait nobody /usr/local/bin/socket socket ftp.foo.com ftp where ftp.foo.com and ftp are the host and port to redirect to, respectively. Where can I get a bandwidth management tool? There are two bandwidth management tools available for FreeBSD. ALTQ is available for free; Bandwidth Manager from Emerging Technologies is a commercial product. BIND (named) is listening on port 53 and some other high-numbered port. Has my host been compromised? Probably not. FreeBSD 3.0 and later use a version of BIND that uses a random high-numbered port for outgoing queries. If you want to use port 53 for outgoing queries, either to get past a firewall or to make yourself feel better, you can try the following in /etc/namedb/named.conf: options { query-source address * port 53; }; You can replace the * with a single IP address if you want to tighten things further. Congratulations, by the way. It is good practice to read your &man.sockstat.1; output and notice odd things! Why do I get /dev/bpf0: device not configured? The Berkeley Packet Filter (&man.bpf.4;) driver needs to be enabled before running programs that utilize it. Add this to your kernel config file and build a new kernel: pseudo-device bpfilter # Berkeley Packet Filter Secondly, after rebooting you will have to create the device node. This can be accomplished by a change to the /dev directory, followed by the execution of: &prompt.root; sh MAKEDEV bpf0 Please see the handbook's entry on device nodes for more information on creating devices. How do I mount a disk from a Windows machine that is on my network, like smbmount in Linux? Use the sharity light package in the ports collection. What are these messages about icmp-response bandwidth limit 300/200 pps in my log files? This is the kernel telling you that some activity is provoking it to send more ICMP or TCP reset (RST) responses than it thinks it should. ICMP responses are often generated as a result of attempted connections to unused UDP ports. TCP resets are generated as a result of attempted connections to unopened TCP ports. Among others, these are the kinds of activities which may cause these messages: Brute-force denial of service (DoS) attacks (as opposed to single-packet attacks which exploit a specific vulnerability). Port scans which attempt to connect to a large number of ports (as opposed to only trying a few well-known ports). The first number in the message tells you how many packets the kernel would have sent if the limit was not in place, and the second number tells you the limit. You can control the limit using the net.inet.icmp.icmplim sysctl variable like this, where 300 is the limit in packets per second: &prompt.root; sysctl -w net.inet.icmp.icmplim=300 If you do not want to see messages about this in your log files, but you still want the kernel to do response limiting, you can use the net.inet.icmp.icmplim_output sysctl variable to disable the output like this: &prompt.root; sysctl -w net.inet.icmp.icmplim_output=0 Finally, if you want to disable response limiting, you can set the net.inet.icmp.icmplim sysctl variable (see above for an example) to 0. Disabling response limiting is discouraged for the reasons listed above. PPP I cannot make &man.ppp.8; work. What am I doing wrong? You should first read the &man.ppp.8; man page and the ppp section of the handbook. Enable logging with the command set log Phase Chat Connect Carrier lcp ipcp ccp command This command may be typed at the ppp command prompt or it may be entered in the /etc/ppp/ppp.conf configuration file (the start of the default section is the best place to put it). Make sure that /etc/syslog.conf (see &man.syslog.conf.5;) contains the lines !ppp *.* /var/log/ppp.log and that the file /var/log/ppp.log exists. You can now find out a lot about what is going on from the log file. Do not worry if it does not all make sense. If you need to get help from someone, it may make sense to them. If your version of ppp does not understand the set log command, you should download the latest version. It will build on FreeBSD version 2.1.5 and higher. Why does &man.ppp.8; hang when I run it? This is usually because your hostname will not resolve. The best way to fix this is to make sure that /etc/hosts is consulted by your resolver first by editing /etc/host.conf and putting the hosts line first. Then, simply put an entry in /etc/hosts for your local machine. If you have no local network, change your localhost line: 127.0.0.1 foo.bar.com foo localhost Otherwise, simply add another entry for your host. Consult the relevant man pages for more details. You should be able to successfully ping -c1 `hostname` when you are done. Why won't &man.ppp.8; dial in -auto mode? First, check that you have got a default route. By running netstat -rn (see &man.netstat.1;), you should see two entries like this: Destination Gateway Flags Refs Use Netif Expire default 10.0.0.2 UGSc 0 0 tun0 10.0.0.2 10.0.0.1 UH 0 0 tun0 This is assuming that you have used the addresses from the handbook, the man page or from the ppp.conf.sample file. If you haven't got a default route, it may be because you are running an old version of &man.ppp.8; that does not understand the word HISADDR in the ppp.conf file. If your version of ppp is from before FreeBSD 2.2.5, change the add 0 0 HISADDR line to one saying add 0 0 10.0.0.2 Another reason for the default route line being missing is that you have mistakenly set up a default router in your /etc/rc.conf (see &man.rc.conf.5;) file (this file was called /etc/sysconfig prior to release 2.2.2), and you have omitted the line saying delete ALL from ppp.conf. If this is the case, go back to the Final system configuration section of the handbook. What does No route to host mean? This error is usually due to a missing MYADDR: delete ALL add 0 0 HISADDR section in your /etc/ppp/ppp.linkup file. This is only necessary if you have a dynamic IP address or do not know the address of your gateway. If you are using interactive mode, you can type the following after entering packet mode (packet mode is indicated by the capitalized PPP in the prompt): delete ALL add 0 0 HISADDR Refer to the PPP and Dynamic IP addresses section of the handbook for further details. Why does my connection drop after about 3 minutes? The default ppp timeout is 3 minutes. This can be adjusted with the line set timeout NNN where NNN is the number of seconds of inactivity before the connection is closed. If NNN is zero, the connection is never closed due to a timeout. It is possible to put this command in the ppp.conf file, or to type it at the prompt in interactive mode. It is also possible to adjust it on the fly while the line is active by connecting to ppps server socket using &man.telnet.1; or &man.pppctl.8;. Refer to the &man.ppp.8; man page for further details. Why does my connection drop under heavy load? If you have Link Quality Reporting (LQR) configured, it is possible that too many LQR packets are lost between your machine and the peer. Ppp deduces that the line must therefore be bad, and disconnects. Prior to FreeBSD version 2.2.5, LQR was enabled by default. It is now disabled by default. LQR can be disabled with the line disable lqr Why does my connection drop after a random amount of time? Sometimes, on a noisy phone line or even on a line with call waiting enabled, your modem may hang up because it thinks (incorrectly) that it lost carrier. There is a setting on most modems for determining how tolerant it should be to temporary losses of carrier. On a USR Sportster for example, this is measured by the S10 register in tenths of a second. To make your modem more forgiving, you could add the following send-expect sequence to your dial string: set dial "...... ATS10=10 OK ......" Refer to your modem manual for details. Why does my connection hang after a random amount of time? Many people experience hung connections with no apparent explanation. The first thing to establish is which side of the link is hung. If you are using an external modem, you can simply try using &man.ping.8; to see if the TD light is flashing when you transmit data. If it flashes (and the RD light does not), the problem is with the remote end. If TD does not flash, the problem is local. With an internal modem, you will need to use the set server command in your ppp.conf file. When the hang occurs, connect to ppp using pppctl. If your network connection suddenly revives (ppp was revived due to the activity on the diagnostic socket) or if you cannot connect (assuming the set socket command succeeded at startup time), the problem is local. If you can connect and things are still hung, enable local async logging with set log local async and use &man.ping.8; from another window or terminal to make use of the link. The async logging will show you the data being transmitted and received on the link. If data is going out and not coming back, the problem is remote. Having established whether the problem is local or remote, you now have two possibilities: The remote end is not responding. What can I do? There is very little you can do about this. Most ISPs will refuse to help if you are not running a Microsoft OS. You can enable lqr in your ppp.conf file, allowing ppp to detect the remote failure and hang up, but this detection is relatively slow and therefore not that useful. You may want to avoid telling your ISP that you are running user-ppp.... First, try disabling all local compression by adding the following to your configuration: disable pred1 deflate deflate24 protocomp acfcomp shortseq vj deny pred1 deflate deflate24 protocomp acfcomp shortseq vj Then reconnect to ensure that this makes no difference. If things improve or if the problem is solved completely, determine which setting makes the difference through trial and error. This will provide good ammunition when you contact your ISP (although it may make it apparent that you are not running a Microsoft product). Before contacting your ISP, enable async logging locally and wait until the connection hangs again. This may use up quite a bit of disk space. The last data read from the port may be of interest. It is usually ascii data, and may even describe the problem (Memory fault, core dumped?). If your ISP is helpful, they should be able to enable logging on their end, then when the next link drop occurs, they may be able to tell you why their side is having a problem. Feel free to send the details to &a.brian;, or even to ask your ISP to contact me directly. &man.ppp.8; has hung. What can I do? Your best bet here is to rebuild ppp by adding CFLAGS+=-g and STRIP= to the end of the Makefile, then doing a make clean && make && make install. When ppp hangs, find the ppp process id with ps ajxww | fgrep ppp and run gdb ppp PID. From the gdb prompt, you can then use bt to get a stack trace. Send the results to brian@Awfulhak.org. Why does nothing happen after the Login OK! message? Prior to FreeBSD version 2.2.5, once the link was established, &man.ppp.8; would wait for the peer to initiate the Line Control Protocol (LCP). Many ISPs will not initiate negotiations and expect the client to do so. To force ppp to initiate the LCP, use the following line: set openmode active It usually does no harm if both sides initiate negotiation, so openmode is now active by default. However, the next section explains when it does do some harm. I keep seeing errors about magic being the same. What does it mean? Occasionally, just after connecting, you may see messages in the log that say magic is the same. Sometimes, these messages are harmless, and sometimes one side or the other exits. Most ppp implementations cannot survive this problem, and even if the link seems to come up, you will see repeated configure requests and configure acknowledgments in the log file until ppp eventually gives up and closes the connection. This normally happens on server machines with slow disks that are spawning a getty on the port, and executing ppp from a login script or program after login. I have also heard reports of it happening consistently when using slirp. The reason is that in the time taken between getty exiting and ppp starting, the client-side ppp starts sending Line Control Protocol (LCP) packets. Because ECHO is still switched on for the port on the server, the client ppp sees these packets reflect back. One part of the LCP negotiation is to establish a magic number for each side of the link so that reflections can be detected. The protocol says that when the peer tries to negotiate the same magic number, a NAK should be sent and a new magic number should be chosen. During the period that the server port has ECHO turned on, the client ppp sends LCP packets, sees the same magic in the reflected packet and NAKs it. It also sees the NAK reflect (which also means ppp must change its magic). This produces a potentially enormous number of magic number changes, all of which are happily piling into the server's tty buffer. As soon as ppp starts on the server, it is flooded with magic number changes and almost immediately decides it has tried enough to negotiate LCP and gives up. Meanwhile, the client, who no longer sees the reflections, becomes happy just in time to see a hangup from the server. This can be avoided by allowing the peer to start negotiating with the following line in your ppp.conf file: set openmode passive This tells ppp to wait for the server to initiate LCP negotiations. Some servers however may never initiate negotiations. If this is the case, you can do something like: set openmode active 3 This tells ppp to be passive for 3 seconds, and then to start sending LCP requests. If the peer starts sending requests during this period, ppp will immediately respond rather than waiting for the full 3 second period. LCP negotiations continue 'till the connection is closed. What is wrong? There is currently an implementation mis-feature in ppp where it does not associate LCP, CCP & IPCP responses with their original requests. As a result, if one ppp implementation is more than 6 seconds slower than the other side, the other side will send two additional LCP configuration requests. This is fatal. Consider two implementations, A and B. A starts sending LCP requests immediately after connecting and B takes 7 seconds to start. When B starts, A has sent 3 LCP REQs. We are assuming the line has ECHO switched off, otherwise we would see magic number problems as described in the previous section. B sends a REQ, then an ACK to the first of A's REQs. This results in A entering the OPENED state and sending and ACK (the first) back to B. In the meantime, B sends back two more ACKs in response to the two additional REQs sent by A before B started up. B then receives the first ACK from A and enters the OPENED state. A receives the second ACK from B and goes back to the REQ-SENT state, sending another (forth) REQ as per the RFC. It then receives the third ACK and enters the OPENED state. In the meantime, B receives the forth REQ from A, resulting in it reverting to the ACK-SENT state and sending another (second) REQ and (forth) ACK as per the RFC. A gets the REQ, goes into REQ-SENT and sends another REQ. It immediately receives the following ACK and enters OPENED. This goes on 'till one side figures out that they are getting nowhere and gives up. The best way to avoid this is to configure one side to be passive - that is, make one side wait for the other to start negotiating. This can be done with the set openmode passive command. Care should be taken with this option. You should also use the set stopped N command to limit the amount of time that ppp waits for the peer to begin negotiations. Alternatively, the set openmode active N command (where N is the number of seconds to wait before starting negotiations) can be used. Check the manual page for details. Why does &man.ppp.8; lock up shortly after connection? Prior to version 2.2.5 of FreeBSD, it was possible that your link was disabled shortly after connection due to ppp mis-handling Predictor1 compression negotiation. This would only happen if both sides tried to negotiate different Compression Control Protocols (CCP). This problem is now corrected, but if you are still running an old version of ppp, the problem can be circumvented with the line disable pred1 Why does &man.ppp.8; lock up when I shell out to test it? When you execute the shell or ! command, ppp executes a shell (or if you have passed any arguments, ppp will execute those arguments). Ppp will wait for the command to complete before continuing. If you attempt to use the ppp link while running the command, the link will appear to have frozen. This is because ppp is waiting for the command to complete. If you wish to execute commands like this, use the !bg command instead. This will execute the given command in the background, and ppp can continue to service the link. How come &man.ppp.8; over a null-modem cable never exits? There is no way for ppp to automatically determine that a direct connection has been dropped. This is due to the lines that are used in a null-modem serial cable. When using this sort of connection, LQR should always be enabled with the line enable lqr LQR is accepted by default if negotiated by the peer. Why does &man.ppp.8; dial for no reason in -auto mode? If ppp is dialing unexpectedly, you must determine the cause, and set up Dial filters (dfilters) to prevent such dialing. To determine the cause, use the following line: set log +tcp/ip This will log all traffic through the connection. The next time the line comes up unexpectedly, you will see the reason logged with a convenient timestamp next to it. You can now disable dialing under these circumstances. Usually, this sort of problem arises due to DNS lookups. To prevent DNS lookups from establishing a connection (this will not prevent ppp from passing the packets through an established connection), use the following: set dfilter 1 deny udp src eq 53 set dfilter 2 deny udp dst eq 53 set dfilter 3 permit 0/0 0/0 This is not always suitable, as it will effectively break your demand-dial capabilities - most programs will need a DNS lookup before doing any other network related things. In the DNS case, you should try to determine what is actually trying to resolve a host name. A lot of the time, &man.sendmail.8; is the culprit. You should make sure that you tell sendmail not to do any DNS lookups in its configuration file. See the section on Mail Configuration for details on how to create your own configuration file and what should go into it. You may also want to add the following line to your .mc file: define(`confDELIVERY_MODE', `d')dnl This will make sendmail queue everything until the queue is run (usually, sendmail is invoked with , telling it to run the queue every 30 minutes) or until a sendmail -q is done (perhaps from your ppp.linkup file). What do these CCP errors mean? I keep seeing the following errors in my log file: CCP: CcpSendConfigReq CCP: Received Terminate Ack (1) state = Req-Sent (6) This is because ppp is trying to negotiate Predictor1 compression, and the peer does not want to negotiate any compression at all. The messages are harmless, but if you wish to remove them, you can disable Predictor1 compression locally too: disable pred1 Why does &man.ppp.8; lock up during file transfers with IO errors? Under FreeBSD 2.2.2 and before, there was a bug in the tun driver that prevents incoming packets of a size larger than the tun interface's MTU size. Receipt of a packet greater than the MTU size results in an IO error being logged via syslogd. The ppp specification says that an MRU of 1500 should always be accepted as a minimum, despite any LCP negotiations, therefore it is possible that should you decrease the MTU to less than 1500, your ISP will transmit packets of 1500 regardless, and you will tickle this non-feature - locking up your link. The problem can be circumvented by never setting an MTU of less than 1500 under FreeBSD 2.2.2 or before. Why doesn't &man.ppp.8; log my connection speed? In order to log all lines of your modem conversation, you must enable the following: set log +connect This will make &man.ppp.8; log everything up until the last requested expect string. If you wish to see your connect speed and are using PAP or CHAP (and therefore do not have anything to chat after the CONNECT in the dial script - no set login script), you must make sure that you instruct ppp to expect the whole CONNECT line, something like this: set dial "ABORT BUSY ABORT NO\\sCARRIER TIMEOUT 4 \ \"\" ATZ OK-ATZ-OK ATDT\\T TIMEOUT 60 CONNECT \\c \\n" Here, we get our CONNECT, send nothing, then expect a line-feed, forcing ppp to read the whole CONNECT response. Why does &man.ppp.8; ignore the \ character in my chat script? Ppp parses each line in your config files so that it can interpret strings such as set phone "123 456 789" correctly (and realize that the number is actually only one argument. In order to specify a " character, you must escape it using a backslash (\). When the chat interpreter parses each argument, it re-interprets the argument in order to find any special escape sequences such as \P or \T (see the man page). As a result of this double-parsing, you must remember to use the correct number of escapes. If you wish to actually send a \ character to (say) your modem, you would need something like: set dial "\"\" ATZ OK-ATZ-OK AT\\\\X OK" resulting in the following sequence: ATZ OK AT\X OK or set phone 1234567 set dial "\"\" ATZ OK ATDT\\T" resulting in the following sequence: ATZ OK ATDT1234567 Why does &man.ppp.8; get a seg-fault, but I see no ppp.core file? Ppp (or any other program for that matter) should never dump core. Because ppp runs with an effective user id of 0, the operating system will not write ppps core image to disk before terminating it. If, however ppp is actually terminating due to a segmentation violation or some other signal that normally causes core to be dumped, and you are sure you are using the latest version (see the start of this section), then you should do the following: &prompt.user; tar xfz ppp-*.src.tar.gz &prompt.user; cd ppp*/ppp &prompt.user; echo STRIP= >>Makefile &prompt.user; echo CFLAGS+=-g >>Makefile &prompt.user; make clean all &prompt.user; su &prompt.root; make install &prompt.root; chmod 555 /usr/sbin/ppp You will now have a debuggable version of ppp installed. You will have to be root to run ppp as all of its privileges have been revoked. When you start ppp, take a careful note of what your current directory was at the time. Now, if and when ppp receives the segmentation violation, it will dump a core file called ppp.core. You should then do the following: &prompt.user; su &prompt.root; gdb /usr/sbin/ppp ppp.core (gdb) bt ..... (gdb) f 0 .... (gdb) i args .... (gdb) l ..... All of this information should be given alongside your question, making it possible to diagnose the problem. If you are familiar with gdb, you may wish to find out some other bits and pieces such as what actually caused the dump and the addresses & values of the relevant variables. Why does the process that forces a dial in auto mode never connect? This was a known problem with ppp set up to negotiate a dynamic local IP number with the peer in auto mode. It is fixed in the latest version - search the man page for iface. The problem was that when that initial program calls &man.connect.2;, the IP number of the tun interface is assigned to the socket endpoint. The kernel creates the first outgoing packet and writes it to the tun device. ppp then reads the packet and establishes a connection. If, as a result of ppps dynamic IP assignment, the interface address is changed, the original socket endpoint will be invalid. Any subsequent packets sent to the peer will usually be dropped. Even if they are not, any responses will not route back to the originating machine as the IP number is no longer owned by that machine. There are several theoretical ways to approach this problem. It would be nicest if the peer would re-assign the same IP number if possible :-) The current version of ppp does this, but most other implementations do not. The easiest method from our side would be to never change the tun interface IP number, but instead to change all outgoing packets so that the source IP number is changed from the interface IP to the negotiated IP on the fly. This is essentially what the iface-alias option in the latest version of ppp is doing (with the help of &man.libalias.3; and ppp's switch) - it is maintaining all previous interface addresses and NATing them to the last negotiated address. Another alternative (and probably the most reliable) would be to implement a system call that changes all bound sockets from one IP to another. ppp would use this call to modify the sockets of all existing programs when a new IP number is negotiated. The same system call could be used by dhcp clients when they are forced to re-bind() their sockets. Yet another possibility is to allow an interface to be brought up without an IP number. Outgoing packets would be given an IP number of 255.255.255.255 up until the first SIOCAIFADDR ioctl is done. This would result in fully binding the socket. It would be up to ppp to change the source IP number, but only if it is set to 255.255.255.255, and only the IP number and IP checksum would need to change. This, however is a bit of a hack as the kernel would be sending bad packets to an improperly configured interface, on the assumption that some other mechanism is capable of fixing things retrospectively. Why don't most games work with the -nat switch? The reason games and the like do not work when libalias is in use is that the machine on the outside will try to open a connection or send (unsolicited) UDP packets to the machine on the inside. The NAT software does not know that it should send these packets to the interior machine. To make things work, make sure that the only thing running is the software that you are having problems with, then either run tcpdump on the tun interface of the gateway or enable ppp tcp/ip logging (set log +tcp/ip) on the gateway. When you start the offending software, you should see packets passing through the gateway machine. When something comes back from the outside, it will be dropped (that is the problem). Note the port number of these packets then shut down the offending software. Do this a few times to see if the port numbers are consistent. If they are, then the following line in the relevant section of /etc/ppp/ppp.conf will make the software functional: nat port proto internalmachine:port port where proto is either tcp or udp, internalmachine is the machine that you want the packets to be sent to and port is the destination port number of the packets. You will not be able to use the software on other machines without changing the above command, and running the software on two internal machines at the same time is out of the question - after all, the outside world is seeing your entire internal network as being just a single machine. If the port numbers are not consistent, there are three more options: Submit support in libalias. Examples of special cases can be found in /usr/src/lib/libalias/alias_*.c (alias_ftp.c is a good prototype). This usually involves reading certain recognised outgoing packets, identifying the instruction that tells the outside machine to initiate a connection back to the internal machine on a specific (random) port and setting up a route in the alias table so that the subsequent packets know where to go. This is the most difficult solution, but it is the best and will make the software work with multiple machines. Use a proxy. The application may support socks5 for example, or (as in the cvsup case) may have a passive option that avoids ever requesting that the peer open connections back to the local machine. Redirect everything to the internal machine using nat addr. This is the sledge-hammer approach. Has anybody made a list of useful port numbers? Not yet, but this is intended to grow into such a list (if any interest is shown). In each example, internal should be replaced with the IP number of the machine playing the game. Asheron's Call nat port udp internal :65000 65000 Manually change the port number within the game to 65000. If you have got a number of machines that you wish to play on assign a unique port number for each (i.e. 65001, 65002, etc) and add a nat port line for each one. Half Life nat port udp internal:27005 27015 PCAnywhere 8.0 nat port udp internal:5632 5632 nat port tcp internal:5631 5631 Quake nat port udp internal:6112 6112 Alternatively, you may want to take a look at www.battle.net for Quake proxy support. Quake 2 nat port udp internal:27901 27910 Red Alert nat port udp internal:8675 8675 nat port udp internal:5009 5009 What are FCS errors? FCS stands for Frame Check Sequence. Each ppp packet has a checksum attached to ensure that the data being received is the data being sent. If the FCS of an incoming packet is incorrect, the packet is dropped and the HDLC FCS count is increased. The HDLC error values can be displayed using the show hdlc command. If your link is bad (or if your serial driver is dropping packets), you will see the occasional FCS error. This is not usually worth worrying about although it does slow down the compression protocols substantially. If you have an external modem, make sure your cable is properly shielded from interference - this may eradicate the problem. If your link freezes as soon as you have connected and you see a large number of FCS errors, this may be because your link is not 8 bit clean. Make sure your modem is not using software flow control (XON/XOFF). If your datalink must use software flow control, use the command set accmap 0x000a0000 to tell ppp to escape the ^Q and ^S characters. Another reason for seeing too many FCS errors may be that the remote end has stopped talking PPP. You may want to enable async logging at this point to determine if the incoming data is actually a login or shell prompt. If you have a shell prompt at the remote end, it is possible to terminate ppp without dropping the line by using the close lcp command (a following term command will reconnect you to the shell on the remote machine. If nothing in your log file indicates why the link might have been terminated, you should ask the remote administrator (your ISP?) why the session was terminated. Why do MacOS and Windows 98 connections freeze when running PPPoE on the gateway? Thanks to Michael Wozniak mwozniak@netcom.ca for figuring this out and Dan Flemming danflemming@mac.com for the Mac solution: This is due to what is called a Black Hole router. MacOS and Windows 98 (and maybe other Microsoft OSs) send TCP packets with a requested segment size too big to fit into a PPPoE frame (MTU is 1500 by default for ethernet) and have the do not fragment bit set (default of TCP) and the Telco router is not sending ICMP must fragment back to the www site you are trying to load. (Alternatively, the router is sending the ICMP packet correctly, but the firewall at the www site is dropping it.) When the www server is sending you frames that do not fit into the PPPoE pipe the Telco router drops them on the floor and your page does not load (some pages/graphics do as they are smaller than a MSS.) This seems to be the default of most Telco PPPoE configurations (if only they knew how to program a router... sigh...) One fix is to use regedit on your 95/98 boxes to add the following registry entry... HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Class\NetTrans\0000\MaxMTU It should be a string with a value 1450 (more accurately it should be 1464 to fit TCP packets into a PPPoE frame perfectly but the 1450 gives you a margin of error for other IP protocols you may encounter). This registry key is reported to have moved to Tcpip\Parameters\Interfaces\ID for adapter\MTU in Windows 2000. Refer to Microsoft Knowledge Base documents Q158474 - Windows TCPIP Registry Entries and Q120642 - TCPIP & NBT Configuration Parameters for Windows NT for more information on changing Windoze MTU to work with a FreeBSD/NAT/PPPoE router. Unfortunately, MacOS does not provide an interface for changing TCP/IP settings. However, there is commercial software available, such as OTAdvancedTuner (OT for OpenTransport, the MacOS TCP/IP stack) by Sustainable Softworks, that will allow users to customize TCP/IP settings. MacOS NAT users should select ip_interface_MTU from the drop-down menu, enter 1450 instead of 1500 in the box, click the box next to Save as Auto Configure, and click Make Active. The latest version of ppp (2.3 or greater) has an enable tcpmssfixup command that will automatically adjust the MSS to an appropriate value. This facility is enabled by default. If you are stuck with an older version of ppp, you may want to look at the tcpmssd port. None of this helps - I am desperate! What can I do? If all else fails, send as much information as you can, including your config files, how you are starting ppp, the relevant parts of your log file and the output of the netstat -rn command (before and after connecting) to the &a.questions; or the comp.unix.bsd.freebsd.misc news group, and someone should point you in the right direction. Serial Communications This section answers common questions about serial communications with FreeBSD. PPP and SLIP are covered in the section. How do I tell if FreeBSD found my serial ports? As the FreeBSD kernel boots, it will probe for the serial ports in your system for which the kernel was configured. You can either watch your system closely for the messages it prints or run the command &prompt.user; dmesg | grep sio after your system is up and running. Here is some example output from the above command: sio0 at 0x3f8-0x3ff irq 4 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A This shows two serial ports. The first is on irq 4, is using port address 0x3f8, and has a 16550A-type UART chip. The second uses the same kind of chip but is on irq 3 and is at port address 0x2f8. Internal modem cards are treated just like serial ports---except that they always have a modem attached to the port. The GENERIC kernel includes support for two serial ports using the same irq and port address settings in the above example. If these settings are not right for your system, or if you've added modem cards or have more serial ports than your kernel is configured for, just reconfigure your kernel. See section about building a kernel for more details. How do I tell if FreeBSD found my modem cards? Refer to the answer to the previous question. I just upgraded to 2.0.5 and my tty0X are missing! How do I solve this problem? Do not worry, they have been merged with the ttydX devices. You will have to change any old configuration files you have, though. How do I access the serial ports on FreeBSD? The third serial port, sio2 (see &man.sio.4;, known as COM3 in DOS), is on /dev/cuaa2 for dial-out devices, and on /dev/ttyd2 for dial-in devices. What is the difference between these two classes of devices? You use ttydX for dial-ins. When opening /dev/ttydX in blocking mode, a process will wait for the corresponding cuaaX device to become inactive, and then wait for the carrier detect line to go active. When you open the cuaaX device, it makes sure the serial port is not already in use by the ttydX device. If the port is available, it steals it from the ttydX device. Also, the cuaaX device does not care about carrier detect. With this scheme and an auto-answer modem, you can have remote users log in and you can still dialout with the same modem and the system will take care of all the conflicts. How do I enable support for a multiport serial card? Again, the section on kernel configuration provides information about configuring your kernel. For a multiport serial card, place an &man.sio.4; line for each serial port on the card in the kernel configuration file. But place the irq and vector specifiers on only one of the entries. All of the ports on the card should share one irq. For consistency, use the last serial port to specify the irq. Also, specify the COM_MULTIPORT option. The following example is for an AST 4-port serial card on irq 7: options "COM_MULTIPORT" device sio4 at isa? port 0x2a0 tty flags 0x781 device sio5 at isa? port 0x2a8 tty flags 0x781 device sio6 at isa? port 0x2b0 tty flags 0x781 device sio7 at isa? port 0x2b8 tty flags 0x781 irq 7 vector siointr The flags indicate that the master port has minor number 7 (0x700), diagnostics enabled during probe (0x080), and all the ports share an irq (0x001). Can FreeBSD handle multiport serial cards sharing irqs? Not yet. You will have to use a different irq for each card. Can I set the default serial parameters for a port? The ttydX (or cuaaX) device is the regular device you will want to open for your applications. When a process opens the device, it will have a default set of terminal I/O settings. You can see these settings with the command &prompt.root; stty -a -f /dev/ttyd1 When you change the settings to this device, the settings are in effect until the device is closed. When it is reopened, it goes back to the default set. To make changes to the default set, you can open and adjust the settings of the initial state device. For example, to turn on CLOCAL mode, 8 bits, and XON/XOFF flow control by default for ttyd5, do: &prompt.root; stty -f /dev/ttyid5 clocal cs8 ixon ixoff A good place to do this is in /etc/rc.serial. Now, an application will have these settings by default when it opens ttyd5. It can still change these settings to its liking, though. You can also prevent certain settings from being changed by an application by making adjustments to the lock state device. For example, to lock the speed of ttyd5 to 57600 bps, do &prompt.root; stty -f /dev/ttyld5 57600 Now, an application that opens ttyd5 and tries to change the speed of the port will be stuck with 57600 bps. Naturally, you should make the initial state and lock state devices writable only by root. The &man.MAKEDEV.8; script does NOT do this when it creates the device entries. How can I enable dialup logins on my modem? So you want to become an Internet service provider, eh? First, you will need one or more modems that can auto-answer. Your modem will need to assert carrier-detect when it detects a carrier and not assert it all the time. It will need to hang up the phone and reset itself when the data terminal ready (DTR) line goes from on to off. It should probably use RTS/CTS flow control or no local flow control at all. Finally, it must use a constant speed between the computer and itself, but (to be nice to your callers) it should negotiate a speed between itself and the remote modem. For many Hayes command-set--compatible modems, this command will make these settings and store them in nonvolatile memory: AT &C1 &D3 &K3 &Q6 S0=1 &W See the section on sending AT commands below for information on how to make these settings without resorting to an MS-DOS terminal program. Next, make an entry in /etc/ttys (see &man.ttys.5;) for the modem. This file lists all the ports on which the operating system will await logins. Add a line that looks something like this: ttyd1 "/usr/libexec/getty std.57600" dialup on insecure This line indicates that the second serial port (/dev/ttyd1) has a modem connected running at 57600 bps and no parity (std.57600, which comes from the file /etc/gettytab, see &man.gettytab.5;). The terminal type for this port is dialup. The port is on and is insecure---meaning root logins on the port are not allowed. For dialin ports like this one, use the ttydX entry. It is common practice to use dialup as the terminal type. Many users set up in their .profile or .login files a prompt for the actual terminal type if the starting type is dialup. The example shows the port as insecure. To become root on this port, you have to login as a regular user, then &man.su.1; to become root. If you use secure then root can login in directly. After making modifications to /etc/ttys, you need to send a hangup or HUP signal to the &man.init.8; process: &prompt.root; kill -HUP 1 This forces the &man.init.8; process to reread /etc/ttys. The init process will then start getty processes on all on ports. You can find out if logins are available for your port by typing &prompt.user; ps -ax | grep '[t]tyd1' You should see something like: 747 ?? I 0:00.04 /usr/libexec/getty std.57600 ttyd1 How can I connect a dumb terminal to my FreeBSD box? If you are using another computer as a terminal into your FreeBSD system, get a null modem cable to go between the two serial ports. If you are using an actual terminal, see its accompanying instructions. Then, modify /etc/ttys (see &man.ttys.5;), like above. For example, if you are hooking up a WYSE-50 terminal to the fifth serial port, use an entry like this: ttyd4 "/usr/libexec/getty std.38400" wyse50 on secure This example shows that the port on /dev/ttyd4 has a wyse50 terminal connected at 38400 bps with no parity (std.38400 from /etc/gettytab, see &man.gettytab.5;) and root logins are allowed (secure). Why can't I run tip or cu? On your system, the programs &man.tip.1; and &man.cu.1; are probably executable only by uucp and group dialer. You can use the group dialer to control who has access to your modem or remote systems. Just add yourself to group dialer. Alternatively, you can let everyone on your system run &man.tip.1; and &man.cu.1; by typing: &prompt.root; chmod 4511 /usr/bin/cu &prompt.root; chmod 4511 /usr/bin/tip My stock Hayes modem is not supported---what can I do? Actually, the man page for &man.tip.1; is out of date. There is a generic Hayes dialer already built in. Just use at=hayes in your /etc/remote (see &man.remote.5;) file. The Hayes driver is not smart enough to recognize some of the advanced features of newer modems---messages like BUSY, NO DIALTONE, or CONNECT 115200 will just confuse it. You should turn those messages off when you use &man.tip.1; (using ATX0&W). Also, the dial timeout for &man.tip.1; is 60 seconds. Your modem should use something less, or else tip will think there is a communication problem. Try ATS7=45&W. Actually, as shipped &man.tip.1; does not yet support it fully. The solution is to edit the file tipconf.h in the directory /usr/src/usr.bin/tip/tip. Obviously you need the source distribution to do this. Edit the line #define HAYES 0 to #define HAYES 1. Then make and make install. Everything works nicely after that. How am I expected to enter these AT commands? Make what is called a direct entry in your /etc/remote file (see &man.remote.5;). For example, if your modem is hooked up to the first serial port, /dev/cuaa0, then put in the following line: cuaa0:dv=/dev/cuaa0:br#19200:pa=none Use the highest bps rate your modem supports in the br capability. Then, type tip cuaa0 (see &man.tip.1;) and you will be connected to your modem. If there is no /dev/cuaa0 on your system, do this: &prompt.root; cd /dev &prompt.root; sh MAKEDEV cuaa0 Or use cu as root with the following command: &prompt.root; cu -lline -sspeed with line being the serial port (e.g. /dev/cuaa0) and speed being the speed (e.g.57600). When you are done entering the AT commands hit ~. to exit. How come the <@> sign for the pn capability does not work? The <@> sign in the phone number capability tells tip to look in /etc/phones for a phone number. But the <@> sign is also a special character in capability files like /etc/remote. Escape it with a backslash: pn=\@ How can I dial a phone number on the command line? Put what is called a generic entry in your /etc/remote file (see &man.remote.5;). For example: tip115200|Dial any phone number at 115200 bps:\ :dv=/dev/cuaa0:br#115200:at=hayes:pa=none:du: tip57600|Dial any phone number at 57600 bps:\ :dv=/dev/cuaa0:br#57600:at=hayes:pa=none:du: Then you can do something like tip -115200 5551234. If you prefer &man.cu.1; over &man.tip.1;, use a generic cu entry: cu115200|Use cu to dial any number at 115200bps:\ :dv=/dev/cuaa1:br#57600:at=hayes:pa=none:du: and type cu 5551234 -s 115200. Do I have to type in the bps rate every time I do that? Put in an entry for tip1200 or cu1200, but go ahead and use whatever bps rate is appropriate with the br capability. &man.tip.1; thinks a good default is 1200 bps which is why it looks for a tip1200 entry. You do not have to use 1200 bps, though. How can I more easily access a number of hosts through a terminal server? Rather than waiting until you are connected and typing CONNECT host each time, use tip's cm capability. For example, these entries in /etc/remote (see &man.remote.5;): pain|pain.deep13.com|Forrester's machine:\ :cm=CONNECT pain\n:tc=deep13: muffin|muffin.deep13.com|Frank's machine:\ :cm=CONNECT muffin\n:tc=deep13: deep13:Gizmonics Institute terminal server:\ :dv=/dev/cuaa2:br#38400:at=hayes:du:pa=none:pn=5551234: will let you type tip pain or tip muffin to connect to the hosts pain or muffin; and tip deep13 to get to the terminal server. Can tip try more than one line for each site? This is often a problem where a university has several modem lines and several thousand students trying to use them... Make an entry for your university in /etc/remote (see &man.remote.5;) and use <\@> for the pn capability: big-university:\ :pn=\@:tc=dialout dialout:\ :dv=/dev/cuaa3:br#9600:at=courier:du:pa=none: Then, list the phone numbers for the university in /etc/phones (see &man.phones.5;): big-university 5551111 big-university 5551112 big-university 5551113 big-university 5551114 &man.tip.1; will try each one in the listed order, then give up. If you want to keep retrying, run &man.tip.1; in a while loop. Why do I have to hit CTRL+P twice to send CTRL+P once? CTRL+P is the default force character, used to tell &man.tip.1; that the next character is literal data. You can set the force character to any other character with the ~s escape, which means set a variable. Type ~sforce=single-char followed by a newline. single-char is any single character. If you leave out single-char, then the force character is the nul character, which you can get by typing CTRL+2 or CTRL+SPACE. A pretty good value for single-char is SHIFT+CTRL+6, which I have seen only used on some terminal servers. You can have the force character be whatever you want by specifying the following in your $HOME/.tiprc file: force=single-char Why is everything I type suddenly in UPPER CASE? You must have pressed CTRL+A, &man.tip.1; raise character, specially designed for people with broken caps-lock keys. Use ~s as above and set the variable raisechar to something reasonable. In fact, you can set it to the same as the force character, if you never expect to use either of these features. Here is a sample .tiprc file perfect for Emacs users who need to type CTRL+2 and CTRL+A a lot: force=^^ raisechar=^^ The ^^ is SHIFT+CTRL+6. How can I do file transfers with tip? If you are talking to another UNIX system, you can send and receive files with ~p (put) and ~t (take). These commands run &man.cat.1; and &man.echo.1; on the remote system to accept and send files. The syntax is: ~p <local-file> [<remote-file>] ~t <remote-file> [<local-file>] There is no error checking, so you probably should use another protocol, like zmodem. How can I run zmodem with tip? First, install one of the zmodem programs from the ports collection (such as one of the two from the comms category, lrzsz or rzsz. To receive files, start the sending program on the remote end. Then, press enter and type ~C rz (or ~C lrz if you installed lrzsz) to begin receiving them locally. To send files, start the receiving program on the remote end. Then, press enter and type ~C sz files (or ~C lsz files) to send them to the remote system. How come FreeBSD cannot seem to find my serial ports, even when the settings are correct? Motherboards and cards with Acer UARTs do not probe properly under the FreeBSD sio probe. Obtain a patch from www.lemis.com to fix your problem. Miscellaneous Questions FreeBSD uses far more swap space than Linux. Why? FreeBSD only appears to use more swap than Linux. In actual fact, it does not. The main difference between FreeBSD and Linux in this regard is that FreeBSD will proactively move entirely idle, unused pages of main memory into swap in order to make more main memory available for active use. Linux tends to only move pages to swap as a last resort. The perceived heavier use of swap is balanced by the more efficient use of main memory. Note that while FreeBSD is proactive in this regard, it does not arbitrarily decide to swap pages when the system is truely idle. Thus you will not find your system all paged out when you get up in the morning after leaving it idle overnight. Why does top show very little free memory even when I have very few programs running? The simple answer is that free memory is wasted memory. Any memory that your programs do not actively allocate is used within the FreeBSD kernel as disk cache. The values shown by &man.top.1; labelled as Inact, Cache, and Buf are all cached data at different aging levels. This cached data means the system does not have to access a slow disk again for data it has accessed recently, thus increasing overall performance. In general, a low value shown for Free memory in &man.top.1; is good, provided it is not very low. Why use (what are) a.out and ELF executable formats? To understand why FreeBSD uses the ELF format, you must first know a little about the 3 currently dominant executable formats for UNIX: Prior to FreeBSD 3.x, FreeBSD used the a.out format. &man.a.out.5; The oldest and classic unix object format. It uses a short and compact header with a magic number at the beginning that is often used to characterize the format (see &man.a.out.5; for more details). It contains three loaded segments: .text, .data, and .bss plus a symbol table and a string table. COFF The SVR3 object format. The header now comprises a section table, so you can have more than just .text, .data, and .bss sections. ELF The successor to COFF, featuring Multiple sections and 32-bit or 64-bit possible values. One major drawback: ELF was also designed with the assumption that there would be only one ABI per system architecture. That assumption is actually quite incorrect, and not even in the commercial SYSV world (which has at least three ABIs: SVR4, Solaris, SCO) does it hold true. FreeBSD tries to work around this problem somewhat by providing a utility for branding a known ELF executable with information about the ABI it is compliant with. See the man page for &man.brandelf.1; for more information. FreeBSD comes from the classic camp and has traditionally used the &man.a.out.5; format, a technology tried and proven through many generations of BSD releases. Though it has also been possible for some time to build and run native ELF binaries (and kernels) on a FreeBSD system, FreeBSD initially resisted the push to switch to ELF as the default format. Why? Well, when the Linux camp made their painful transition to ELF, it was not so much to flee the a.out executable format as it was their inflexible jump-table based shared library mechanism, which made the construction of shared libraries very difficult for vendors and developers alike. Since the ELF tools available offered a solution to the shared library problem and were generally seen as the way forward anyway, the migration cost was accepted as necessary and the transition made. In FreeBSD's case, our shared library mechanism is based more closely on Sun's SunOS-style shared library mechanism and, as such, is very easy to use. However, starting with 3.0, FreeBSD officially supports ELF binaries as the default format. Even though the a.out executable format has served us well, the GNU people, who author the compiler tools we use, have dropped support for the a.out format. This has forced us to maintain a divergent version of the compiler and linker, and has kept us from reaping the benefits of the latest GNU development efforts. Also the demands of ISO-C++, notably constructors and destructors, has also led to native ELF support in future FreeBSD releases. Yes, but why are there so many different formats? Back in the dim, dark past, there was simple hardware. This simple hardware supported a simple, small system. a.out was completely adequate for the job of representing binaries on this simple system (a PDP-11). As people ported unix from this simple system, they retained the a.out format because it was sufficient for the early ports of unix to architectures like the Motorola 68k, VAXen, etc. Then some bright hardware engineer decided that if he could force software to do some sleazy tricks, then he would be able to shave a few gates off the design and allow his CPU core to run faster. While it was made to work with this new kind of hardware (known these days as RISC), a.out was ill-suited for this hardware, so many formats were developed to get to a better performance from this hardware than the limited, simple a.out format could offer. Things like COFF, ECOFF, and a few obscure others were invented and their limitations explored before things seemed to settle on ELF. In addition, program sizes were getting huge and disks (and physical memory) were still relatively small so the concept of a shared library was born. The VM system also became more sophisticated. While each one of these advancements was done using the a.out format, its usefulness was stretched more and more with each new feature. In addition, people wanted to dynamically load things at run time, or to junk parts of their program after the init code had run to save in core memory and/or swap space. Languages became more sophisticated and people wanted code called before main automatically. Lots of hacks were done to the a.out format to allow all of these things to happen, and they basically worked for a time. In time, a.out was not up to handling all these problems without an ever increasing overhead in code and complexity. While ELF solved many of these problems, it would be painful to switch from the system that basically worked. So ELF had to wait until it was more painful to remain with a.out than it was to migrate to ELF. However, as time passed, the build tools that FreeBSD derived their build tools from (the assembler and loader especially) evolved in two parallel trees. The FreeBSD tree added shared libraries and fixed some bugs. The GNU folks that originally write these programs rewrote them and added simpler support for building cross compilers, plugging in different formats at will, etc. Since many people wanted to build cross compilers targeting FreeBSD, they were out of luck since the older sources that FreeBSD had for as and ld were not up to the task. The new gnu tools chain (binutils) does support cross compiling, ELF, shared libraries, C++ extensions, etc. In addition, many vendors are releasing ELF binaries, and it is a good thing for FreeBSD to run them. And if it is running ELF binaries, why bother having a.out any more? It is a tired old horse that has proven useful for a long time, but it is time to turn him out to pasture for his long, faithful years of service. ELF is more expressive than a.out and will allow more extensibility in the base system. The ELF tools are better maintained, and offer cross compilation support, which is important to many people. ELF may be a little slower than a.out, but trying to measure it can be difficult. There are also numerous details that are different between the two in how they map pages, handle init code, etc. None of these are very important, but they are differences. In time support for a.out will be moved out of the GENERIC kernel, and eventually removed from the kernel once the need to run legacy a.out programs is past. Why won't chmod change the permissions on symlinks? Symlinks do not have permissions, and by default, &man.chmod.1; will not follow symlinks to change the permissions on the target file. So if you have a file, foo, and a symlink to that file, bar, then this command will always succeed. &prompt.user; chmod g-w bar However, the permissions on foo will not have changed. You have to use either or together with the option to make this work. See the &man.chmod.1; and &man.symlink.7; man pages for more info. The option does a RECURSIVE &man.chmod.1;. Be careful about specifying directories or symlinks to directories to &man.chmod.1;. If you want to change the permissions of a directory referenced by a symlink, use &man.chmod.1; without any options and follow the symlink with a trailing slash (/). For example, if foo is a symlink to directory bar, and you want to change the permissions of foo (actually bar), you would do something like: &prompt.user; chmod 555 foo/ With the trailing slash, &man.chmod.1; will follow the symlink, foo, to change the permissions of the directory, bar. Why are login names still restricted to 8 characters? You would think it would be easy enough to change UT_NAMESIZE and rebuild the whole world, and everything would just work. Unfortunately there are often scads of applications and utilities (including system tools) that have hard-coded small numbers (not always 8 or 9, but oddball ones like 15 and 20) in structures and buffers. Not only will this get you log files which are trashed (due to variable-length records getting written when fixed records were expected), but it can break Suns NIS clients and potentially cause other problems in interacting with other UNIX systems. In FreeBSD 3.0 and later, the maximum name length has been increased to 16 characters and those various utilities with hard-coded name sizes have been found and fixed. The fact that this touched so many areas of the system is why, in fact, the change was not made until 3.0. If you are absolutely confident in your ability to find and fix these sorts of problems for yourself when and if they pop up, you can increase the login name length in earlier releases by editing /usr/include/utmp.h and changing UT_NAMESIZE accordingly. You must also update MAXLOGNAME in /usr/include/sys/param.h to match the UT_NAMESIZE change. Finally, if you build from sources, do not forget that /usr/include is updated each time! Change the appropriate files in /usr/src/.. instead. Can I run DOS binaries under FreeBSD? Yes, starting with version 3.0 you can using BSDI's doscmd DOS emulation which has been integrated and enhanced. Send mail to the &a.emulation; if you are interested in joining this ongoing effort! For pre-3.0 systems, there is a neat utility called pcemu in the ports collection which emulates an 8088 and enough BIOS services to run DOS text mode applications. It requires the X Window System (provided as XFree86). What do I need to do to translate a FreeBSD document into my native language? See the Translation FAQ in the FreeBSD Documentation Project Primer. Where can I find a free FreeBSD account? While FreeBSD does not provide open access to any of their servers, others do provide open access Unix systems. The charge varies and limited services may be available. Arbornet, Inc, also known as M-Net, has been providing open access to Unix systems since 1983. Starting on an Altos running System III, the site switched to BSD/OS in 1991. In June of 2000, the site switched again to FreeBSD. M-Net can be accessed via telnet and SSH and provides basic access to the entire FreeBSD software suite. However, network access is limited to members and patrons who donate to the system, which is run as a non-profit organization. M-Net also provides an bulletin board system and interactive chat. Grex provides a site very similar to M-Net including the same bulletin board and interactive chat software. However, the machine is a Sun 4M and is running SunOS What is sup, and how do I use it? SUP stands for Software Update Protocol, and was developed by CMU for keeping their development trees in sync. We used it to keep remote sites in sync with our central development sources. SUP is not bandwidth friendly, and has been retired. The current recommended method to keep your sources up to date is Handbook entry on CVSup How cool is FreeBSD? Q. Has anyone done any temperature testing while running FreeBSD? I know Linux runs cooler than dos, but have never seen a mention of FreeBSD. It seems to run really hot. A. No, but we have done numerous taste tests on blindfolded volunteers who have also had 250 micrograms of LSD-25 administered beforehand. 35% of the volunteers said that FreeBSD tasted sort of orange, whereas Linux tasted like purple haze. Neither group mentioned any significant variances in temperature. We eventually had to throw the results of this survey out entirely anyway when we found that too many volunteers were wandering out of the room during the tests, thus skewing the results. We think most of the volunteers are at Apple now, working on their new scratch and sniff GUI. It's a funny old business we're in! Seriously, both FreeBSD and Linux use the HLT (halt) instruction when the system is idle thus lowering its energy consumption and therefore the heat it generates. Also if you have APM (advanced power management) configured, then FreeBSD can also put the CPU into a low power mode. Who is scratching in my memory banks?? Q. Is there anything odd that FreeBSD does when compiling the kernel which would cause the memory to make a scratchy sound? When compiling (and for a brief moment after recognizing the floppy drive upon startup, as well), a strange scratchy sound emanates from what appears to be the memory banks. A. Yes! You will see frequent references to daemons in the BSD documentation, and what most people do not know is that this refers to genuine, non-corporeal entities that now possess your computer. The scratchy sound coming from your memory is actually high-pitched whispering exchanged among the daemons as they best decide how to deal with various system administration tasks. If the noise gets to you, a good fdisk /mbr from DOS will get rid of them, but do not be surprised if they react adversely and try to stop you. In fact, if at any point during the exercise you hear the satanic voice of Bill Gates coming from the built-in speaker, take off running and don't ever look back! Freed from the counterbalancing influence of the BSD daemons, the twin demons of DOS and Windows are often able to re-assert total control over your machine to the eternal damnation of your soul. Now that you know, given a choice you would probably prefer to get used to the scratchy noises, no? What does MFC mean? MFC is an acronym for Merged From -CURRENT. It is used in the CVS logs to denote when a change was migrated from the CURRENT to the STABLE branches. What does BSD mean? It stands for something in a secret language that only members can know. It does not translate literally but its ok to tell you that BSD's translation is something between, Formula-1 Racing Team, Penguins are tasty snacks, and We have a better sense of humor than Linux. :-) Seriously, BSD is an acronym for Berkeley Software Distribution, which is the name the Berkeley CSRG (Computer Systems Research Group) chose for their Unix distribution way back when. What is a repo-copy? A repo-copy (which is a short form of repository copy) refers to the direct copying of files within the CVS repository. Without a repo-copy, if a file needed to be copied or moved to another place in the repository, the committer would run cvs add to put the file in its new location, and then cvs rm on the old file if the old copy was being removed. The disadvantage of this method is that the history (i.e. the entries in the CVS logs) of the file would not be copied to the new location. As the FreeBSD Project considers this history very useful, a repository copy is often used instead. This is a process where one of the repository meisters will copy the files directly within the repository, rather than using the &man.cvs.1; program. Why should I care what color the bikeshed is? The really, really short answer is that you should not. The somewhat longer answer is that just because you are capable of building a bikeshed doesn't mean you should stop others from building one just because you don't like the color they plan to paint it. This is a metaphor indicating that you need not argue about every little feature just because you know enough to do so. Some people have commented that the amount of noise generated by a change is inversely proportional to the complexity of the change. The longer and more complete answer is that after a very long argument about whether &man.sleep.1; should take fractional second arguments, &a.phk; posted a long message entitled A bike shed (any colour will do) on greener grass.... The appropriate portions of that message are quoted below.
&a.phk; on freebsd-hackers, October 2, 1999 What is it about this bike shed? Some of you have asked me. It is a long story, or rather it is an old story, but it is quite short actually. C. Northcote Parkinson wrote a book in the early 1960'ies, called Parkinson's Law, which contains a lot of insight into the dynamics of management. [snip a bit of commentary on the book] In the specific example involving the bike shed, the other vital component is an atomic power-plant, I guess that illustrates the age of the book. Parkinson shows how you can go in to the board of directors and get approval for building a multi-million or even billion dollar atomic power plant, but if you want to build a bike shed you will be tangled up in endless discussions. Parkinson explains that this is because an atomic plant is so vast, so expensive and so complicated that people cannot grasp it, and rather than try, they fall back on the assumption that somebody else checked all the details before it got this far. Richard P. Feynmann gives a couple of interesting, and very much to the point, examples relating to Los Alamos in his books. A bike shed on the other hand. Anyone can build one of those over a weekend, and still have time to watch the game on TV. So no matter how well prepared, no matter how reasonable you are with your proposal, somebody will seize the chance to show that he is doing his job, that he is paying attention, that he is here. In Denmark we call it setting your fingerprint. It is about personal pride and prestige, it is about being able to point somewhere and say There! I did that. It is a strong trait in politicians, but present in most people given the chance. Just think about footsteps in wet cement.
How many FreeBSD hackers does it take to change a lightbulb? One thousand, one hundred and seventy-two: Twenty-three to complain to -CURRENT about the lights being out; Four to claim that it is a configuration problem, and that such matters really belong on -questions; Three to submit PRs about it, one of which is misfiled under doc and consists only of "it's dark"; One to commit an untested lightbulb which breaks buildworld, then back it out five minutes later; Eight to flame the PR originators for not including patches in their PRs; Five to complain about buildworld being broken; Thirty-one to answer that it works for them, and they must have cvsupped at a bad time; One to post a patch for a new lightbulb to -hackers; One to complain that he had patches for this three years ago, but when he sent them to -CURRENT they were just ignored, and he has had bad experiences with the PR system; besides, the proposed new lightbulb is non-reflexive; Thirty-seven to scream that lightbulbs do not belong in the base system, that committers have no right to do things like this without consulting the Community, and WHAT IS -CORE DOING ABOUT IT!? Two hundred to complain about the color of the bicycle shed; Three to point out that the patch breaks &man.style.9;; Seventeen to complain that the proposed new lightbulb is under GPL; Five hundred and eighty-six to engage in a flame war about the comparative advantages of the GPL, the BSD license, the MIT license, the NPL, and the personal hygiene of unnamed FSF founders; Seven to move various portions of the thread to -chat and -advocacy; One to commit the suggested lightbulb, even though it shines dimmer than the old one; Two to back it out with a furious flame of a commit message, arguing that FreeBSD is better off in the dark than with a dim lightbulb; Forty-six to argue vociferously about the backing out of the dim lightbulb and demanding a statement from -core; Eleven to request a smaller lightbulb so it will fit their Tamagotchi if we ever decide to port FreeBSD to that platform; Seventy-three to complain about the SNR on -hackers and -chat and unsubscribe in protest; Thirteen to post "unsubscribe", "How do I unsubscribe?", or "Please remove me from the list", followed by the usual footer; One to commit a working lightbulb while everybody is too busy flaming everybody else to notice; Thirty-one to point out that the new lightbulb would shine 0.364% brighter if compiled with TenDRA (although it will have to be reshaped into a cube), and that FreeBSD should therefore switch to TenDRA instead of EGCS; One to complain that the new lightbulb lacks fairings; Nine (including the PR originators) to ask "what is MFC?"; Fifty-seven to complain about the lights being out two weeks after the bulb has been changed. &a.nik; adds: I was laughing quite hard at this. And then I thought, "Hang on, shouldn't there be '1 to document it.' in that list somewhere?" And then I was enlightened :-) This entry is Copyright (c) 1999 &a.des;. Please do not reproduce without attribution. Advanced Topics What are SNAPs and RELEASEs? There are currently three active/semi-active branches in the FreeBSD CVS Repository (the RELENG_2 branch is probably only changed twice a year, which is why there are only three active branches of development): RELENG_2_2 AKA 2.2-STABLE RELENG_3 AKA 3.X-STABLE RELENG_4 AKA 4-STABLE HEAD AKA -CURRENT AKA 5.0-CURRENT HEAD is not an actual branch tag, like the other two; it is simply a symbolic constant for the current, non-branched development stream which we simply refer to as -CURRENT. Right now, -CURRENT is the 5.0 development stream and the 4-STABLE branch, RELENG_4, forked off from -CURRENT in Mar 2000. The 2.2-STABLE branch, RELENG_2_2, departed -CURRENT in November 1996, and has pretty much been retired. How do I make my own custom release? To make a release you need to do three things: First, you need to be running a kernel with the &man.vn.4; driver configured in. Add this to your kernel config file and build a new kernel: pseudo-device vn #Vnode driver (turns a file into a device) Second, you have to have the whole CVS repository at hand. To get this you can use CVSUP but in your supfile set the release name to cvs and remove any tag or date fields: *default prefix=/home/ncvs *default base=/a *default host=cvsup.FreeBSD.org *default release=cvs *default delete compress use-rel-suffix ## Main Source Tree src-all src-eBones src-secure # Other stuff ports-all www doc-all Then run cvsup -g supfile to suck all the good bits onto your box... Finally, you need a chunk of empty space to build into. Let's say it is in /some/big/filesystem, and from the example above you have got the CVS repository in /home/ncvs: &prompt.root; setenv CVSROOT /home/ncvs # or export CVSROOT=/home/ncvs &prompt.root; cd /usr/src &prompt.root; make buildworld &prompt.root; cd /usr/src/release -&prompt.root; make release BUILDNAME=3.0-MY-SNAP CHROOTDIR=/some/big/filesystem/release - +&prompt.root; make release BUILDNAME=3.0-MY-SNAP CHROOTDIR=/some/big/filesystem/release Please note that you do not need to build world if you already have a populated /usr/obj. An entire release will be built in /some/big/filesystem/release and you will have a full FTP-type installation in /some/big/filesystem/release/R/ftp when you are done. If you want to build your SNAP along some other branch than -CURRENT, you can also add RELEASETAG=SOMETAG to the make release command line above, e.g. RELEASETAG=RELENG_2_2 would build an up-to-the- minute 2.2-STABLE snapshot. How do I create customized installation disks? The entire process of creating installation disks and source and binary archives is automated by various targets in /usr/src/release/Makefile. The information there should be enough to get you started. However, it should be said that this involves doing a make world and will therefore take up a lot of time and disk space. Why does make world clobber my existing installed binaries? Yes, this is the general idea; as its name might suggest, make world rebuilds every system binary from scratch, so you can be certain of having a clean and consistent environment at the end (which is why it takes so long). If the environment variable DESTDIR is defined while running make world or make install, the newly-created binaries will be deposited in a directory tree identical to the installed one, rooted at ${DESTDIR}. Some random combination of shared libraries modifications and program rebuilds can cause this to fail in make world however. How come when my system boots, it says (bus speed defaulted)? The Adaptec 1542 SCSI host adapters allow the user to configure their bus access speed in software. Previous versions of the 1542 driver tried to determine the fastest usable speed and set the adapter to that. We found that this breaks some users' systems, so you now have to define the TUNE_1542 kernel configuration option in order to have this take place. Using it on those systems where it works may make your disks run faster, but on those systems where it does not, your data could be corrupted. Can I follow current with limited Internet access? Yes, you can do this without downloading the whole source tree by using the CTM facility. How did you split the distribution into 240k files? Newer BSD based systems have a option to split that allows them to split files on arbitrary byte boundaries. Here is an example from /usr/src/Makefile. bin-tarball: (cd ${DISTDIR}; \ tar cf - . \ gzip --no-name -9 -c | \ split -b 240640 - \ ${RELEASEDIR}/tarballs/bindist/bin_tgz.) I have written a kernel extension, who do I send it to? Please take a look at The Handbook entry on how to submit code. And thanks for the thought! How are Plug N Play ISA cards detected and initialized? By: Frank Durda IV uhclem@nemesis.lonestar.org In a nutshell, there a few I/O ports that all of the PnP boards respond to when the host asks if anyone is out there. So when the PnP probe routine starts, he asks if there are any PnP boards present, and all the PnP boards respond with their model # to a I/O read of the same port, so the probe routine gets a wired-OR yes to that question. At least one bit will be on in that reply. Then the probe code is able to cause boards with board model IDs (assigned by Microsoft/Intel) lower than X to go off-line. It then looks to see if any boards are still responding to the query. If the answer was 0, then there are no boards with IDs above X. Now probe asks if there are any boards below X. If so, probe knows there are boards with a model numbers below X. Probe then asks for boards greater than X-(limit/4) to go off-line. If repeats the query. By repeating this semi-binary search of IDs-in-range enough times, the probing code will eventually identify all PnP boards present in a given machine with a number of iterations that is much lower than what 2^64 would take. The IDs are two 32-bit fields (hence 2ˆ64) + 8 bit checksum. The first 32 bits are a vendor identifier. They never come out and say it, but it appears to be assumed that different types of boards from the same vendor could have different 32-bit vendor ids. The idea of needing 32 bits just for unique manufacturers is a bit excessive. The lower 32 bits are a serial #, ethernet address, something that makes this one board unique. The vendor must never produce a second board that has the same lower 32 bits unless the upper 32 bits are also different. So you can have multiple boards of the same type in the machine and the full 64 bits will still be unique. The 32 bit groups can never be all zero. This allows the wired-OR to show non-zero bits during the initial binary search. Once the system has identified all the board IDs present, it will reactivate each board, one at a time (via the same I/O ports), and find out what resources the given board needs, what interrupt choices are available, etc. A scan is made over all the boards to collect this information. This info is then combined with info from any ECU files on the hard disk or wired into the MLB BIOS. The ECU and BIOS PnP support for hardware on the MLB is usually synthetic, and the peripherals do not really do genuine PnP. However by examining the BIOS info plus the ECU info, the probe routines can cause the devices that are PnP to avoid those devices the probe code cannot relocate. Then the PnP devices are visited once more and given their I/O, DMA, IRQ and Memory-map address assignments. The devices will then appear at those locations and remain there until the next reboot, although there is nothing that says you cannot move them around whenever you want. There is a lot of oversimplification above, but you should get the general idea. Microsoft took over some of the primary printer status ports to do PnP, on the logic that no boards decoded those addresses for the opposing I/O cycles. I found a genuine IBM printer board that did decode writes of the status port during the early PnP proposal review period, but MS said tough. So they do a write to the printer status port for setting addresses, plus that use that address + 0x800, and a third I/O port for reading that can be located anywhere between 0x200 and 0x3ff. Can you assign a major number for a device driver I have written? This depends on whether or not you plan on making the driver publicly available. If you do, then please send us a copy of the driver source code, plus the appropriate modifications to files.i386, a sample configuration file entry, and the appropriate &man.MAKEDEV.8; code to create any special files your device uses. If you do not, or are unable to because of licensing restrictions, then character major number 32 and block major number 8 have been reserved specifically for this purpose; please use them. In any case, we would appreciate hearing about your driver on &a.hackers;. What about alternative layout policies for directories? In answer to the question of alternative layout policies for directories, the scheme that is currently in use is unchanged from what I wrote in 1983. I wrote that policy for the original fast filesystem, and never revisited it. It works well at keeping cylinder groups from filling up. As several of you have noted, it works poorly for find. Most filesystems are created from archives that were created by a depth first search (aka ftw). These directories end up being striped across the cylinder groups thus creating a worst possible scenario for future depth first searches. If one knew the total number of directories to be created, the solution would be to create (total / fs_ncg) per cylinder group before moving on. Obviously, one would have to create some heuristic to guess at this number. Even using a small fixed number like say 10 would make an order of magnitude improvement. To differentiate restores from normal operation (when the current algorithm is probably more sensible), you could use the clustering of up to 10 if they were all done within a ten second window. Anyway, my conclusion is that this is an area ripe for experimentation. Kirk McKusick, September 1998 How can I make the most of the data I see when my kernel panics? [This section was extracted from a mail written by &a.wpaul; on the freebsd-current mailing list by &a.des;, who fixed a few typos and added the bracketed comments] From: Bill Paul <wpaul@skynet.ctr.columbia.edu> Subject: Re: the fs fun never stops To: ben@rosengart.com Date: Sun, 20 Sep 1998 15:22:50 -0400 (EDT) Cc: current@FreeBSD.org [<ben@rosengart.com> posted the following panic message] > Fatal trap 12: page fault while in kernel mode > fault virtual address = 0x40 > fault code = supervisor read, page not present > instruction pointer = 0x8:0xf014a7e5 ^^^^^^^^^^ > stack pointer = 0x10:0xf4ed6f24 > frame pointer = 0x10:0xf4ed6f28 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 80 (mount) > interrupt mask = > trap number = 12 > panic: page fault [When] you see a message like this, it is not enough to just reproduce it and send it in. The instruction pointer value that I highlighted up there is important; unfortunately, it is also configuration dependent. In other words, the value varies depending on the exact kernel image that you are using. If you are using a GENERIC kernel image from one of the snapshots, then it is possible for somebody else to track down the offending function, but if you are running a custom kernel then only you can tell us where the fault occurred. What you should do is this: Write down the instruction pointer value. Note that the 0x8: part at the beginning is not significant in this case: it is the 0xf0xxxxxx part that we want. When the system reboots, do the following: &prompt.user; nm -n /kernel.that.caused.the.panic | grep f0xxxxxx where f0xxxxxx is the instruction pointer value. The odds are you will not get an exact match since the symbols in the kernel symbol table are for the entry points of functions and the instruction pointer address will be somewhere inside a function, not at the start. If you do not get an exact match, omit the last digit from the instruction pointer value and try again, i.e.: &prompt.user; nm -n /kernel.that.caused.the.panic | grep f0xxxxx If that does not yield any results, chop off another digit. Repeat until you get some sort of output. The result will be a possible list of functions which caused the panic. This is a less than exact mechanism for tracking down the point of failure, but it is better than nothing. I see people constantly show panic messages like this but rarely do I see someone take the time to match up the instruction pointer with a function in the kernel symbol table. The best way to track down the cause of a panic is by capturing a crash dump, then using &man.gdb.1; to generate a stack trace on the crash dump. In any case, the method I normally use is this: Set up a kernel config file, optionally adding options DDB if you think you need the kernel debugger for something. (I use this mainly for setting breakpoints if I suspect an infinite loop condition of some kind.) Use config -g KERNELCONFIG to set up the build directory. cd /sys/compile/ KERNELCONFIG; make Wait for kernel to finish compiling. make install reboot The &man.make.1; process will have built two kernels. kernel and kernel.debug. kernel was installed as /kernel, while kernel.debug can be used as the source of debugging symbols for &man.gdb.1;. To make sure you capture a crash dump, you need edit /etc/rc.conf and set dumpdev to point to your swap partition. This will cause the &man.rc.8; scripts to use the &man.dumpon.8; command to enable crash dumps. You can also run &man.dumpon.8; manually. After a panic, the crash dump can be recovered using &man.savecore.8;; if dumpdev is set in /etc/rc.conf, the &man.rc.8; scripts will run &man.savecore.8; automatically and put the crash dump in /var/crash. FreeBSD crash dumps are usually the same size as the physical RAM size of your machine. That is, if you have 64MB of RAM, you will get a 64MB crash dump. Therefore you must make sure there is enough space in /var/crash to hold the dump. Alternatively, you run &man.savecore.8; manually and have it recover the crash dump to another directory where you have more room. It is possible to limit the size of the crash dump by using options MAXMEM=(foo) to set the amount of memory the kernel will use to something a little more sensible. For example, if you have 128MB of RAM, you can limit the kernel's memory usage to 16MB so that your crash dump size will be 16MB instead of 128MB. Once you have recovered the crash dump, you can get a stack trace with &man.gdb.1; as follows: &prompt.user; gdb -k /sys/compile/KERNELCONFIG/kernel.debug /var/crash/vmcore.0 (gdb) where Note that there may be several screens worth of information; ideally you should use &man.script.1; to capture all of them. Using the unstripped kernel image with all the debug symbols should show the exact line of kernel source code where the panic occurred. Usually you have to read the stack trace from the bottom up in order to trace the exact sequence of events that lead to the crash. You can also use &man.gdb.1; to print out the contents of various variables or structures in order to examine the system state at the time of the crash. Now, if you are really insane and have a second computer, you can also configure &man.gdb.1; to do remote debugging such that you can use &man.gdb.1; on one system to debug the kernel on another system, including setting breakpoints, single-stepping through the kernel code, just like you can do with a normal user-mode program. I have not played with this yet as I do not often have the chance to set up two machines side by side for debugging purposes. [Bill adds: "I forgot to mention one thing: if you have DDB enabled and the kernel drops into the debugger, you can force a panic (and a crash dump) just by typing 'panic' at the ddb prompt. It may stop in the debugger again during the panic phase. If it does, type 'continue' and it will finish the crash dump." -ed] Why has dlsym() stopped working for ELF executables? The ELF toolchain does not, by default, make the symbols defined in an executable visible to the dynamic linker. Consequently dlsym() searches on handles obtained from calls to dlopen(NULL, flags) will fail to find such symbols. If you want to search, using dlsym(), for symbols present in the main executable of a process, you need to link the executable using the option to the ELF linker (&man.ld.1;). How can I increase or reduce the kernel address space? By default, the kernel address space is 256 MB on FreeBSD 3.x and 1 GB on FreeBSD 4.x. If you run a network-intensive server (e.g. a large FTP or HTTP server), you might find that 256 MB is not enough. So how do you increase the address space? There are two aspects to this. First, you need to tell the kernel to reserve a larger portion of the address space for itself. Second, since the kernel is loaded at the top of the address space, you need to lower the load address so it does not bump its head against the ceiling. The first goal is achieved by increasing the value of NKPDE in src/sys/i386/include/pmap.h. Here is what it looks like for a 1 GB address space: #ifndef NKPDE #ifdef SMP #define NKPDE 254 /* addressable number of page tables/pde's */ #else #define NKPDE 255 /* addressable number of page tables/pde's */ #endif /* SMP */ #endif To find the correct value of NKPDE, divide the desired address space size (in megabytes) by four, then subtract one for UP and two for SMP. To achieve the second goal, you need to compute the correct load address: simply subtract the address space size (in bytes) from 0x100100000; the result is 0xc0100000 for a 1 GB address space. Set LOAD_ADDRESS in src/sys/i386/conf/Makefile.i386 to that value; then set the location counter in the beginning of the section listing in src/sys/i386/conf/kernel.script to the same value, as follows: OUTPUT_FORMAT("elf32-i386", "elf32-i386", "elf32-i386") OUTPUT_ARCH(i386) ENTRY(btext) SEARCH_DIR(/usr/lib); SEARCH_DIR(/usr/obj/elf/home/src/tmp/usr/i386-unknown-freebsdelf/lib); SECTIONS { /* Read-only sections, merged into text segment: */ . = 0xc0100000 + SIZEOF_HEADERS; .interp : { *(.interp) } Then reconfig and rebuild your kernel. You will probably have problems with &man.ps.1; &man.top.1; and the like; make world should take care of it (or a manual rebuild of libkvm, &man.ps.1; and &man.top.1; after copying the patched pmap.h to /usr/include/vm/. NOTE: the size of the kernel address space must be a multiple of four megabytes. [&a.dg; adds: I think the kernel address space needs to be a power of two, but I am not certain about that. The old(er) boot code used to monkey with the high order address bits and I think expected at least 256MB granularity.] Acknowledgments
FreeBSD Core Team If you see a problem with this FAQ, or wish to submit an entry, please mail the &a.faq;. We appreciate your feedback, and cannot make this a better FAQ without your help!
&a.jkh; Occasional fits of FAQ-reshuffling and updating. &a.dwhite; Services above and beyond the call of duty on freebsd-questions &a.joerg; Services above and beyond the call of duty on Usenet &a.wollman; Networking and formatting Jim Lowe Multicast information &a.pds; FreeBSD FAQ typing machine slavey The FreeBSD Team Kvetching, moaning, submitting data And to any others we have forgotten, apologies and heartfelt thanks! Bibliography 4.4BSD System Manager's Manual Computer Systems Research Group, University of California, Berkeley O'Reilly and Associates 1st Edition June 1994 804 pages ISBN 1-56592-080-5 4.4BSD User's Reference Manual Computer Systems Research Group, University of California, Berkeley O'Reilly and Associates 1st Edition June 1994 905 pages ISBN 1-56592-075-9 4.4BSD User's Supplementary Documents Computer Systems Research Group, University of California, Berkeley O'Reilly and Associates 1st Edition June 1994 712 pages ISBN 1-56592-076-7 4.4BSD Programmer's Reference Manual Computer Systems Research Group, University of California, Berkeley O'Reilly and Associates 1st Edition June 1994 866 pages ISBN 1-56592-078-3 4.4BSD Programmer's Supplementary Documents Computer Systems Research Group, University of California, Berkeley O'Reilly and Associates 1st Edition June 1994 596 pages ISBN 1-56592-079-1 The Design and Implementation of the 4.4BSD Operating System M. K. McKusick Kirk Marshall Keith Bostic Michael J Karels John Quarterman Addison-Wesley
Reading MA
1996 ISBN 0-201-54979-4 Unix System Administration Handbook Evi Nemeth Garth Snyder Scott Seebass Trent R. Hein John Quarterman Prentice-Hall 3rd edition 2000 ISBN 0-13-020601-6 The Complete FreeBSD Greg Lehey Walnut Creek 3rd edition June 1999 773 pages ISBN 1-57176-246-9 The FreeBSD Handbook FreeBSD Documentation Project BSDi 1st Edition November 1999 489 pages ISBN 1-57176-241-8 McKusick et al, 1994 Berkeley Software Architecture Manual, 4.4BSD Edition M. K. McKusick M. J. Karels S. J. Leffler W. N. Joy R. S. Faber 5:1-42 diff --git a/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml b/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml index 1d1b221468..b5dd51e108 100644 --- a/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml @@ -1,3877 +1,3858 @@ Advanced Networking Synopsis The following chapter will cover some of the more frequently used network services on UNIX systems. This, of course, will pertain to configuring said services on your FreeBSD system. Gateways and Routes Contributed by &a.gryphon;. 6 October 1995. route routing gateway subnet For one machine to be able to find another, there must be a mechanism in place to describe how to get from one to the other. This is called Routing. A route is a defined pair of addresses: a destination and a gateway. The pair indicates that if you are trying to get to this destination, send along through this gateway. There are three types of destinations: individual hosts, subnets, and default. The default route is used if none of the other routes apply. We will talk a little bit more about default routes later on. There are also three types of gateways: individual hosts, interfaces (also called links), and ethernet hardware addresses. An example To illustrate different aspects of routing, we will use the following example which is the output of the command netstat -r: Destination Gateway Flags Refs Use Netif Expire default outside-gw UGSc 37 418 ppp0 localhost localhost UH 0 181 lo0 test0 0:e0:b5:36:cf:4f UHLW 5 63288 ed0 77 10.20.30.255 link#1 UHLW 1 2421 foobar.com link#1 UC 0 0 host1 0:e0:a8:37:8:1e UHLW 3 4601 lo0 host2 0:e0:a8:37:8:1e UHLW 0 5 lo0 => host2.foobar.com link#1 UC 0 0 224 link#1 UC 0 0 default route The first two lines specify the default route (which we will cover in the next section) and the localhost route. loopback device The interface (Netif column) that it specifies to use for localhost is lo0, also known as the loopback device. This says to keep all traffic for this destination internal, rather than sending it out over the LAN, since it will only end up back where it started anyway. Ethernet MAC address The next thing that stands out are the 0:e0:... addresses. These are ethernet hardware addresses. FreeBSD will automatically identify any hosts (test0 in the example) on the local ethernet and add a route for that host, directly to it over the ethernet interface, ed0. There is also a timeout (Expire column) associated with this type of route, which is used if we fail to hear from the host in a specific amount of time. In this case the route will be automatically deleted. These hosts are identified using a mechanism known as RIP (Routing Information Protocol), which figures out routes to local hosts based upon a shortest path determination. subnet FreeBSD will also add subnet routes for the local subnet (10.20.30.255 is the broadcast address for the subnet 10.20.30, and foobar.com is the domain name associated with that subnet). The designation link#1 refers to the first ethernet card in the machine. You will notice no additional interface is specified for those. Both of these groups (local network hosts and local subnets) have their routes automatically configured by a daemon called routed. If this is not run, then only routes which are statically defined (ie. entered explicitly) will exist. The host1 line refers to our host, which it knows by ethernet address. Since we are the sending host, FreeBSD knows to use the loopback interface (lo0) rather than sending it out over the ethernet interface. The two host2 lines are an example of what happens when we use an ifconfig alias (see the section of ethernet for reasons why we would do this). The => symbol after the lo0 interface says that not only are we using the loopback (since this is address also refers to the local host), but specifically it is an alias. Such routes only show up on the host that supports the alias; all other hosts on the local network will simply have a link#1 line for such. The final line (destination subnet 224) deals with MultiCasting, which will be covered in a another section. The other column that we should talk about are the Flags. Each route has different attributes that are described in the column. Below is a short table of some of these flags and their meanings: U Up: The route is active. H Host: The route destination is a single host. G Gateway: Send anything for this destination on to this remote system, which will figure out from there where to send it. S Static: This route was configured manually, not automatically generated by the system. C Clone: Generates a new route based upon this route for machines we connect to. This type of route is normally used for local networks. W WasCloned: Indicated a route that was auto-configured based upon a local area network (Clone) route. L Link: Route involves references to ethernet hardware. Default routes default route When the local system needs to make a connection to remote host, it checks the routing table to determine if a known path exists. If the remote host falls into a subnet that we know how to reach (Cloned routes), then the system checks to see if it can connect along that interface. If all known paths fail, the system has one last option: the default route. This route is a special type of gateway route (usually the only one present in the system), and is always marked with a c in the flags field. For hosts on a local area network, this gateway is set to whatever machine has a direct connection to the outside world (whether via PPP link, or your hardware device attached to a dedicated data line). If you are configuring the default route for a machine which itself is functioning as the gateway to the outside world, then the default route will be the gateway machine at your Internet Service Provider's (ISP) site. Let us look at an example of default routes. This is a common configuration: [Local2] <--ether--> [Local1] <--PPP--> [ISP-Serv] <--ether--> [T1-GW] The hosts Local1 and Local2 are at your site, with the formed being your PPP connection to your ISP's Terminal Server. Your ISP has a local network at their site, which has, among other things, the server where you connect and a hardware device (T1-GW) attached to the ISP's Internet feed. The default routes for each of your machines will be: host default gateway interface Local2 Local1 ethernet Local1 T1-GW PPP A common question is Why (or how) would we set the T1-GW to be the default gateway for Local1, rather than the ISP server it is connected to?. Remember, since the PPP interface is using an address on the ISP's local network for your side of the connection, routes for any other machines on the ISP's local network will be automatically generated. Hence, you will already know how to reach the T1-GW machine, so there is no need for the intermediate step of sending traffic to the ISP server. As a final note, it is common to use the address ...1 as the gateway address for your local network. So (using the same example), if your local class-C address space was 10.20.30 and your ISP was using 10.9.9 then the default routes would be: Local2 (10.20.30.2) --> Local1 (10.20.30.1) Local1 (10.20.30.1, 10.9.9.30) --> T1-GW (10.9.9.1) Dual homed hosts dual homed hosts There is one other type of configuration that we should cover, and that is a host that sits on two different networks. Technically, any machine functioning as a gateway (in the example above, using a PPP connection) counts as a dual-homed host. But the term is really only used to refer to a machine that sits on two local-area networks. In one case, the machine as two ethernet cards, each having an address on the separate subnets. Alternately, the machine may only have one ethernet card, and be using ifconfig aliasing. The former is used if two physically separate ethernet networks are in use, the latter if there is one physical network segment, but two logically separate subnets. Either way, routing tables are set up so that each subnet knows that this machine is the defined gateway (inbound route) to the other subnet. This configuration, with the machine acting as a Bridge between the two subnets, is often used when we need to implement packet filtering or firewall security in either or both directions. Routing propagation routing propogation We have already talked about how we define our routes to the outside world, but not about how the outside world finds us. We already know that routing tables can be set up so that all traffic for a particular address space (in our examples, a class-C subnet) can be sent to a particular host on that network, which will forward the packets inbound. When you get an address space assigned to your site, your service provider will set up their routing tables so that all traffic for your subnet will be sent down your PPP link to your site. But how do sites across the country know to send to your ISP? There is a system (much like the distributed DNS information) that keeps track of all assigned address-spaces, and defines their point of connection to the Internet Backbone. The Backbone are the main trunk lines that carry Internet traffic across the country, and around the world. Each backbone machine has a copy of a master set of tables, which direct traffic for a particular network to a specific backbone carrier, and from there down the chain of service providers until it reaches your network. It is the task of your service provider to advertise to the backbone sites that they are the point of connection (and thus the path inward) for your site. This is known as route propagation. Troubleshooting traceroute Sometimes, there is a problem with routing propagation, and some sites are unable to connect to you. Perhaps the most useful command for trying to figure out where a routing is breaking down is the &man.traceroute.8; command. It is equally useful if you cannot seem to make a connection to a remote machine (i.e. &man.ping.8; fails). The &man.traceroute.8; command is run with the name of the remote host you are trying to connect to. It will show the gateway hosts along the path of the attempt, eventually either reaching the target host, or terminating because of a lack of connection. For more information, see the manual page for &man.traceroute.8;. Bridging Written by Steve Peterson steve@zpfe.com. Introduction IP subnet bridge It is sometimes useful to divide one physical network (i.e., an Ethernet segment) into two separate network segments, without having to create IP subnets and use a router to connect the segments together. A device that connects two networks together in this fashion is called a bridge. and a FreeBSD system with two network interface cards can act as a bridge. The bridge works by learning the MAC layer addresses (i.e., Ethernet addresses) of the devices on each of its network interfaces. It forwards traffic between two networks only when its source and destination are on different networks. In many respects, a bridge is like an Ethernet switch with very few ports. Situations where bridging is appropriate There are two common situations in which a bridge is used today. High traffic on a segment Situation one is where your physical network segment is overloaded with traffic, but you don't want for whatever reason to subnet the network and interconnect the subnets with a router. Let's consider an example of a newspaper where the Editorial and Production departments are on the same subnetwork. The Editorial users all use server A for file service, and the Production users are on server B. An Ethernet is used to connect all users together, and high loads on the network are slowing things down. If the Editorial users could be segregated on one network segment and the Production users on another, the two network segments could be connected with a bridge. Only the network traffic destined for interfaces on the "other" side of the bridge would be sent to the other network, reducing congestion on each network segment. Filtering/traffic shaping firewall firewall IP Masquerading The second common situation is where firewall functionality is needed without IP Masquerading (NAT). An example is a small company that is connected via DSL or ISDN to their ISP. They have a 13 address global IP allocation for their ISP and have 10 PCs on their network. In this situation, using a router-based firewall is difficult because of subnetting issues. router DSL ISDN A bridge-based firewall can be configured and dropped into the path just downstream of their DSL/ISDN router without any IP numbering issues. Configuring a bridge Network interface card selection A bridge requires at least two network cards to function. Unfortunately, not all network interface cards as of FreeBSD 4.0 support bridging. Read &man.bridge.4; for details on the cards that are supported. Install and test the two network cards before continuing. Kernel configuration changes kernel configuration kernel configuration options BRIDGE To enable kernel support for bridging, add the options BRIDGE statement to your kernel configuration file, and rebuild your kernel. Firewall support firewall If you are planning to use the bridge as a firewall, you will need to add the IPFIREWALL option as well. Read for general information on configuring the bridge as a firewall. If you need to allow non-IP packets (such as ARP) to flow through the bridge, there is an undocumented firewall option that must be set. This option is IPFIREWALL_DEFAULT_TO_ACCEPT. Note that this changes the default rule for the firewall to accept any packet. Make sure you know how this changes the meaning of your ruleset before you set it. Traffic shaping support If you want to use the bridge as a traffic shaper, you will need to add the DUMMYNET option to your kernel configuration. Read &man.dummynet.4; for further information. Enabling the bridge Add the line net.link.ether.bridge=1 to /etc/sysctl.conf to enable the bridge at runtime. If you want the bridged packets to be filtered by ipfw, you should also add net.link.ether.bridge_ipfw=1 as well. Performance My bridge/firewall is a Pentium 90 with one 3Com 3C900B and one 3C905B. The protected side of the network runs at 10mbps half duplex and the connection between the bridge and my router (a Cisco 675) runs at 100mbps full duplex. With no filtering enabled, I've found that the bridge adds about 0.4 milliseconds of latency to pings from the protected 10mbps network to the Cisco 675. Other information If you want to be able to telnet into the bridge from the network, it is OK to assign one of the network cards an IP address. The consensus is that assigning both cards an address is a bad idea. If you have multiple bridges on your network, there cannot be more than one path between any two workstations. Technically, this means that there is no support for spanning tree link management. NFS Written by &a.unfurl;, 4 March 2000. NFS Among the many different file systems that FreeBSD supports is a very unique type, the Network File System or NFS. NFS allows you to share directories and files on one machine with one or more other machines via the network they are attached to. Using NFS, users and programs can access files on remote systems as if they were local files. NFS has several benefits: Local workstations don't need as much disk space because commonly used data can be stored on a single machine and still remain accessible to everyone on the network. There is no need for users to have unique home directories on every machine on your network. Once they have an established directory that is available via NFS it can be accessed from anywhere. Storage devices such as floppies and CD-ROM drives can be used by other machines on the network eliminating the need for extra hardware. How It Works NFS is composed of two sides – a client side and a server side. Think of it as a want/have relationship. The client wants the data that the server side has. The server shares its data with the client. In order for this system to function properly a few processes have to be configured and running properly. The server has to be running the following daemons: NFS server portmap mountd nfsd nfsd - The NFS Daemon which services requests from NFS clients. mountd - The NFS Mount Daemon which actually carries out requests that nfsd passes on to it. portmap - The portmapper daemon which allows NFS clients to find out which port the NFS server is using. The client side only needs to run a single daemon: NFS client nfsiod nfsiod - The NFS async I/O Daemon which services requests from its NFS server. Configuring NFS NFS configuration Luckily for us, on a FreeBSD system this setup is a snap. The processes that need to be running can all be run at boot time with a few modifications to your /etc/rc.conf file. On the NFS server make sure you have: portmap_enable="YES" nfs_server_enable="YES" nfs_server_flags="-u -t -n 4" mountd_flags="-r" mountd is automatically run whenever the NFS server is enabled. The and flags to nfsd tell it to serve UDP and TCP clients. The flag tells nfsd to start 4 copies of itself. On the client, make sure you have: nfs_client_enable="YES" nfs_client_flags="-n 4" Like nfsd, the tells nfsiod to start 4 copies of itself. The last configuration step requires that you create a file called /etc/exports. The exports file specifies which file systems on your server will be shared (a.k.a., exported) and with what clients they will be shared. Each line in the file specifies a file system to be shared. There are a handful of options that can be used in this file but only a few will be mentioned here. You can find out about the rest in the &man.exports.5; man page. Here are a few example /etc/exports entries: NFS exporting filesystems The following line exports /cdrom to three silly machines that have the same domain name as the server (hence the lack of a domain name for each) or have entries in your /etc/hosts file. The flag makes the shared file system read-only. With this flag, the remote system will not be able to make any changes to the shared file system. /cdrom -ro moe larry curly The following line exports /home to three hosts by IP address. This is a useful setup if you have a private network but do not have DNS running. The flag allows all the directories below the specified file system to be exported as well. /home -alldirs 10.0.0.2 10.0.0.3 10.0.0.4 The following line exports /a to two machines that have different domain names than the server. The flag allows the root user on the remote system to write to the shared file system as root. Without the -maproot=0 flag even if someone has root access on the remote system they won't be able to modify files on the shared file system. /a -maproot=0 host.domain.com box.example.com In order for a client to share an exported file system it must have permission to do so. Make sure your client is listed in your /etc/exports file. It's important to remember that you must restart mountd whenever you modify /etc/exports so that your changes take effect. This can be accomplished by sending the hangup signal to the mountd process : &prompt.root; kill -HUP `cat /var/run/mountd.pid` Now that you have made all these changes you can just reboot and let FreeBSD start everything for you at boot time or you can run the following commands as root: On the NFS server: &prompt.root; portmap &prompt.root; nfsd -u -t -n 4 &prompt.root; mountd -r On the NFS client: &prompt.root; nfsiod -n 4 Now you should be ready to actually mount a remote file system. This can be done one of two ways. In these examples the server's name will be server and the client's name will be client. If you just want to temporarily mount a remote file system or just want to test out your config you can run a command like this as root on the client: NFS mounting filesystems &prompt.root; mount server:/home /mnt This will mount /home on the server on /mnt on the client. If everything is setup correctly you should be able to go into /mnt on the client and see all the files that are on the server. If you want to permanently (each time you reboot) mount a remote file system you need to add it to your /etc/fstab file. Here is an example line: server:/home /mnt nfs rw 0 0 Read the &man.fstab.5; man page for more options. Practical Uses There are many very cool uses for NFS. Some of the more common ones are listed below. NFS uses Have several machines on a network and share a CD-ROM or floppy drive among them. This is cheaper and often more convenient. With so many machines on a network, it gets old having your personal files strewn all over the place. You can have a central NFS server that houses all user home directories and shares them with the rest of the machines on the LAN, so no matter where you log in you will have the same home directory. When you get to reinstalling FreeBSD on one of your machines, NFS is the way to go! Just pop your distribution CD-ROM into your file server and away you go! Have a common /usr/ports/distfiles directory that all your machines share. That way, when you go to install a port that you've already installed on a different machine, you do not have to download the source all over again! Problems integrating with other systems Contributed by &a.jlind;. Certain Ethernet adapters for ISA PC systems have limitations which can lead to serious network problems, particularly with NFS. This difficulty is not specific to FreeBSD, but FreeBSD systems are affected by it. The problem nearly always occurs when (FreeBSD) PC systems are networked with high-performance workstations, such as those made by Silicon Graphics, Inc., and Sun Microsystems, Inc. The NFS mount will work fine, and some operations may succeed, but suddenly the server will seem to become unresponsive to the client, even though requests to and from other systems continue to be processed. This happens to the client system, whether the client is the FreeBSD system or the workstation. On many systems, there is no way to shut down the client gracefully once this problem has manifested itself. The only solution is often to reset the client, because the NFS situation cannot be resolved. Though the correct solution is to get a higher performance and capacity Ethernet adapter for the FreeBSD system, there is a simple workaround that will allow satisfactory operation. If the FreeBSD system is the server, include the option on the mount from the client. If the FreeBSD system is the client, then mount the NFS file system with the option . These options may be specified using the fourth field of the fstab entry on the client for automatic mounts, or by using the parameter of the mount command for manual mounts. It should be noted that there is a different problem, sometimes mistaken for this one, when the NFS servers and clients are on different networks. If that is the case, make certain that your routers are routing the necessary UDP information, or you will not get anywhere, no matter what else you are doing. In the following examples, fastws is the host (interface) name of a high-performance workstation, and freebox is the host (interface) name of a FreeBSD system with a lower-performance Ethernet adapter. Also, /sharedfs will be the exported NFS filesystem (see man exports), and /project will be the mount point on the client for the exported file system. In all cases, note that additional options, such as or and may be desirable in your application. Examples for the FreeBSD system (freebox) as the client: in /etc/fstab on freebox: fastws:/sharedfs /project nfs rw,-r=1024 0 0 As a manual mount command on freebox: &prompt.root; mount -t nfs -o -r=1024 fastws:/sharedfs /project Examples for the FreeBSD system as the server: in /etc/fstab on fastws: freebox:/sharedfs /project nfs rw,-w=1024 0 0 As a manual mount command on fastws: &prompt.root; mount -t nfs -o -w=1024 freebox:/sharedfs /project Nearly any 16-bit Ethernet adapter will allow operation without the above restrictions on the read or write size. For anyone who cares, here is what happens when the failure occurs, which also explains why it is unrecoverable. NFS typically works with a block size of 8k (though it may do fragments of smaller sizes). Since the maximum Ethernet packet is around 1500 bytes, the NFS block gets split into multiple Ethernet packets, even though it is still a single unit to the upper-level code, and must be received, assembled, and acknowledged as a unit. The high-performance workstations can pump out the packets which comprise the NFS unit one right after the other, just as close together as the standard allows. On the smaller, lower capacity cards, the later packets overrun the earlier packets of the same unit before they can be transferred to the host and the unit as a whole cannot be reconstructed or acknowledged. As a result, the workstation will time out and try again, but it will try again with the entire 8K unit, and the process will be repeated, ad infinitum. By keeping the unit size below the Ethernet packet size limitation, we ensure that any complete Ethernet packet received can be acknowledged individually, avoiding the deadlock situation. Overruns may still occur when a high-performance workstations is slamming data out to a PC system, but with the better cards, such overruns are not guaranteed on NFS units. When an overrun occurs, the units affected will be retransmitted, and there will be a fair chance that they will be received, assembled, and acknowledged. Diskless Operation Contributed by &a.martin;. diskless workstation netboot.com/netboot.rom allow you to boot your FreeBSD machine over the network and run FreeBSD without having a disk on your client. Under 2.0 it is now possible to have local swap. Swapping over NFS is also still supported. Supported Ethernet cards include: Western Digital/SMC 8003, 8013, 8216 and compatibles; NE1000/NE2000 and compatibles (requires recompile) Setup Instructions Find a machine that will be your server. This machine will require enough disk space to hold the FreeBSD 2.0 binaries and have bootp, tftp and NFS services available. Tested machines: HP-UX HP9000/8xx running HP-UX 9.04 or later (pre 9.04 doesn't work) Solaris Sun/Solaris 2.3. (you may need to get bootp) Set up a bootp server to provide the client with IP, gateway, netmask. diskless:\ :ht=ether:\ :ha=0000c01f848a:\ :sm=255.255.255.0:\ :hn:\ :ds=192.1.2.3:\ :ip=192.1.2.4:\ :gw=192.1.2.5:\ :vm=rfc1048: TFTP bootp Set up a TFTP server (on same machine as bootp server) to provide booting information to client. The name of this file is cfg.X.X.X.X (or /tftpboot/cfg.X.X.X.X, it will try both) where X.X.X.X is the IP address of the client. The contents of this file can be any valid netboot commands. Under 2.0, netboot has the following commands: help print help list ip print/set client's IP address server print/set bootp/tftp server address netmask print/set netmask hostname name print/set hostname kernel print/set kernel name rootfs print/set root filesystem swapfs print/set swap filesystem swapsize set diskless swapsize in KBytes diskboot boot from disk autoboot continue boot process trans | turn transceiver on|off flags set boot flags A typical completely diskless cfg file might contain: rootfs 192.1.2.3:/rootfs/myclient swapfs 192.1.2.3:/swapfs swapsize 20000 hostname myclient.mydomain A cfg file for a machine with local swap might contain: rootfs 192.1.2.3:/rootfs/myclient hostname myclient.mydomain Ensure that your NFS server has exported the root (and swap if applicable) filesystems to your client, and that the client has root access to these filesystems A typical /etc/exports file on FreeBSD might look like: /rootfs/myclient -maproot=0:0 myclient.mydomain /swapfs -maproot=0:0 myclient.mydomain And on HP-UX: /rootfs/myclient -root=myclient.mydomain /swapfs -root=myclient.mydomain NFS swapping over If you are swapping over NFS (completely diskless configuration) create a swap file for your client using dd. If your swapfs command has the arguments /swapfs and the size 20000 as in the example above, the swapfile for myclient will be called /swapfs/swap.X.X.X.X where X.X.X.X is the client's IP addr, e.g.: &prompt.root; dd if=/dev/zero of=/swapfs/swap.192.1.2.4 bs=1k count=20000 Also, the client's swap space might contain sensitive information once swapping starts, so make sure to restrict read and write access to this file to prevent unauthorized access: &prompt.root; chmod 0600 /swapfs/swap.192.1.2.4 Unpack the root filesystem in the directory the client will use for its root filesystem (/rootfs/myclient in the example above). On HP-UX systems: The server should be running HP-UX 9.04 or later for HP9000/800 series machines. Prior versions do not allow the creation of device files over NFS. When extracting /dev in /rootfs/myclient, beware that some systems (HPUX) will not create device files that FreeBSD is happy with. You may have to go to single user mode on the first bootup (press control-c during the bootup phase), cd /dev and do a sh ./MAKEDEV all from the client to fix this. Run netboot.com on the client or make an EPROM from the netboot.rom file Using Shared <filename>/</filename> and <filename>/usr</filename> filesystems Although this is not an officially sanctioned or supported way of doing this, some people report that it works quite well. If anyone has any suggestions on how to do this cleanly, please tell &a.doc;. Compiling netboot for specific setups Netboot can be compiled to support NE1000/2000 cards by changing the configuration in /sys/i386/boot/netboot/Makefile. See the comments at the top of this file. ISDN A good resource for information on ISDN technology and hardware is Dan Kegel's ISDN Page. A quick simple road map to ISDN follows: If you live in Europe you might want to investigate the ISDN card section. If you are planning to use ISDN primarily to connect to the Internet with an Internet Provider on a dial-up non-dedicated basis, you might look into Terminal Adapters. This will give you the most flexibility, with the fewest problems, if you change providers. If you are connecting two LANs together, or connecting to the Internet with a dedicated ISDN connection, you might consider the stand alone router/bridge option. Cost is a significant factor in determining what solution you will choose. The following options are listed from least expensive to most expensive. ISDN Cards Contributed by &a.hm;. ISDN cards This section is really only relevant to ISDN users in countries where the DSS1/Q.931 ISDN standard is supported. Some growing number of PC ISDN cards are supported under FreeBSD 2.2.x and up by the isdn4bsd driver package. It is still under development but the reports show that it is successfully used all over Europe. isdn4bsd The latest isdn4bsd version is available from ftp://isdn4bsd@ftp.consol.de/pub/, the main isdn4bsd ftp site (you have to log in as user isdn4bsd , give your mail address as the password and change to the pub directory. Anonymous ftp as user ftp or anonymous will not give the desired result). Isdn4bsd allows you to connect to other ISDN routers using either IP over raw HDLC or by using synchronous PPP. A telephone answering machine application is also available. Many ISDN PC cards are supported, mostly the ones with a Siemens ISDN chipset (ISAC/HSCX), support for other chipsets (from Motorola, Cologne Chip Designs) is currently under development. For an up-to-date list of supported cards, please have a look at the README file. In case you are interested in adding support for a different ISDN protocol, a currently unsupported ISDN PC card or otherwise enhancing isdn4bsd, please get in touch with hm@kts.org. A majordomo maintained mailing list is available. To join the list, send mail to &a.majordomo; and specify: subscribe freebsd-isdn in the body of your message. ISDN Terminal Adapters Terminal adapters(TA), are to ISDN what modems are to regular phone lines. modem Most TA's use the standard hayes modem AT command set, and can be used as a drop in replacement for a modem. A TA will operate basically the same as a modem except connection and throughput speeds will be much faster than your old modem. You will need to configure PPP exactly the same as for a modem setup. Make sure you set your serial speed as high as possible. PPP The main advantage of using a TA to connect to an Internet Provider is that you can do Dynamic PPP. As IP address space becomes more and more scarce, most providers are not willing to provide you with a static IP anymore. Most stand-alone routers are not able to accommodate dynamic IP allocation. TA's completely rely on the PPP daemon that you are running for their features and stability of connection. This allows you to upgrade easily from using a modem to ISDN on a FreeBSD machine, if you already have PPP setup. However, at the same time any problems you experienced with the PPP program and are going to persist. If you want maximum stability, use the kernel PPP option, not the user-land iijPPP. The following TA's are know to work with FreeBSD. Motorola BitSurfer and Bitsurfer Pro Adtran Most other TA's will probably work as well, TA vendors try to make sure their product can accept most of the standard modem AT command set. The real problem with external TA's is like modems you need a good serial card in your computer. You should read the serial ports section in the handbook for a detailed understanding of serial devices, and the differences between asynchronous and synchronous serial ports. A TA running off a standard PC serial port (asynchronous) limits you to 115.2Kbs, even though you have a 128Kbs connection. To fully utilize the 128Kbs that ISDN is capable of, you must move the TA to a synchronous serial card. Do not be fooled into buying an internal TA and thinking you have avoided the synchronous/asynchronous issue. Internal TA's simply have a standard PC serial port chip built into them. All this will do, is save you having to buy another serial cable, and find another empty electrical socket. A synchronous card with a TA is at least as fast as a stand-alone router, and with a simple 386 FreeBSD box driving it, probably more flexible. The choice of sync/TA v.s. stand-alone router is largely a religious issue. There has been some discussion of this in the mailing lists. I suggest you search the archives for the complete discussion. Stand-alone ISDN Bridges/Routers ISDN stand-alone bridges/routers ISDN bridges or routers are not at all specific to FreeBSD or any other operating system. For a more complete description of routing and bridging technology, please refer to a Networking reference book. In the context of this page, the terms router and bridge will be used interchangeably. As the cost of low end ISDN routers/bridges comes down, it will likely become a more and more popular choice. An ISDN router is a small box that plugs directly into your local Ethernet network(or card), and manages its own connection to the other bridge/router. It has all the software to do PPP and other protocols built in. A router will allow you much faster throughput that a standard TA, since it will be using a full synchronous ISDN connection. The main problem with ISDN routers and bridges is that interoperability between manufacturers can still be a problem. If you are planning to connect to an Internet provider, you should discuss your needs with them. If you are planning to connect two LAN segments together, ie: home LAN to the office LAN, this is the simplest lowest maintenance solution. Since you are buying the equipment for both sides of the connection you can be assured that the link will work. For example to connect a home computer or branch office network to a head office network the following setup could be used. Branch office or Home network 10 base 2 Network uses a bus based topology with 10 base 2 Ethernet ("thinnet"). Connect router to network cable with AUI/10BT transceiver, if necessary. ---Sun workstation | ---FreeBSD box | ---Windows 95 (Do not admit to owning it) | Stand-alone router | ISDN BRI line 10 Base 2 Ethernet If your home/branch office is only one computer you can use a twisted pair crossover cable to connect to the stand-alone router directly. Head office or other LAN 10 base T Network uses a star topology with 10 base T Ethernet ("Twisted Pair"). -------Novell Server | H | | ---Sun | | | U ---FreeBSD | | | ---Windows 95 | B | |___---Stand-alone router | ISDN BRI line ISDN Network Diagram One large advantage of most routers/bridges is that they allow you to have 2 separate independent PPP connections to 2 separate sites at the same time. This is not supported on most TA's, except for specific(expensive) models that have two serial ports. Do not confuse this with channel bonding, MPP etc. This can be very useful feature, for example if you have an dedicated ISDN connection at your office and would like to tap into it, but don't want to get another ISDN line at work. A router at the office location can manage a dedicated B channel connection (64Kbs) to the internet, as well as a use the other B channel for a separate data connection. The second B channel can be used for dial-in, dial-out or dynamically bond(MPP etc.) with the first B channel for more bandwidth. IPX/SPX An Ethernet bridge will also allow you to transmit more than just IP traffic, you can also send IPX/SPX or whatever other protocols you use. NIS/YP Written by &a.unfurl;, 21 January 2000, enhanced with parts and comments from Eric Ogren eogren@earthlink.net and Udo Erdelhoff ue@nathan.ruhr.de in June 2000. What is it? NIS Solaris HP-UX AIX Linux NetBSD OpenBSD NIS, which stands for Network Information Services, was developed by Sun Microsystems to centralize administration of Unix (originally SunOS) systems. It has now essentially become an industry standard; all major Unices (Solaris, HP-UX, AIX, Linux, NetBSD, OpenBSD, FreeBSD, etc) support NIS. yellow pages (see NIS) NIS was formerly known as Yellow Pages (or yp), but due to copyright violations, Sun was forced to change the name. NIS domains It is a RPC-based client/server system that allows a group of machines within an NIS domain to share a common set of configuration files. This permits a system administrator to set up NIS client systems with only minimal configuration data and add, remove or modify configuration data from a single location. Windows NT It is similar to Windows NT's domain system; although the internal implementation of the two aren't at all similar, the basic functionality can be compared. Terms/processes you should know There are several terms and several important user processes that you will come across when attempting to implement NIS on FreeBSD, whether you are trying to create an NIS server or act an NIS client: The NIS domainname. An NIS master server and all of its clients (including its slave servers) have a NIS domainname. Similar to an NT domain name, the NIS domainname does not have anything to do with DNS. portmap portmap. portmap must be running in order to enable RPC (Remote Procedure Call, a network protocol used by NIS). If portmap is not running, it will be impossible to run an NIS server, or to act as an NIS client. ypbind. ypbind “binds” an NIS client to its NIS server. It will take the NIS domainname from the system, and using RPC, connect to the server. ypbind is the core of client-server communication in an NIS environment; if ypbind dies on a client machine, it will not be able to access the NIS server. ypserv. ypserv, which should only be running on NIS servers, is the NIS server process itself. If ypserv dies, then the server will no longer be able to respond to NIS requests (hopefully, there is a slave server to take over for it). There are some implementations of NIS (but not the FreeBSD one), that don't try to reconnect to another server if the server it used before dies. Often, the only thing that helps in this case is to restart the server process (or even the whole server) or the ypbind process on the client. rpc.yppasswdd. rpc.yppasswdd, another process that should only be running on NIS master servers, is a daemon that will allow NIS clients to change their NIS passwords. If this daemon is not running, users will have to login to the NIS master server and change their passwords there. How does it work? There are three types of hosts in an NIS environment; master servers, slave servers, and clients. Servers act as a central repository for host configuration information. Master servers hold the authoritative copy of this information, while slave servers mirror this information for redundancy. Clients rely on the servers to provide this information to them. Information in many files can be shared in this manner. The master.passwd, group, and hosts files are commonly shared via NIS. Whenever a process on a client needs information that would normally be found in these files locally, it makes a query to the server it is bound to, to get this information. Machine types NIS master server A NIS master server. This server, analogous to a Windows NT primary domain controller, maintains the files used by all of the NIS clients. The passwd, group, and other various files used by the NIS clients live on the master server. It is possible for one machine to be an NIS master server for more than one NIS domain. However, this will not be covered in this introduction, which assumes a relatively small-scale NIS environment. NIS slave server NIS slave servers. Similar to NT's backup domain controllers, NIS slave servers maintain copies of the NIS master's data files. NIS slave servers provide the redundancy, which is needed in important environments. They also help to balance the load of the master server: NIS Clients always attach to the NIS server whose response they get first, and this includes slave-server-replies. NIS client NIS clients. NIS clients, like most NT workstations, authenticate against the NIS server (or the NT domain controller in the NT Workstation case) to log on. Using NIS/YP This section will deal with setting up a sample NIS environment. This section assumes that you are running FreeBSD 3.3 or later. The instructions given here will probably work for any version of FreeBSD greater than 3.0, but there are no guarantees that this is true. Planning Let's assume that you are the administrator of a small university lab. This lab, which consists of 15 FreeBSD machines, currently has no centralized point of administration; each machine has its own /etc/passwd and /etc/master.passwd. These files are kept in sync with each other only through manual intervention; currently, when you add a user to the lab, you must run adduser on all 15 machines. Clearly, this has to change, so you have decided to convert the lab to use NIS, using two of the machines as servers. Therefore, the configuration of the lab now looks something like: Machine name IP address Machine role ellington 10.0.0.2 NIS master coltrane 10.0.0.3 NIS slave basie 10.0.0.4 Faculty workstation bird 10.0.0.5 Client machine cli[1-11] 10.0.0.[6-17] Other client machines If you are setting up a NIS scheme for the first time, it is a good idea to think through how you want to go about it. No matter what the size of your network, there are a few decisions that need to be made. Choosing a NIS Domain Name NIS domainname This might not be the domainname that you are used to. It is more accurately called the NIS domainname. When a client broadcasts its requests for info, it includes the name of the NIS domain that it is part of. This is how multiple servers on one network can tell which server should answer which request. Think of the NIS domainname as the name for a group of hosts that are related in some way. Some organizations choose to use their Internet domainname for their NIS domainname. This is not recommended as it can cause confusion when trying to debug network problems. The NIS domainname should be unique within your network and it is helpful if it describes the group of machines it represents. For example, the Art department at Acme Inc. might be in the "acme-art" NIS domain. For this example, assume you have chosen the name test-domain. SunOS However, some operating systems (notably SunOS) use their NIS domain name as their Internet domain name. If one or more machines on your network have this restriction, you must use the Internet domain name as your NIS domain name. Physical Server Requirements There are several things to keep in mind when choosing a machine to use as a NIS server. One of the unfortunate things about NIS is the level of dependency the clients have on the server. If a client cannot contact the server for its NIS domain, very often the machine becomes unusable. The lack of user and group information causes most systems to temporarily freeze up. With this in mind you should make sure to choose a machine that won't be prone to being rebooted regularly, or one that might be used for development. The NIS server should ideally be a stand alone machine whose sole purpose in life is to be an NIS server. If you have a network that is not very heavily used, it is acceptable to put the NIS server on a machine running other services, just keep in mind that if the NIS server becomes unavailable, it will affect all of your NIS clients adversely. NIS Servers The canonical copies of all NIS information are stored on a single machine called the NIS master server. The databases used to store the information are called NIS maps. In FreeBSD, these maps are stored in /var/yp/[domainname] where [domainname] is the name of the NIS domain being served. A single NIS server can support several domains at once, therefore it is possible to have several such directories, one for each supported domain. Each domain will have its own independent set of maps. NIS master and slave servers handle all NIS requests with the ypserv daemon. Ypserv is responsible for receiving incoming requests from NIS clients, translating the requested domain and map name to a path to the corresponding database file and transmitting data from the database back to the client. Setting up a NIS master server NIS server configuration Setting up a master NIS server can be relatively straight forward, depending on your needs. FreeBSD comes with support for NIS out-of-the-box. All you need is to add the following lines to /etc/rc.conf, and FreeBSD will do the rest for you. nisdomainname="test-domain" This line will set the NIS domainname to test-domain upon network setup (e.g. after reboot). nis_server_enable="YES" This will tell FreeBSD to start up the NIS server processes when the networking is next brought up. nis_yppasswdd_enable="YES" This will enable the rpc.yppasswdd daemon, which, as mentioned above, will allow users to change their NIS password from a client machine. Now, all you have to do is to run the command /etc/netstart as superuser. It will setup everything for you, using the values you defined in /etc/rc.conf. Initializing the NIS maps NIS maps The NIS maps are database files, that are kept in the /var/yp directory. They are generated from configuration files in the /etc directory of the NIS master, with one exception: the /etc/master.passwd file. This is for a good reason; you don't want to propagate passwords to your root and other administrative accounts to all the servers in the NIS domain. Therefore, before we initialize the NIS maps, you should: - -&prompt.root; cp /etc/master.passwd /var/yp/master.passwd + &prompt.root; cp /etc/master.passwd /var/yp/master.passwd &prompt.root; cd /var/yp -&prompt.root; vi master.passwd - +&prompt.root; vi master.passwd You should remove all entries regarding system accounts (bin, tty, kmem, games, etc), as well as any accounts that you don't want to be propagated to the NIS clients (for example root and any other UID 0 (superuser) accounts). Make sure the /var/yp/master.passwd is neither group nor world readable (mode 600)! Use the chmod command, if appropriate. Tru64 Unix When you have finished, it's time to initialize the NIS maps! FreeBSD includes a script named ypinit to do this for you (see its man page for more information). Note that this script is available on most UNIX OSs, but not on all. On Digital Unix/Compaq Tru64 Unix it is called ypsetup. Because we are generating maps for an NIS master, we are going to pass the option to ypinit. To generate the NIS maps, assuming you already performed the steps above, run: - -ellington&prompt.root; ypinit -m test-domain + ellington&prompt.root; ypinit -m test-domain Server Type: MASTER Domain: test-domain Creating an YP server will require that you answer a few questions. Questions will all be asked at the beginning of the procedure. Do you want this procedure to quit on non-fatal errors? [y/n: n] n Ok, please remember to go back and redo manually whatever fails. If you don't, something might not work. At this point, we have to construct a list of this domains YP servers. rod.darktech.org is already known as master server. Please continue to add any slave servers, one per line. When you are done with the list, type a <control D>. master server : ellington next host to add: coltrane next host to add: ^D The current list of NIS servers looks like this: ellington coltrane Is this correct? [y/n: y] y [..output from map generation..] NIS Map update completed. -ellington has been setup as an YP master server without any errors. - +ellington has been setup as an YP master server without any errors. ypinit should have created /var/yp/Makefile from /var/yp/Makefile.dist. When created, this file assumes that you are operating in a single server NIS environment with only FreeBSD machines. Since test-domain has a slave server as well, you must edit /var/yp/Makefile: - -ellington&prompt.root; vi /var/yp/Makefile - + ellington&prompt.root; vi /var/yp/Makefile You should comment out the line that says `NOPUSH = "True"' (if it is not commented out already). Setting up a NIS slave server NIS configuring a slave server Setting up an NIS slave server is even more simple than setting up the master. Log on to the slave server and edit the file /etc/rc.conf as you did before. The only difference is that we now must use the option when running ypinit. The option requires the name of the NIS master be passed to it as well, so our command line looks like: - -coltrane&prompt.root; ypinit -s ellington test-domain + coltrane&prompt.root; ypinit -s ellington test-domain Server Type: SLAVE Domain: test-domain Master: ellington Creating an YP server will require that you answer a few questions. Questions will all be asked at the beginning of the procedure. Do you want this procedure to quit on non-fatal errors? [y/n: n] n Ok, please remember to go back and redo manually whatever fails. If you don't, something might not work. There will be no further questions. The remainder of the procedure should take a few minutes, to copy the databases from ellington. Transferring netgroup... ypxfr: Exiting: Map successfully transferred Transferring netgroup.byuser... ypxfr: Exiting: Map successfully transferred Transferring netgroup.byhost... ypxfr: Exiting: Map successfully transferred Transferring master.passwd.byuid... ypxfr: Exiting: Map successfully transferred Transferring passwd.byuid... ypxfr: Exiting: Map successfully transferred Transferring passwd.byname... ypxfr: Exiting: Map successfully transferred Transferring group.bygid... ypxfr: Exiting: Map successfully transferred Transferring group.byname... ypxfr: Exiting: Map successfully transferred Transferring services.byname... ypxfr: Exiting: Map successfully transferred Transferring rpc.bynumber... ypxfr: Exiting: Map successfully transferred Transferring rpc.byname... ypxfr: Exiting: Map successfully transferred Transferring protocols.byname... ypxfr: Exiting: Map successfully transferred Transferring master.passwd.byname... ypxfr: Exiting: Map successfully transferred Transferring networks.byname... ypxfr: Exiting: Map successfully transferred Transferring networks.byaddr... ypxfr: Exiting: Map successfully transferred Transferring netid.byname... ypxfr: Exiting: Map successfully transferred Transferring hosts.byaddr... ypxfr: Exiting: Map successfully transferred Transferring protocols.bynumber... ypxfr: Exiting: Map successfully transferred Transferring ypservers... ypxfr: Exiting: Map successfully transferred Transferring hosts.byname... ypxfr: Exiting: Map successfully transferred coltrane has been setup as an YP slave server without any errors. Don't forget to update map ypservers on ellington. You should now have a directory called /var/yp/test-domain. Copies of the NIS master server's maps should be in this directory. You will need to make sure that these stay updated. The following /etc/crontab entries on your slave servers should do the job: 20 * * * * root /usr/libexec/ypxfr passwd.byname 21 * * * * root /usr/libexec/ypxfr passwd.byuid These two lines force the slave to sync its maps with the maps on the master server. Although this is not mandatory, because the master server tries to make sure any changes to its NIS maps are communicated to its slaves, the password information is so vital to systems that depend on the server, that it is a good idea to force the updates. This is more important on busy networks where map updates might not always complete. Now, run the command /etc/netstart on the slave server as well, which again starts the NIS server. NIS Clients An NIS client establishes what is called a binding to a particular NIS server using the ypbind daemon. ypbind checks the system's default domain (as set by the domainname command), and begins broadcasting RPC requests on the local network. These requests specify the name of the domain for which ypbind is attempting to establish a binding. If a server that has been configured to serve the requested domain receives one of the broadcasts, it will respond to ypbind, which will record the server's address. If there are several servers available (a master and several slaves, for example), ypbind will use the address of the first one to respond. From that point on, the client system will direct all of its NIS requests to that server. Ypbind will occasionally ping the server to make sure it is still up and running. If it fails to receive a reply to one of its pings within a reasonable amount of time, ypbind will mark the domain as unbound and begin broadcasting again in the hopes of locating another server. Setting up an NIS client NIS client configuration Setting up a FreeBSD machine to be a NIS client is fairly straightforward. Edit the file /etc/rc.conf and add the following lines in order to set the NIS domainname and start ypbind upon network startup: nisdomainname="test-domain" nis_client_enable="YES" To import all possible password entries from the NIS server, add this line to your /etc/master.passwd file, using vipw: +::::::::: This line will afford anyone with a valid account in the NIS server's password maps an account. There are many ways to configure your NIS client by changing this line. See the netgroups part below for more information. For more detailed reading see O'Reilly's book on Managing NFS and NIS. To import all possible group entries from the NIS server, add this line to your /etc/group file: +:*:: After completing these steps, you should be able to run ypcat passwd and see the NIS server's passwd map. NIS Security In general, any remote user can issue an RPC to ypserv and retrieve the contents of your NIS maps, provided the remote user knows your domainname. To prevent such unauthorized transactions, ypserv supports a feature called securenets which can be used to restrict access to a given set of hosts. At startup, ypserv will attempt to load the securenets information from a file called /var/yp/securenets. This path varies depending on the path specified with the option. This file contains entries that consist of a network specification and a network mask separated by white space. Lines starting with # are considered to be comments. A sample securenets file might look like this: # allow connections from local host -- mandatory 127.0.0.1 255.255.255.255 # allow connections from any host # on the 192.168.128.0 network 192.168.128.0 255.255.255.0 # allow connections from any host # between 10.0.0.0 to 10.0.15.255 # this includes the machines in the testlab 10.0.0.0 255.255.240.0 If ypserv receives a request from an address that matches one of these rules, it will process the request normally. If the address fails to match a rule, the request will be ignored and a warning message will be logged. If the /var/yp/securenets file does not exist, ypserv will allow connections from any host. The ypserv program also has support for Wietse Venema's tcpwrapper package. This allows the administrator to use the tcpwrapper configuration files for access control instead of /var/yp/securenets. While both of these access control mechanisms provide some security, they, like the privileged port test, are vulnerable to IP spoofing attacks. All NIS-related traffic should be blocked at your firewall. Servers using /var/yp/securenets may fail to serve legitimate NIS clients with archaic TCP/IP implementations. Some of these implementations set all host bits to zero when doing broadcasts and/or fail to observe the subnet mask when calculating the broadcast address. While some of these problems can be fixed by changing the client configuration, other problems may force the retirement of the client systems in question or the abandonment of /var/yp/securenets. Using /var/yp/securenets on a server with such an archaic implementation of TCP/IP is a really bad idea and will lead to loss of NIS functionality for large parts of your network. tcpwrapper The use of the tcpwrapper package increases the latency of your NIS server. The additional delay may be long enough to cause timeouts in client programs, especially in busy networks or with slow NIS servers. If one or more of your client systems suffers from these symptoms, you should convert the client systems in question into NIS slave servers and force them to bind to themselves. Barring some users from logging on In our lab, there is a machine basie that is supposed to be a faculty only workstation. We don't want to take this machine out of the NIS domain, yet the passwd file on the master NIS server contains accounts for both faculty and students. What can we do? There is a way to bar specific users from logging on to a machine, even if they are present in the NIS database. To do this, all you must do is add -username to the end of the /etc/master.passwd file on the client machine, where username is the username of the user you wish to bar from logging in. This should preferably be done using vipw, since vipw will sanity check your changes to /etc/master.passwd, as well as automatically rebuild the password database when you finish editing. For example, if we wanted to bar user bill from logging on to basie we would: - -basie&prompt.root; vipw + basie&prompt.root; vipw [add -bill to the end, exit] vipw: rebuilding the database... vipw: done basie&prompt.root; cat /etc/master.passwd root:[password]:0:0::0:0:The super-user:/root:/bin/csh toor:[password]:0:0::0:0:The other super-user:/root:/bin/sh daemon:*:1:1::0:0:Owner of many system processes:/root:/sbin/nologin operator:*:2:5::0:0:System &:/:/sbin/nologin bin:*:3:7::0:0:Binaries Commands and Source,,,:/:/sbin/nologin tty:*:4:65533::0:0:Tty Sandbox:/:/sbin/nologin kmem:*:5:65533::0:0:KMem Sandbox:/:/sbin/nologin games:*:7:13::0:0:Games pseudo-user:/usr/games:/sbin/nologin news:*:8:8::0:0:News Subsystem:/:/sbin/nologin man:*:9:9::0:0:Mister Man Pages:/usr/share/man:/sbin/nologin bind:*:53:53::0:0:Bind Sandbox:/:/sbin/nologin uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/libexec/uucp/uucico xten:*:67:67::0:0:X-10 daemon:/usr/local/xten:/sbin/nologin pop:*:68:6::0:0:Post Office Owner:/nonexistent:/sbin/nologin nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/sbin/nologin +::::::::: -bill basie&prompt.root; Using netgroups netgroups The netgroups part was contributed by Udo Erdelhoff ue@nathan.ruhr.de in July 2000. The method shown in the previous chapter works reasonably well if you need special rules for a very small number of users and/or machines. On larger networks, you will forget to bar some users from logging onto sensitive machines, or you may even have to modify each machine separately, thus losing the main benefit of NIS, centralized administration. The NIS developers' solution for this problem is called netgroups. Their purpose and semantics can be compared to the normal groups used by Unix file systems. The main differences are the lack of a numeric id and the ability to define a netgroup by including both user accounts and other netgroups. Netgroups were developed to handle large, complex networks with hundreds of users and machines. On one hand, this is a Good Thing if you are forced to deal with such a situation. On the other hand, this complexity makes it almost impossible to explain netgroups with really simple examples. The example used in the remainder of this chapter demonstrates this problem. Let us assume that your successful introduction of NIS in your laboratory caught your superiors' interest. Your next job is to extend your NIS domain to cover some of the other machines on campus. The two tables contain the names of the new users and new machines as well as brief descriptions of them. User Name(s) Description alpha, beta Normal employees of the IT department charlie, delta The new apprentices of the IT department echo, foxtrott, golf, ... Ordinary employees able, baker, ... The current interns Machine Name(s) Description war, death, famine, pollution Your most important servers. Only the IT employees are allowed to log onto these machines. pride, greed, envy, wrath, lust, sloth Less important servers. All members of the IT department are allowed to login onto these machines. one, two, three, four, ... Ordinary workstations. Only the real employees are allowed to use these machines. trashcan A very old machine without any critical data. Even the intern is allowed to use this box. If you tried to implement these restrictions by separately blocking each user, you would have to add one -user line to each system's passwd for each user who is not allowed to login onto that system. If you forget just one entry, you could be in trouble. It may be feasible to do this correctly during the initial setup, however you will eventually forget to add the lines for new users during day-to-day operations. After all, Murphy was an optimist. Handling this situation with netgroups offers several advantages. Each user need not be handled separately; you assign a user to one or more netgroups and allow or forbid logins for all members of the netgroup. If you add a new machine, you will only have to define login restrictions for netgroups. If a new user is added, you will only have to add the user to one or more netgroups. Those changes are independent of each other; no more for each combination of user and machine do... If your NIS setup is planned carefully, you will only have to modify exactly one central configuration file to grant or deny access to machines. The first step is the initialization of the NIS map netgroup. FreeBSD's ypinit does not create this map by default, but its NIS implementation will support it once it has been created. To create an empty map, simply type - -ellington&prompt.root; vi /var/yp/netgroup - + ellington&prompt.root; vi /var/yp/netgroup and start adding content. For our example, we need at least four netgroups: IT employees, IT apprentices, normal employees and interns. IT_EMP (,alpha,test-domain) (,beta,test-domain) IT_APP (,charlie,test-domain) (,delta,test-domain) USERS (,echo,test-domain) (,foxtrott,test-domain) \ (,golf,test-domain) INTERNS (,able,test-domain) (,baker,test-domain) IT_EMP, IT_APP etc. are the names of the netgroups. Each bracketed group adds one or more user accounts to it. The three fields inside a group are: The name of the host(s) where the following items are valid. If you do not specify a hostname, the entry is valid on all hosts. If you do specify a hostname, you will enter a realm of darkness, horror and utter confusion. The name of the account that belongs to this netgroup. The NIS domain for the account. You can import accounts from other NIS domains into your netgroup if you are one of unlucky fellows with more than one NIS domain. Each of these fields can contain wildcards. See &man.netgroup.5; for details. netgroups Netgroup names longer than 8 characters should not be used, especially if you have machines running other operating systems within your NIS domain. The names are case sensitive; using capital letters for your netgroup names is an easy way to distinguish between user, machine and netgroup names. Some NIS clients (other than FreeBSD) cannot handle netgroups with a large number of entries. For example, some older versions of SunOS start to cause trouble if a netgroup contains more than 15 entries. You can circumvent this limit by creating several sub-netgroups with 15 users or less and a real netgroup that consists of the sub-netgroups: BIGGRP1 (,joe1,domain) (,joe2,domain) (,joe3,domain) [...] BIGGRP2 (,joe16,domain) (,joe17,domain) [...] BIGGRP3 (,joe31,domain) (,joe32,domain) BIGGROUP BIGGRP1 BIGGRP2 BIGGRP3 You can repeat this process if you need more than 225 users within a single netgroup. Activating and distributing your new NIS map is easy: - -ellington&prompt.root; cd /var/yp -ellington&prompt.root; make - + ellington&prompt.root; cd /var/yp +ellington&prompt.root; make This will generate the three NIS maps netgroup, netgroup.byhost and netgroup.byuser. Use &man.ypcat.1; to check if your new NIS maps are available: - -ellington&prompt.user; ypcat -k netgroup + ellington&prompt.user; ypcat -k netgroup ellington&prompt.user; ypcat -k netgroup.byhost -ellington&prompt.user; ypcat -k netgroup.byuser - +ellington&prompt.user; ypcat -k netgroup.byuser The output of the first command should resemble the contents of /var/yp/netgroup. The second command will not produce output if you have not specified host-specific netgroups. The third command can be used to get the list of netgroups for a user. The client setup is quite simple. To configure the server war, you only have to start &man.vipw.8; and replace the line +::::::::: with +@IT_EMP::::::::: Now, only the data for the users defined in the netgroup IT_EMP is imported into war's password database and only these users are allowed to login. Unfortunately, this limitation also applies to the ~ function of the shell and all routines converting between user names and numerical user ids. In other words, cd ~user will not work, ls -l will show the numerical id instead of the username and find . -user joe -print will fail with No such user. To fix this, you will have to import all user entries without allowing them to login onto your servers. This can be achieved by adding another line to /etc/master.passwd. This line should contain +:::::::::/sbin/nologin, meaning Import all entries but replace the shell with /sbin/nologin in the imported entries. You can replace any field in the passwd entry by placing a default value in your /etc/master.passwd. Make sure that the line +:::::::::/sbin/nologin is placed after +@IT_EMP:::::::::. Otherwise, all user accounts imported from NIS will have /sbin/nologin as their login shell. After this change, you will only have to change one NIS map if a new employee joins the IT department. You could use a similar approach for the less important servers by replacing the old +::::::::: in their local version of /etc/master.passwd with something like this: +@IT_EMP::::::::: +@IT_APP::::::::: +:::::::::/sbin/nologin The corresponding lines for the normal workstations could be: +@IT_EMP::::::::: +@USERS::::::::: +:::::::::/sbin/nologin And everything would be fine until there is a policy change a few weeks later: The IT department starts hiring interns. The IT interns are allowed to use the normal workstations and the less important servers; and the IT apprentices are allowed to login onto the main servers. You add a new netgroup IT_INTERN, add the new IT interns to this netgroup and start to change the config on each and every machine... As the old saying goes: Errors in centralized planning lead to global mess. NIS' ability to create netgroups from other netgroups can be used to prevent situations like these. One possibility is the creation of role-based netgroups. For example, you could create a netgroup called BIGSRV to define the login restrictions for the important servers, another netgroup called SMALLSRV for the less important servers and a third netgroup called USERBOX for the normal workstations. Each of these netgroups contains the netgroups that are allowed to login onto these machines. The new entries for your NIS map netgroup should look like this: BIGSRV IT_EMP IT_APP SMALLSRV IT_EMP IT_APP ITINTERN USERBOX IT_EMP ITINTERN USERS This method of defining login restrictions works reasonably well if you can define groups of machines with identical restrictions. Unfortunately, this is the exception and not the rule. Most of the time, you will need the ability to define login restrictions on a per-machine basis. Machine-specific netgroup definitions are the other possibility to deal with the policy change outlined above. In this scenario, the /etc/master.passwd of each box contains two lines starting with ``+''. The first of them adds a netgroup with the accounts allowed to login onto this machine, the second one adds all other accounts with /sbin/nologin as shell. It is a good idea to use the ALL-CAPS version of the machine name as the name of the netgroup. In other words, the lines should look like this: +@BOXNAME::::::::: +:::::::::/sbin/nologin Once you have completed this task for all your machines, you will not have to modify the local versions of /etc/master.passwd ever again. All further changes can be handled by modifying the NIS map. Here is an example of a possible netgroup map for this scenario with some additional goodies. # Define groups of users first IT_EMP (,alpha,test-domain) (,beta,test-domain) IT_APP (,charlie,test-domain) (,delta,test-domain) DEPT1 (,echo,test-domain) (,foxtrott,test-domain) DEPT2 (,golf,test-domain) (,hotel,test-domain) DEPT3 (,india,test-domain) (,juliet,test-domain) ITINTERN (,kilo,test-domain) (,lima,test-domain) D_INTERNS (,able,test-domain) (,baker,test-domain) # # Now, define some groups based on roles USERS DEPT1 DEPT2 DEPT3 BIGSRV IT_EMP IT_APP SMALLSRV IT_EMP IT_APP ITINTERN USERBOX IT_EMP ITINTERN USERS # # And a groups for a special tasks # Allow echo and golf to access our anti-virus-machine SECURITY IT_EMP (,echo,test-domain) (,golf,test-domain) # # machine-based netgroups # Our main servers WAR BIGSRV FAMINE BIGSRV # User india needs access to this server POLLUTION BIGSRV (,india,test-domain) # # This one is really important and needs more access restrictions DEATH IT_EMP # # The anti-virus-machine mentioned above ONE SECURITY # # Restrict a machine to a single user TWO (,hotel,test-domain) # [...more groups to follow] If you are using some kind of database to manage your user accounts, you should be able to create the first part of the map with your database's report tools. This way, new users will automatically have access to the boxes. One last word of caution: It may not always be advisable to use machine-based netgroups. If you are deploying a couple dozen or even hundreds of identical machines for student labs, you should use role-based netgroups instead of machine-based netgroups to keep the size of the NIS map within reasonable limits. Important things to remember There are still a couple of things that you will need to do differently now that you are in an NIS environment. Every time you wish to add a user to the lab, you must add it to the master NIS server only, and you must remember to rebuild the NIS maps. If you forget to do this, the new user will not be able to login anywhere except on the NIS master. For example, if we needed to add a new user “jsmith” to the lab, we would: - -&prompt.root; pw useradd jsmith + &prompt.root; pw useradd jsmith &prompt.root; cd /var/yp &prompt.root; make test-domain You could also run adduser jsmith instead of pw useradd jsmith. Keep the administration accounts out of the NIS maps. You don't want to be propagating administrative accounts and passwords to machines that will have users that shouldn't have access to those accounts. Keep the NIS master and slave secure, and minimize their downtime. If somebody either hacks or simply turns off these machines, they have effectively rendered many people without the ability to login to the lab. This is the chief weakness of any centralized administration system, and it is probably the most important weakness. If you do not protect your NIS servers, you will have a lot of angry users! NIS v1 compatibility FreeBSD's ypserv has some support for serving NIS v1 clients. FreeBSD's NIS implementation only uses the NIS v2 protocol, however other implementations include support for the v1 protocol for backwards compatibility with older systems. The ypbind daemons supplied with these systems will try to establish a binding to an NIS v1 server even though they may never actually need it (and they may persist in broadcasting in search of one even after they receive a response from a v2 server). Note that while support for normal client calls is provided, this version of ypserv does not handle v1 map transfer requests; consequently, it can not be used as a master or slave in conjunction with older NIS servers that only support the v1 protocol. Fortunately, there probably are not any such servers still in use today. NIS servers that are also NIS clients Care must be taken when running ypserv in a multi-server domain where the server machines are also NIS clients. It is generally a good idea to force the servers to bind to themselves rather than allowing them to broadcast bind requests and possibly become bound to each other. Strange failure modes can result if one server goes down and others are dependent upon on it. Eventually all the clients will time out and attempt to bind to other servers, but the delay involved can be considerable and the failure mode is still present since the servers might bind to each other all over again. You can force a host to bind to a particular server by running ypbind with the flag. libscrypt v.s. libdescrypt NIS crypto library One of the most common issues that people run into when trying to implement NIS is crypt library compatibility. If your NIS server is using the DES crypt libraries, it will only support clients that are using DES as well. To check which one your server and clients are using look at the symlinks in /usr/lib. If the machine is configured to use the DES libraries, it will look something like this: - -&prompt.user; ls -l /usr/lib/*crypt* + &prompt.user; ls -l /usr/lib/*crypt* lrwxrwxrwx 1 root wheel 13 Jul 15 08:55 /usr/lib/libcrypt.a@ -> libdescrypt.a lrwxrwxrwx 1 root wheel 14 Jul 15 08:55 /usr/lib/libcrypt.so@ -> libdescrypt.so lrwxrwxrwx 1 root wheel 16 Jul 15 08:55 /usr/lib/libcrypt.so.2@ -> libdescrypt.so.2 lrwxrwxrwx 1 root wheel 15 Jul 15 08:55 /usr/lib/libcrypt_p.a@ -> libdescrypt_p.a -r--r--r-- 1 root wheel 13018 Nov 8 14:27 /usr/lib/libdescrypt.a lrwxr-xr-x 1 root wheel 16 Nov 8 14:27 /usr/lib/libdescrypt.so@ -> libdescrypt.so.2 -r--r--r-- 1 root wheel 12965 Nov 8 14:27 /usr/lib/libdescrypt.so.2 -r--r--r-- 1 root wheel 14750 Nov 8 14:27 /usr/lib/libdescrypt_p.a If the machine is configured to use the standard FreeBSD MD5 crypt libraries they will look something like this: - -&prompt.user; ls -l /usr/lib/*crypt* + &prompt.user; ls -l /usr/lib/*crypt* lrwxrwxrwx 1 root wheel 13 Jul 15 08:55 /usr/lib/libcrypt.a@ -> libscrypt.a lrwxrwxrwx 1 root wheel 14 Jul 15 08:55 /usr/lib/libcrypt.so@ -> libscrypt.so lrwxrwxrwx 1 root wheel 16 Jul 15 08:55 /usr/lib/libcrypt.so.2@ -> libscrypt.so.2 lrwxrwxrwx 1 root wheel 15 Jul 15 08:55 /usr/lib/libcrypt_p.a@ -> libscrypt_p.a -r--r--r-- 1 root wheel 6194 Nov 8 14:27 /usr/lib/libscrypt.a lrwxr-xr-x 1 root wheel 14 Nov 8 14:27 /usr/lib/libscrypt.so@ -> libscrypt.so.2 -r--r--r-- 1 root wheel 7579 Nov 8 14:27 /usr/lib/libscrypt.so.2 -r--r--r-- 1 root wheel 6684 Nov 8 14:27 /usr/lib/libscrypt_p.a If you have trouble authenticating on an NIS client, this is a pretty good place to start looking for possible problems. If you want to deploy an NIS server for a heterogenous network, you will probably have to use DES on all systems because it is the lowest common standard. DHCP Written by &a.gsutter;, March 2000. What is DHCP? Dynamic Host Configuration Protocol (DHCP) Internet Software Consortium (ISC) DHCP, the Dynamic Host Configuration Protocol, describes the means by which a system can connect to a network and obtain the necessary information for communication upon that network. FreeBSD uses the ISC (Internet Software Consortium) DHCP implementation, so all implementation-specific information here is for use with the ISC distribution. What This Section Covers This handbook section attempts to describe only the parts of the DHCP system that are integrated with FreeBSD; consequently, the server portions are not described. The DHCP manual pages, in addition to the references below, are useful resources. How it Works UDP When dhclient, the DHCP client, is executed on the client machine, it begins broadcasting requests for configuration information. By default, these requests are on UDP port 68. The server replies on UDP 67, giving the client an IP address and other relevant network information such as netmask, router, and DNS servers. All of this information comes in the form of a DHCP "lease" and is only valid for a certain time (configured by the DHCP server maintainer). In this manner, stale IP addresses for clients no longer connected to the network can be automatically reclaimed. DHCP clients can obtain a great deal of information from the server. An exhaustive list may be found in &man.dhcp-options.5;. FreeBSD Integration FreeBSD fully integrates the ISC DHCP client, dhclient. DHCP client support is provided within both the installer and the base system, obviating the need for detailed knowledge of network configurations on any network that runs a DHCP server. dhclient has been included in all FreeBSD distributions since 3.2. sysinstall DHCP is supported by sysinstall. When configuring a network interface within sysinstall, the first question asked is, "Do you want to try dhcp configuration of this interface?" Answering affirmatively will execute dhclient, and if successful, will fill in the network configuration information automatically. There are two things you must do to have your system use DHCP upon startup: DHCP requirements Make sure that the bpf device is compiled into your kernel. To do this, add pseudo-device bpf to your kernel configuration file, and rebuild the kernel. For more information about building kernels, see . The bpf device is already part of the GENERIC kernel that is supplied with FreeBSD, so if you don't have a custom kernel, you shouldn't need to create one in order to get DHCP working. For those who are particularly security conscious, you should be warned that bpf is also the device that allows packet sniffers to work correctly (although they still have to be run as root). bpf is required to use DHCP, but if you are very sensitive about security, you probably shouldn't add bpf to your kernel in the expectation that at some point in the future you will be using DHCP. Edit your /etc/rc.conf to include the following: ifconfig_fxp0="DHCP" Be sure to replace fxp0 with the designation for the interface that you wish to dynamically configure. If you are using a different location for dhclient, or if you wish to pass additional flags to dhclient, also include the following (editing as necessary): dhcp_program="/sbin/dhclient" dhcp_flags="" DHCP server The DHCP server, dhcpd, is included as part of the isc-dhcp2 port in the ports collection. This port contains the full ISC DHCP distribution, consisting of client, server, relay agent and documentation. Files DHCP configuration files /etc/dhclient.conf dhclient requires a configuration file, /etc/dhclient.conf. Typically the file contains only comments, the defaults being reasonably sane. This configuration file is described by the &man.dhclient.conf.5; man page. /sbin/dhclient dhclient is statically linked and resides in /sbin. The &man.dhclient.8; manual page gives more information about dhclient. /sbin/dhclient-script dhclient-script is the FreeBSD-specific DHCP client configuration script. It is described in &man.dhclient-script.8;, but should not need any user modification to function properly. /var/db/dhclient.leases The DHCP client keeps a database of valid leases in this file, which is written as a log. &man.dhclient.leases.5; gives a slightly longer description. Further Reading The DHCP protocol is fully described in RFC 2131. An informational resource has also been set up at dhcp.org. DNS Contributed by &a.chern;, April 12, 2001. Overview BIND FreeBSD utilizes, by default, a version of BIND (Berkeley Internet Name Domain), which is the most common implementation of the DNS protocol. DNS is the protocol through which names are mapped to IPs, and vice versa. For example, a query for www.freebsd.org will send back a reply for the IP address of The FreeBSD Project's webpage, whereas, a query for ftp.freebsd.org will return the IP of the corresponding ftp machine. Likewise, the opposite can happen. A query for an IP address can resolve its hostname. DNS DNS is coordinated across the Internet through a somewhat complex system of authoritative root name servers, and other smaller-scale nameservers who host and relay individual domain information. This document refers to BIND 8.x, as it is the most current, stable version used in FreeBSD. RFC1034 and RFC1035 dictates the DNS protocol. Currently, BIND is maintained by the Internet Software Consortium (www.isc.org) Terminology zones zone - Each individual domain, subdomain, or 'area' dictated by DNS is considered a zone. Examples of zones: . is the root zone org. is a zone under the root zone foobardomain.org is a zone under the org. zone foo.foobardomain.org. is a subdomain, a zone under the foobardomain.org. zone 1.2.3.in-addr.arpa is a zone referencing all ips which fall under the 3.2.1.* ip space. named, bind, name server - these are all common names for the BIND name server package within FreeBSD. resolver resolver - a network process by which a system queries a nameserver for answers root zone root zone - literally, a '.', refers to the root, or beginning zone. All zones fall under this, as do all files in fall under the root directory. It is the beginning of the Internet zone hierarchy origin - refers to the point of start for the particular zone forward dns - mapping of hostnames to ip addresses reverse DNS reverse dns - the opposite, mapping of ip addresses to hostnames Reasons to run a name server You need your machine to host DNS information to the world An authoritative nameserver replies exclusively to requests. For example, you register foobardomain.org and wish to assign hostnames to the proper IP addresses. A slave nameserver, which replies to queries for a domain when the primary is down or inaccessible. The above two can also be done with in-addr.arpa, IP to hostname entries You wish your machine to act as a local relay of DNS information DNS traffic has been measured to be about 5% or more of the total Internet traffic. A local DNS server may have some added benefit by providing a local cache of DNS information. For example, when one queries for www.freebsd.org, their resolver goes out to (usually) your ISP's name server, and retrieves the query. With a local, caching DNS server, the query only has to be made once to the outside world. Every additional query will not have to go outside of the local network, since the information is cached. How it works A DNS server in FreeBSD relies on the BIND daemon. This daemon is called 'named' for obvious reasons. named - the bind daemon ndc - name daemon control program /etc/namedb - directory where all the bind information resides /etc/namedb/named.conf - daemon configuration file zone files are usually contained within the /etc/namedb directory, and contain the information (query answers from your site) served by your name server. Starting BIND BIND starting Since bind is installed by default, configuring it all is relatively simple. To ensure the named daemon is started at boot, put the following modifications in your /etc/rc.conf named_enable="YES" To start the daemon manually (after configuring it) &prompt.root; ndc start Configuration files BIND configuration files make-localhost Be sure to &prompt.root; cd /etc/namedb -&prompt.root; sh make-localhost - +&prompt.root; sh make-localhost to properly create your local reverse dns zone file in /etc/namedb/localhost.rev. <filename>/etc/namedb/named.conf</filename> - // $FreeBSD: doc/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml,v 1.49 2001/06/26 00:09:52 murray Exp $ + // $FreeBSD: doc/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml,v 1.50 2001/06/26 20:10:25 murray Exp $ // // Refer to the named(8) man page for details. If you are ever going // to setup a primary server, make sure you've understood the hairy // details of how DNS is working. Even with simple mistakes, you can // break connectivity for affected parties, or cause huge amount of // useless Internet traffic. options { directory "/etc/namedb"; // In addition to the "forwarders" clause, you can force your name // server to never initiate queries of its own, but always ask its // forwarders only, by enabling the following line: // // forward only; // If you've got a DNS server around at your upstream provider, enter // its IP address here, and enable the line below. This will make you // benefit from its cache, thus reduce overall DNS traffic in the Internet. /* forwarders { 127.0.0.1; }; */ Just as the comment says, if you want to benefit from your uplink's cache, you can enable this section of the config file. Normally, your nameserver will recursively query different nameservers until it finds the answer it is looking for. Having this enabled will have it automatically see if your uplink's (or whatever provided) ns has the requested query. If your uplink has a heavily trafficked, fast nameserver, enabling this properly could work to your advantage. 127.0.0.1 will *NOT* work here; change this to the IP of a nameserver at your uplink. /* * If there is a firewall between you and nameservers you want * to talk to, you might need to uncomment the query-source * directive below. Previous versions of BIND always asked * questions using port 53, but BIND 8.1 uses an unprivileged * port by default. */ // query-source address * port 53; /* * If running in a sandbox, you may have to specify a different * location for the dumpfile. */ // dump-file "s/named_dump.db"; }; // Note: the following will be supported in a future release. /* host { any; } { topology { 127.0.0.0/8; }; }; */ // Setting up secondaries is way easier and the rough picture for this // is explained below. // // If you enable a local name server, don't forget to enter 127.0.0.1 // into your /etc/resolv.conf so this server will be queried first. // Also, make sure to enable it in /etc/rc.conf. zone "." { type hint; file "named.root"; }; zone "0.0.127.IN-ADDR.ARPA" { type master; file "localhost.rev"; }; zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.IP6.INT" { type master; file "localhost.rev"; }; // NB: Do not use the IP addresses below, they are faked, and only // serve demonstration/documentation purposes! // // Example secondary config entries. It can be convenient to become // a secondary at least for the zone where your own domain is in. Ask // your network administrator for the IP address of the responsible // primary. // // Never forget to include the reverse lookup (IN-ADDR.ARPA) zone! // (This is the first bytes of the respective IP address, in reverse // order, with ".IN-ADDR.ARPA" appended.) // // Before starting to setup a primary zone, better make sure you fully // understand how DNS and BIND works, however. There are sometimes // unobvious pitfalls. Setting up a secondary is comparably simpler. // // NB: Don't blindly enable the examples below. :-) Use actual names // and addresses instead. // // NOTE!!! FreeBSD runs bind in a sandbox (see named_flags in rc.conf). // The directory containing the secondary zones must be write accessible // to bind. The following sequence is suggested: // // mkdir /etc/namedb/s // chown bind:bind /etc/namedb/s // chmod 750 /etc/namedb/s /* zone "domain.com" { type slave; file "s/domain.com.bak"; masters { 192.168.1.1; }; }; zone "0.168.192.in-addr.arpa" { type slave; file "s/0.168.192.in-addr.arpa.bak"; masters { 192.168.1.1; }; }; */ These are example slave entries, read below to see more. For each new domain added to your nameserver, you must add one of these entries to your named.conf The simplest zone entry, can look like zone "foobardomain.org" { type master; file "foorbardomain.org"; }; For a master entry with the zone information within foobardomain.org, or zone "foobardomain.org" { type slave; file "foobardomain.org"; }; for a slave. Note that slave zones automatically query the listed master (authoritative) name servers for the zone file. Zone files An example master 'foobardomain.org' (existing within /etc/namedb/foobardomain.org) is as follows: $TTL 3600 foobardomain.org. IN SOA ns1.foobardomain.org. admin.foobardomain.org. ( 5 ; Serial 10800 ; Refresh 3600 ; Retry 604800 ; Expire 86400 ) ; Minimum TTL ; DNS Servers @ IN NS ns1.foobardomain.org. @ IN NS ns2.foobardomain.org. ; Machine Names localhost IN A 127.0.0.1 ns1 IN A 3.2.1.2 ns2 IN A 3.2.1.3 mail IN A 3.2.1.10 @ IN A 3.2.1.30 ; Aliases www IN CNAME @ ; MX Record @ IN MX 10 mail.foobardomain.org. Note that every hostname ending in a '.' is an exact hostname, whereas everything without a trailing '.' is referenced to the origin. For example, www is translated into www + origin. In our fictitious zone file, our origin is foobardomain.org, so www would be www.foobardomain.org. The format of this file follows: recordname IN recordtype value DNS records The most commonly used DNS records: SOA - start of zone authority NS - an authoritative nameserver A - A host address CNAME - the canonical name for an alias MX - mail exchange PTR - a domain name pointer (used in reverse dns) foobardomain.org. IN SOA ns1.foobardomain.org. admin.foobardomain.org. ( 5 ; Serial 10800 ; Refresh after 3 hours 3600 ; Retry after 1 hour 604800 ; Expire after 1 week 86400 ) ; Minimum TTL of 1 day foobardomain.org. - the domain name, also the origin for this zone file. ns1.foobardomain.org. - the primary/authoritative nameserver for this zone admin.foobardomain.org. - the responsible person for this zone, e-mail address with @ replaced. (admin@foobardomain.org becomes admin.foobardomain.org) 5 - the serial number of the file. this must be incremented each time the zone file is modified. Nowadays, many admins prefer a yyyymmddrr format for the serial number. 2001041002 would mean last modified 04/10/2001, the latter 02 being the second time the zone file has been modified this day. The serial number is important as it alerts slave nameservers for a zone when it is updated. @ IN NS ns1.foobardomain.org. This is an NS entry. Every nameserver that is going to reply authoritatively for the zone must have one of these entries. The @ as seen here could have been 'foobardomain.org.' The @ translates to the origin. localhost IN A 127.0.0.1 ns1 IN A 3.2.1.2 ns2 IN A 3.2.1.3 mail IN A 3.2.1.10 @ IN A 3.2.1.30 The A record indicates machine names. As seen above, ns1.foobardomain.org would resolve to 3.2.1.2. Again, the origin symbol, @, is used here, thus meaning foobardomain.org would resolve to 3.2.1.30. www IN CNAME @ The canonical name record is usually used for giving aliases to a machine. In the example, www is aliased to the machine addressed to the origin, or foobardomain.org (3.2.1.30). CNAMEs can be used to provide alias hostnames, or round robin one hostname among multiple machines. @ IN MX 10 mail.foobardomain.org. The MX record indicates which mail servers are responsible for handling incoming mail for the zone. mail.foobardomain.org is the hostname of the mail server, and 10 being the priority of that mailserver. One can have several mailservers, with priorities of 3, 2, 1. A mail server attempting to deliver to foobardomain.org would first try the highest priority MX, then the second highest, etc, until the mail can be properly delivered. For in-addr.arpa zone files (reverse dns), the same format is used, except with PTR entries instead of A or CNAME. $TTL 3600 1.2.3.in-addr.arpa. IN SOA ns1.foobardomain.org. admin.foobardomain.org. ( 5 ; Serial 10800 ; Refresh 3600 ; Retry 604800 ; Expire 3600 ) ; Minimum @ IN NS ns1.foobardomain.org. @ IN NS ns2.foobardomain.org. 2 IN PTR ns1.foobardomain.org. 3 IN PTR ns2.foobardomain.org. 10 IN PTR mail.foobardomain.org. 30 IN PTR foobardomain.org. This file gives the proper IP to hostname mappings of our above fictitious domain. Caching Name Server BIND caching name server A caching nameserver is simply a nameserver that is not authoritative for any zones. It simply asks queries of its own, and remembers them for later use. To set one up, just configure the name server as usual, omitting any inclusions of zones. Running named in a Sandbox BIND running in a sandbox Contributed by Mike Makonnen mike_makonnen@yahoo.com, May 1, 2001 chroot For added security you may want to run &man.named.8; in a sandbox. This will reduce the potential damage should it be compromised. If you include a sandbox directory in its command line, named will &man.chroot.8; into that directory immediately upon finishing processing its command line. It is also a good idea to have named run as a non-privileged user in the sandbox. The default FreeBSD install contains a user bind with group bind. If we wanted the sandbox in the /etc/namedb/sandbox directory the command line for named would look like this: - &prompt.root; /usr/sbin/named -u bind -g bind -t /etc/namedb/sandbox <path_to_named.conf> - + &prompt.root; /usr/sbin/named -u bind -g bind -t /etc/namedb/sandbox <path_to_named.conf> The following steps should be taken in order to successfully run named in a sandbox. Throughout the following discussion we will assume the path to your sandbox is /etc/namedb/sandbox Create the sandbox directory: /etc/namedb/sandbox Create other necessary directories off of the sandbox directory: etc and var/run copy /etc/localtime to sandbox/etc make bind:bind the owner of all files and directories in the sandbox: &prompt.root; chown -R bind:bind /etc/namedb/sandbox &prompt.root; chmod -R 750 /etc/namedb/sandbox There are some issues you need to be aware of when running named in a sandbox. Your &man.named.conf.5; file and all your zone files must be in the sandbox sandbox/etc/localtime is needed in order to have the correct time for your time zone in log messages. &man.named.8; will write its process id to a file in sandbox/var/run The Unix socket used for communication by the &man.ndc.8; utility will be created in sandbox/var/run When using the ndc utility you need to specify the location of the Unix socket created in the sandbox, by &man.named.8;, by using the -c switch: &prompt.root; ndc -c /etc/namedb/sandbox/var/run/ndc If you enable logging to file, the log files must be in the sandbox &man.named.8; can be started in a sandbox properly, if the following is in /etc/rc.conf named_flags="-u bind -g bind -t /etc/namedb/sandbox" How to use the nameserver If setup properly, the nameserver should be accessible through the network and locally. /etc/resolv.conf must contain a nameserver entry with the local ip so it will query the local name server first. To access it over the network, the machine must have the nameserver's IP address set properly in its own nameserver configuration options. Security Although BIND is the most common implementation of DNS, there is always the issue of security. Possible and exploitable security holes are sometimes found. It is a good idea to subscribe to CERT and freebsd-announce to stay up to date with the current Internet and FreeBSD security issues. If a problem arises, keeping your sources up to date and having a fresh build of named can't hurt. Further Reading &man.ndc.8; &man.named.8; &man.named.conf.5; Official ISC BIND Page http://www.isc.org/products/BIND/ BIND FAQ http://www.nominum.com/resources/faqs/bind-faqs.html O'Reilly DNS and BIND 4th Edition RFC1034 - Domain Names - Concepts and Facilities RFC1035 - Domain Names - Implementation and Specification Network Address Translation daemon (natd) Contributed by &a.chern;, June 2001. Overview natd FreeBSD's Network Address Translation daemon, commonly known as &man.natd.8; is a daemon that accepts incoming raw IP packets, changes the source to the local machine and re-injects these packets back into the outgoing IP packet stream. natd does this by changing the source ip and port such that when data is received back, it is able to determine the original location of the data and forward it back to its original requestor. Internet connection sharing IP masquerading The most common use of NAT is to perform what is commonly known as Internet Connection Sharing. Setup Due to the diminishing ip space in ipv4, and the increased number of users on high-speed consumer lines such as cable or DSL, people are in more and more need of an Internet Connection Sharing solution. The ability to connect several computers online through one connection and ip makes &man.natd.8; a reasonable choice. Most commonly, a user has a machine connected to a cable or DSL line with one ip and wishes to use this one connected computer to provide internet access to several more over a LAN. To do this, the FreeBSD machine on the Internet must act as a gateway. This gateway machine must have two NICs--one for connecting to the Internet router, the other connecting to a LAN. All the machines on the LAN are connected through a hub or switch. _______ __________ ________ | | | | | | | Hub |-----| Client B |-----| Router |----- Internet |_______| |__________| |________| | ____|_____ | | | Client A | |__________| Network Layout With this setup, the machine without Internet access can use the machine with access as a gateway to access the outside world. kernel configuration Configuration The following options must be in the kernel configuration file: options IPFIREWALL options IPDIVERT Additionally, at choice, the following may also be suitable: options IPFIREWALL_DEFAULT_TO_ACCEPT options IPFIREWALL_VERBOSE The following must be in /etc/rc.conf: gateway_enable="YES" firewall_enable="YES" firewall_type="OPEN" natd_enable="YES" natd_interface="fxp0" natd_flags="" gateway_enable="YES" Sets up the machine to act as a gateway. Running sysctl -w net.inet.ip.forwarding=1 would have the same effect. firewall_enable="YES" Enables the firewall rules in /etc/rc.firewall at boot. firewall_type="OPEN" This specifies a predefined firewall ruleset that allows anything in. See /etc/rc.firewall for additional types. natd_interface="fxp0" Indicates which interface to forward packets through. (the interface connected to the Internet) natd_flags="" Any additional configuration options passed to &man.natd.8; on boot. Having the previous options defined in /etc/rc.conf would run natd -interface fxp0 at boot. This can also be run manually. Each machine and interface behind the LAN should be assigned ip numbers in the private network space as defined by RFC 1918 and have a default gateway of the natd machine's internal ip. For example, client a and b behind the LAN have ips of 192.168.0.2 and 192.168.0.3, while the natd machine's LAN interface has an ip of 192.168.0.1. Client a and b's default gateway must be set to that of the natd machine, 192.168.0.1. The natd machine's external, or Internet interface does not require any special modification for natd to work. Port Redirection The drawback with natd is that the LAN clients are not accessible from the Internet. Clients on the LAN can make outgoing connections to the world but cannot receive incoming ones. This presents a problem if trying to run Internet services on one of the LAN client machines. A simple way around this is to redirect selected Internet ports on the natd machine to a LAN client. For example, an IRC server runs on Client A, and a web server runs on Client B. For this to work properly, connections received on ports 6667 (irc) and 80 (web) must be redirected to the respective machines. The -redirect_port must be passed to &man.natd.8; with the proper options. The syntax is as follows: -redirect_port proto targetIP:targetPORT[-targetPORT] [aliasIP:]aliasPORT[-aliasPORT] [remoteIP[:remotePORT[-remotePORT]]] In the above example, the argument should be: -redirect_port tcp 192.168.0.2:6667 6667 -redirect_port tcp 192.168.0.3:80 80 This will redirect the proper tcp ports to the LAN client machines. The -redirect_port argument can be used to indicate port ranges over individual ports. For example, tcp 192.168.0.2:2000-3000 2000-3000 would redirect all connections received on ports 2000 to 3000 to ports 2000 to 3000 on Client A. These options can be used when directly running &man.natd.8; or placed within the natd_flags="" option in /etc/rc.conf. For further configuration options, consult &man.natd.8; Address Redirection address redirection Address redirection is useful if several ips are available, yet they must be on one machine. With this, &man.natd.8; can assign each LAN client its own external ip. &man.natd.8; then rewrites outgoing packets from the LAN clients with the proper external ip and redirects all traffic incoming on that particular ip back to the specific LAN client. This is also known as static NAT. For example, the ips 128.1.1.1, 128.1.1.2, and 128.1.1.3 belong to the natd gateway machine. 128.1.1.1 can be used as the natd gateway machine's external ip address, while 128.1.1.2 and 128.1.1.3 are forwarded back to LAN clients A and B. The -redirect_address syntax is as follows: -redirect_address localIP publicIP localIP The internal ip of the LAN client. publicIP The external ip corresponding to the LAN client. In the example, this argument would read: -redirect_address 192.168.0.2 128.1.1.2 -redirect_address 192.168.0.3 128.1.1.3 Like -redirect_port, these arguments are also placed within natd_flags of /etc/rc.conf. With address redirection, there is no need for port redirection since all data received on a particular ip address is redirected. The external ips on the natd machine must be active and aliased to the external interface. Look at &man.rc.conf.5; to do so. diff --git a/en_US.ISO8859-1/books/handbook/boot/chapter.sgml b/en_US.ISO8859-1/books/handbook/boot/chapter.sgml index 234167ff5c..3d728b73c4 100644 --- a/en_US.ISO8859-1/books/handbook/boot/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/boot/chapter.sgml @@ -1,576 +1,575 @@ The FreeBSD Booting Process Synopsis booting bootstrap FreeBSD uses a three-stage bootstrap by default, which basically entails three programs which call each other in order (two boot blocks, and the loader). Each of these three build on the previous program's understanding and provide increasing amounts of sophistication. kernel init The kernel is then started, which will then probe for devices and initialize them for use. Once the kernel boot process is finished, the kernel passes control to the user process &man.init.8;, which then makes sure the disks are in a usable state. &man.init.8; then starts the user-level resource configuration which then mounts filesystems, sets up network cards to act on the network, and generally starts all the processes that usually are run on a FreeBSD system at startup. The Boot Blocks: Bootstrap Stages 1 and 2 Bootstrapping is the process whereby a computer probes and initializes its devices, and works out what programs it is supposed to run. This involves the use of special Read Only Memory chips, which determine what further operations to do, and these usually pass control to other chips that do consistency and memory tests, configure devices, and provide a mechanism for programs to determine what configuration details were determined. BIOS CMOS In standard personal computers, this involves the BIOS (which oversees the bootstrap), and CMOS (which stores configuration). BIOS and CMOS understand disks, and also understand where on the disk to find a program that will know how to load up an operating system. This chapter will not deal with this first part of the bootstrap process. Instead it will focus on what happens after control is passed to the program on the disk. The boot blocks are responsible for finding (usually) the loader, and running it, and thus need to understand how to find that program on the filesystem, how to run the program, and also allow minor configuration of how they work. boot0 Master Boot Record (MBR) There is actually a preceding bootblock, named boot0, which lives on the Master Boot Record, the special part of the disk that the system bootstrap looks for and runs, and it simply shows a list of possible slices to boot from. boot0 is very simple, since the program in the MBR can only be 512 bytes in size. It displays something like this: boot0 screenshot - -F1 DOS + F1 DOS F2 FreeBSD F3 Linux F4 ?? F5 Drive 1 Default: F2 boot1 boot1 is found on the boot sector of the boot slice, which is where boot0, or any other program on the MBR expects to find the program to run to continue the boot process. boot1 is very simple, since it too can only be 512 bytes in size, and knows just enough about the FreeBSD disklabel, which stores information about the slice, to find and execute boot2. boot2 boot2 is slightly more sophisticated, and understands the FreeBSD filesystem enough to find files on it, and can provide a simple interface to choose the kernel or loader to run. Since the loader is much more sophisticated, and provides a nice easy-to-use boot configuration, boot2 usually runs it, but previously it was tasked to run the kernel directly. boot2 screenshot >> FreeBSD/i386 BOOT Default: 0:wd(0,a)/kernel boot: Loader: Bootstrap Stage Three boot-loader The loader is the final stage of the three-stage bootstrap, and is located on the filesystem, usually as /boot/loader. While /boot/boot0, /boot/boot1, and /boot/boot2 are files there, they are not the actual copies in the MBR, the boot sector, or the disklabel respectively. The loader is intended as a user-friendly method for configuration, using an easy-to-use built-in command set, backed up by a more powerful interpreter, with a more complex command set. Loader Program Flow During initialization, the loader will probe for a console and for disks, and figure out what disk it is booting from. It will set variables accordingly, and then the interpreter is started, and the easy-to-use commands are explained to it. loader loader configuration loader will then read /boot/loader.rc, which by default reads in /boot/defaults/loader.conf which sets reasonable defaults for variables and reads /boot/loader.conf for local changes to those variables. loader.rc then acts on these variables, loading whichever modules and kernel are selected. Finally, by default, the loader issues a 10 second wait for key presses, and boots the kernel if it is not interrupted. If interrupted, the user is presented with a prompt which understands the easy-to-use command set, where the user may adjust variables, unload all modules, load modules, and then finally boot or reboot. A more technical discussion of the process is available in &man.loader.8; Loader Built-In Commands The easy-to-use command set comprises of: autoboot seconds Proceeds to boot the kernel if not interrupted within the time span given, in seconds. It displays a countdown, and the default timespan is 10 seconds. boot -options kernelname Immediately proceeds to boot the kernel, with the given options, if any, and with the kernel name given, if it is. boot-conf Goes through the same automatic configuration of modules based on variables as what happens at boot. This only makes sense if you use unload first, and change some variables, most commonly kernel. help topic Shows help messages read from /boot/loader.help. If the topic given is index, then the list of available topics is given. include filename … Processes the file with the given filename. The file is read in, and interpreted line by line. An error immediately stops the include command. load type filename Loads the kernel, kernel module, or file of the type given, with the filename given. Any arguments after filename are passed to the file. ls path Displays a listing of files in the given path, or the root directory, if the path is not specified. If is specified, file sizes will be shown too. lsdev Lists all of the devices from which it may be possible to load modules. If is specified, more details are printed. lsmod Displays loaded modules. If is specified, more details are shown. more filename Display the files specified, with a pause at each LINES displayed. reboot Immediately reboots the system. set variable set variable=value Set loader's environment variables. unload Removes all loaded modules. Loader Examples Here are some practical examples of loader usage. single-user mode To simply boot your usual kernel, but in single-user mode: boot -s To unload your usual kernel and modules, and then load just your old (or another) kernel: kernel.old unload load kernel.old You can use kernel.GENERIC to refer to the generic kernel that comes on the install disk, or kernel.old to refer to your previously installed kernel (when you've upgraded or configured your own kernel, for example). Use the following to load your usual modules with another kernel: unload set kernel="kernel.old" boot-conf To load a kernel configuration script (an automated script which does the things you'd normally do in the kernel boot-time configurator): load -t userconfig_script /boot/kernel.conf Kernel Interaction During Boot kernel boot interaction Once the kernel is loaded by either loader (as usual) or boot2 (bypassing the loader), it examines its boot flags, if any, and adjusts its behavior as necessary. kernel bootflags Kernel Boot Flags Here are the more common boot flags: during kernel initialization, ask for the device to mount as the root file system. boot from CDROM. run UserConfig, the boot-time kernel configurator boot into single-user mode be more verbose during kernel startup There are other boot flags, read &man.boot.8; for more information on them. init Init: Process Control Initialization Once the kernel has finished booting, it passes control to the user process init, which is located at /sbin/init, or the program path specified in the init_path variable in loader. Automatic Reboot Sequence The automatic reboot sequence makes sure that the filesystems available on the system are consistent. If they are not, and fsck can not fix the inconsistencies, init drops the system into single-user mode for the system administrator to take care of the problems directly. Single-User Mode single-user mode console This mode can be reached through the automatic reboot sequence, or by the user booting with the or setting the boot_single variable in loader. It can also be reached by calling shutdown without the reboot () or halt () options, from multi-user mode. If the system console console is set to insecure in /etc/ttys, then the system prompts for the root password before initiating single-user mode. An insecure console in /etc/ttys # name getty type status comments # # This entry needed for asking password when init goes to single-user mode # If you want to be asked for password, change "secure" to "insecure" here console none unknown off insecure An insecure console means that you consider your physical security to the console to be insecure, and want to make sure only someone who knows the root password may use single-user mode, and it does not mean that you want to run your console insecurely. Thus, if you want security, choose insecure, not secure. Multi-User Mode multi-user mode If init finds your filesystems to be in order, or once the user has finished in single-user mode, the system enters multi-user mode, in which it starts the resource configuration of the system. rc files Resource Configuration (rc) The resource configuration system reads in configuration defaults from /etc/defaults/rc.conf, and system-specific details from /etc/rc.conf, and then proceeds to mount the system filesystems mentioned in /etc/fstab, start up networking services, starts up miscellaneous system daemons, and finally runs the startup scripts of locally installed packages. &man.rc.8; is a good reference to the resource configuration system, as is examining the scripts themselves. Shutdown Sequence shutdown Upon controlled shutdown, via shutdown, init will attempt to run the script /etc/rc.shutdown, and then proceed to send all processes the terminate signal, and subsequently the kill signal to any that don't terminate timely. diff --git a/en_US.ISO8859-1/books/handbook/disks/chapter.sgml b/en_US.ISO8859-1/books/handbook/disks/chapter.sgml index a2a7aef698..a1c1392f7f 100644 --- a/en_US.ISO8859-1/books/handbook/disks/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/disks/chapter.sgml @@ -1,1133 +1,1132 @@ Disks Synopsis This chapter covers how to use disks, whether physical, memory, or networked, on FreeBSD. BIOS Drive Numbering Before you install and configure FreeBSD on your system, there is an important subject that you should be aware of if, especially if you have multiple hard drives. DOS Microsoft Windows In a PC running DOS or any of the BIOS-dependent operating systems (WINxxx), the BIOS is able to abstract the normal disk drive order, and the operating system goes along with the change. This allows the user to boot from a disk drive other than the so-called primary master. This is especially convenient for some users who have found that the simplest and cheapest way to keep a system backup is to buy an identical second hard drive, and perform routine copies of the first drive to the second drive using Ghost or XCOPY. Then, if the first drive fails, or is attacked by a virus, or is scribbled upon by an operating system defect, he can easily recover by instructing the BIOS to logically swap the drives. It's like switching the cables on the drives, but without having to open the case. SCSI BIOS More expensive systems with SCSI controllers often include BIOS extensions which allow the SCSI drives to be re-ordered in a similar fashion for up to seven drives. A user who is accustomed to taking advantage of these features may become surprised when the results with FreeBSD are not as expected. FreeBSD does not use the BIOS, and does not know the logical BIOS drive mapping. This can lead to very perplexing situations, especially when drives are physically identical in geometry, and have also been made as data clones of one another. When using FreeBSD, always restore the BIOS to natural drive numbering before installing FreeBSD, and then leave it that way. If you need to switch drives around, then do so, but do it the hard way, and open the case and move the jumpers and cables. An illustration from the files of Bill and Fred's Exceptional Adventures: Bill breaks-down an older Wintel box to make another FreeBSD box for Fred. Bill installs a single SCSI drive as SCSI unit zero, and installs FreeBSD on it. Fred begins using the system, but after several days notices that the older SCSI drive is reporting numerous soft errors, and reports this fact to Bill. After several more days, Bill decides it's time to address the situation, so he grabs an identical SCSI drive from the disk drive "archive" in the back room. An initial surface scan indicates that this drive is functioning well, so Bill installs this drive as SCSI unit four, and makes an image copy from drive zero to drive four. Now that the new drive is installed and functioning nicely, Bill decides that it's a good idea to start using it, so he uses features in the SCSI BIOS to re-order the disk drives so that the system boots from SCSI unit four. FreeBSD boots and runs just fine. Fred continues his work for several days, and soon Bill and Fred decide that it's time for a new adventure -- time to upgrade to a newer version of FreeBSD. Bill removes SCSI unit zero because it was a bit flaky, and replaces it with another identical disk drive from the "archive." Bill then installs the new version of FreeBSD onto the new SCSI unit zero using Fred's magic Internet FTP floppies. The installation goes well. Fred uses the new version of FreeBSD for a few days, and certifies that it is good enough for use in the engineering department...it's time to copy all of his work from the old version. So Fred mounts SCSI unit four (the latest copy of the older FreeBSD version). Fred is dismayed to find that none of his precious work is present on SCSI unit four. Where did the data go? When Bill made an image copy of the original SCSI unit zero onto SCSI unit four, unit four became the "new clone," When Bill re-ordered the SCSI BIOS so that he could boot from SCSI unit four, he was only fooling himself. FreeBSD was still running on SCSI unit zero. Making this kind of BIOS change will cause some or all of the Boot and Loader code to be fetched from the selected BIOS drive, but when the FreeBSD kernel drivers take-over, the BIOS drive numbering will be ignored, and FreeBSD will transition back to normal drive numbering. In the illustration at hand, the system continued to operate on the original SCSI unit zero, and all of Fred's data was there, not on SCSI unit four. The fact that the system appeared to be running on SCSI unit four was simply an artifact of human expectations. We are delighted to mention that no data bytes were killed or harmed in any way by our discovery of this phenomenon. The older SCSI unit zero was retrieved from the bone pile, and all of Fred's work was returned to him, (and now Bill knows that he can count as high as zero). Although SCSI drives were used in this illustration, the concepts apply equally to IDE drives. Disk Naming IDE SCSI RAID fash memory Physical drives come in two main flavors, IDE, or SCSI; but there are also drives backed by RAID controllers, flash memory, and so forth. Since these behave quite differently, they have their own drivers and devices. Physical Disk Naming Conventions Drive type Drive device name IDE hard drives ad in 4.0-RELEASE, wd before 4.0-RELEASE. IDE CDROM drives acd from 4.1-RELEASE, wcd before 4.0-RELEASE. SCSI hard drives da from 3.0-RELEASE, sd before 3.0-RELEASE. SCSI CDROM drives cd Assorted non-standard CDROM drives mcd for Mitsumi CD-ROM, scd for Sony CD-ROM, matcd for Matsushita/Panasonic CD-ROM Floppy drives fd SCSI tape drives sa from 3.0-RELEASE, st before 3.0-RELEASE. IDE tape drives ast from 4.0-RELEASE, wst before 4.0-RELEASE. Flash drives fla for DiskOnChip Flash device from 3.3-RELEASE. RAID drives myxd for Mylex, and amrd for AMI MegaRAID, idad for Compaq Smart RAID. from 4.0-RELEASE. id between 3.2-RELEASE and 4.0-RELEASE.
Slices and Partitions slices partitions dangerously dedicated Physical disks usually contain slices, unless they are dangerously dedicated. Slice numbers follow the device name, prefixed with an s: da0s1. Slices, dangerously dedicated physical drives, and other drives contain partitions, which represented as letters from a to h. b is reserved for swap partitions, and c is an unused partition the size of the entire slice or drive. This is explained in . Mounting and Unmounting Filesystems The filesystem is best visualized as a tree, rooted, as it were, at /. /dev, /usr, and the other directories in the root directory are branches, which may have their own branches, such as /usr/local, and so on. root filesystem There are various reasons to house some of these directories on separate filesystems. /var contains log, spool, and various types of temporary files, and as such, may get filled up. Filling up the root filesystem isn't a good idea, so splitting /var from / is often a good idea. Another common reason to contain certain directory trees on other filesystems is if they are to be housed on separate physical disks, or are separate virtual disks, such as Network File System mounts, or CDROM drives. The fstab File filesystems mounted with fstab During the boot process, filesystems listed in /etc/fstab are automatically mounted (unless they are listed with ). The /etc/fstab file contains a list of lines of the following format: device /mount-point fstype options dumpfreq passno device is a device name (which should exist), as explained in the Disk naming conventions above. mount-point is a directory (which should exist), on which to mount the filesystem. fstype is the filesystem type to pass to &man.mount.8;. The default FreeBSD filesystem is ufs. options is either for read-write filesystems, or for read-only filesystems, followed by any other options that may be needed. A common option is for filesystems not normally mounted during the boot sequence. Other options in the &man.mount.8; manual page. dumpfreq is the number of days the filesystem should be dumped, and passno is the pass number during which the filesystem is checked during the boot sequence. The mount Command filesystemsmounting The &man.mount.8; command is what is ultimately used to mount filesystems. In its most basic form, you use: &prompt.root; mount device mountpoint There are plenty of options, as mentioned in the &man.mount.8; manual page, but the most common are: mount options Mount all filesystems in /etc/fstab, as modified by , if given. Do everything but actually mount the filesystem. Force the mounting the filesystem. Mount the filesystem read-only. fstype Mount the given filesystem as the given filesystem type, or mount only filesystems of the given type, if given the option. ufs is the default filesystem type. Update mount options on the filesystem. Be verbose. Mount the filesystem read-write. The takes a comma-separated list of the options, including the following: nodev Do not interpret special devices on the filesystem. Useful security option. noexec Do not allow execution of binaries on this filesystem. Useful security option. nosuid Do not interpret setuid or setgid flags on the filesystem. Useful security option. The umount Command filesystemsunmounting The umount command takes, as a parameter, one of a mountpoint, a device name, or the or option. All forms take to force unmounting, and for verbosity. and are used to unmount all mounted filesystems, possibly modified by the filesystem types listed after . , however, doesn't attempt to unmount the root filesystem. Adding Disks disksadding Originally contributed by &a.obrien; 26 April 1998 Lets say we want to add a new SCSI disk to a machine that currently only has a single drive. First turn off the computer and install the drive in the computer following the instructions of the computer, controller, and drive manufacturer. Due the wide variations of procedures to do this, the details are beyond the scope of this document. Login as user root. After you've installed the drive, inspect /var/run/dmesg.boot to ensure the new disk was found. Continuing with our example, the newly added drive will be da1 and we want to mount it on /1 (if you are adding an IDE drive, it will be wd1 in pre-4.0 systems, or ad1 in most 4.X systems). partitions slices fdisk Because FreeBSD runs on IBM-PC compatible computers, it must take into account the PC BIOS partitions. These are different from the traditional BSD partitions. A PC disk has up to four BIOS partition entries. If the disk is going to be truly dedicated to FreeBSD, you can use the dedicated mode. Otherwise, FreeBSD will have to live with in one of the PC BIOS partitions. FreeBSD calls the PC BIOS partitions slices so as not to confuse them with traditional BSD partitions. You may also use slices on a disk that is dedicated to FreeBSD, but used in a computer that also has another operating system installed. This is to not confuse the fdisk utility of the other operating system. In the slice case the drive will be added as /dev/da1s1e. This is read as: SCSI disk, unit number 1 (second SCSI disk), slice 1 (PC BIOS partition 1), and e BSD partition. In the dedicated case, the drive will be added simply as /dev/da1e. Using sysinstall sysinstalladding disks You may use /stand/sysinstall to partition and label a new disk using its easy to use menus. Either login as user root or use the su command. Run /stand/sysinstall and enter the Configure menu. With in the FreeBSD Configuration Menu, scroll down and select the Partition item. Next you should be presented with a list of hard drives installed in your system. If you do not see da1 listed, you need to recheck your physical installation and dmesg output in the file /var/run/dmesg.boot. Select da1 to enter the FDISK Partition Editor. Choose A to use the entire disk for FreeBSD. When asked if you want to remain cooperative with any future possible operating systems, answer YES. Write the changes to the disk using W. Now exit the FDISK editor using q. Next you will be asked about the Master Boot Record. Since you are adding a disk to an already running system, choose None. BSD partitions Next enter the Disk Label Editor. This is where you will create the traditional BSD partitions. A disk can have up to eight partitions, labeled a-h. A few of the partition labels have special uses. The a partition is used for the root partition (/). Thus only your system disk (e.g, the disk you boot from) should have an a partition. The b partition is used for swap partitions, and you may have many disks with swap partitions. The c partition addresses the entire disk in dedicated mode, or the entire FreeBSD slice in slice mode. The other partitions are for general use. Sysinstall's Label editor favors the e partition for non-root, non-swap partitions. With in the Label editor, create a single file system using C. When prompted if this will be a FS (file system) or swap, choose FS and give a mount point (e.g, /mnt). When adding a disk in post-install mode, Sysinstall will not create entries in /etc/fstab for you, so the mount point you specify isn't important. You are now ready to write the new label to the disk and create a file system on it. Do this by hitting W. Ignore any errors from Sysinstall that it could not mount the new partition. Exit the Label Editor and Sysinstall completely. The last step is to edit /etc/fstab to add an entry for your new disk. Using Command Line Utilities Using Slices This setup will allow your disk to work correctly with other operating systems that might be installed on your computer and will not confuse other operating systems' fdisk utilities. It is recommended to use this method for new disk installs. Only use dedicated mode if you have a good reason to do so! &prompt.root; dd if=/dev/zero of=/dev/rda1 bs=1k count=1 &prompt.root; fdisk -BI da1 #Initialize your new disk &prompt.root; disklabel -B -w -r da1s1 auto #Label it. &prompt.root; disklabel -e da1s1 # Now edit the disklabel you just created and add any partitions. &prompt.root; mkdir -p /1 &prompt.root; newfs /dev/da1s1e # Repeat this for every partition you created. &prompt.root; mount -t ufs /dev/da1s1e /1 # Mount the partition(s) &prompt.root; vi /etc/fstab # When satisfied, add the appropriate entry/entries to your /etc/fstab. If you have an IDE disk, substitute ad for da. On pre-4.x systems use wd. Dedicated OS/2 If you will not be sharing the new drive with another operating system, you may use the dedicated mode. Remember this mode can confuse Microsoft operating systems; however, no damage will be done by them. IBM's OS/2 however, will appropriate any partition it finds which it doesn't understand. &prompt.root; dd if=/dev/zero of=/dev/rda1 bs=1k count=1 &prompt.root; disklabel -Brw da1 auto &prompt.root; disklabel -e da1 # create the `e' partition &prompt.root; newfs -d0 /dev/rda1e &prompt.root; mkdir -p /1 &prompt.root; vi /etc/fstab # add an entry for /dev/da1e &prompt.root; mount /1 An alternate method is: &prompt.root; dd if=/dev/zero of=/dev/rda1 count=2 &prompt.root; disklabel /dev/rda1 | disklabel -BrR da1 /dev/stdin &prompt.root; newfs /dev/rda1e &prompt.root; mkdir -p /1 &prompt.root; vi /etc/fstab # add an entry for /dev/da1e &prompt.root; mount /1 Virtual Disks: Network, Memory, and File-Based Filesystems virtual disks disksvirtual Aside from the disks you physically insert into your computer: floppies, CDs, hard drives, and so forth; other forms of disks are understood by FreeBSD - the virtual disks. NFS Coda disksmemory These include network filesystems such as the Network Filesystem and Coda, memory-based filesystems such as md and file-backed filesystems created by vnconfig. vnconfig: file-backed filesystem disksfile-backed &man.vnconfig.8; configures and enables vnode pseudo disk devices. A vnode is a representation of a file, and is the focus of file activity. This means that &man.vnconfig.8; uses files to create and operate a filesystem. One possible use is the mounting of floppy or CD images kept in files. To mount an existing filesystem image: Using vnconfig to mount an existing filesystem image &prompt.root; vnconfig vn0 diskimage &prompt.root; mount /dev/vn0c /mnt To create a new filesystem image with vnconfig: Creating a New File-Backed Disk with vnconfig &prompt.root; dd if=/dev/zero of=newimage bs=1k count=5k 5120+0 records in 5120+0 records out &prompt.root; vnconfig -s labels -c vn0 newimage &prompt.root; disklabel -r -w vn0 auto &prompt.root; newfs vn0c Warning: 2048 sector(s) in last cylinder unallocated /dev/rvn0c: 10240 sectors in 3 cylinders of 1 tracks, 4096 sectors 5.0MB in 1 cyl groups (16 c/g, 32.00MB/g, 1280 i/g) super-block backups (for fsck -b #) at: 32 &prompt.root; mount /dev/vn0c /mnt &prompt.root; df /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/vn0c 4927 1 4532 0% /mnt md: Memory Filesystem disksmemory md is a simple, efficient means to do memory filesystems. Simply take a filesystem you've prepared with, for example, &man.vnconfig.8;, and: md memory disk &prompt.root; dd if=newimage of=/dev/md0 5120+0 records in 5120+0 records out &prompt.root; mount /dev/md0c /mnt &prompt.root; df /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md0c 4927 1 4532 0% /mnt Disk Quotas accountingdisk space disk quotas Quotas are an optional feature of the operating system that allow you to limit the amount of disk space and/or the number of files a user, or members of a group, may allocate on a per-file system basis. This is used most often on timesharing systems where it is desirable to limit the amount of resources any one user or group of users may allocate. This will prevent one user from consuming all of the available disk space. Configuring Your System to Enable Disk Quotas Before attempting to use disk quotas it is necessary to make sure that quotas are configured in your kernel. This is done by adding the following line to your kernel configuration file: options QUOTA The stock GENERIC kernel does not have this enabled by default, so you will have to configure, build and install a custom kernel in order to use disk quotas. Please refer to the Configuring the FreeBSD Kernel section for more information on kernel configuration. Next you will need to enable disk quotas in /etc/rc.conf. This is done by adding the line: enable_quotas=YES disk quotaschecking For finer control over your quota startup, there is an additional configuration variable available. Normally on bootup, the quota integrity of each file system is checked by the quotacheck program. The quotacheck facility insures that the data in the quota database properly reflects the data on the file system. This is a very time consuming process that will significantly affect the time your system takes to boot. If you would like to skip this step, a variable is made available for the purpose: check_quotas=NO If you are running FreeBSD prior to 3.2-RELEASE, the configuration is simpler, and consists of only one variable. Set the following in your /etc/rc.conf: check_quotas=YES Finally you will need to edit /etc/fstab to enable disk quotas on a per-file system basis. This is where you can either enable user or group quotas or both for all of your file systems. To enable per-user quotas on a file system, add the userquota option to the options field in the /etc/fstab entry for the file system you want to enable quotas on. For example: /dev/da1s2g /home ufs rw,userquota 1 2 Similarly, to enable group quotas, use the groupquota option instead of the userquota keyword. To enable both user and group quotas, change the entry as follows: /dev/da1s2g /home ufs rw,userquota,groupquota 1 2 By default the quota files are stored in the root directory of the file system with the names quota.user and quota.group for user and group quotas respectively. See man fstab for more information. Even though that man page says that you can specify an alternate location for the quota files, this is not recommended because the various quota utilities do not seem to handle this properly. At this point you should reboot your system with your new kernel. /etc/rc will automatically run the appropriate commands to create the initial quota files for all of the quotas you enabled in /etc/fstab, so there is no need to manually create any zero length quota files. In the normal course of operations you should not be required to run the quotacheck, quotaon, or quotaoff commands manually. However, you may want to read their man pages just to be familiar with their operation. Setting Quota Limits disk quotaslimits Once you have configured your system to enable quotas, verify that they really are enabled. An easy way to do this is to run: &prompt.root; quota -v You should see a one line summary of disk usage and current quota limits for each file system that quotas are enabled on. You are now ready to start assigning quota limits with the edquota command. You have several options on how to enforce limits on the amount of disk space a user or group may allocate, and how many files they may create. You may limit allocations based on disk space (block quotas) or number of files (inode quotas) or a combination of both. Each of these limits are further broken down into two categories; hard and soft limits. hard limit A hard limit may not be exceeded. Once a user reaches his hard limit he may not make any further allocations on the file system in question. For example, if the user has a hard limit of 500 blocks on a file system and is currently using 490 blocks, the user can only allocate an additional 10 blocks. Attempting to allocate an additional 11 blocks will fail. soft limit Soft limits, on the other hand, can be exceeded for a limited amount of time. This period of time is known as the grace period, which is one week by default. If a user stays over his or her soft limit longer than the grace period, the soft limit will turn into a hard limit and no further allocations will be allowed. When the user drops back below the soft limit, the grace period will be reset. The following is an example of what you might see when you run the edquota command. When the edquota command is invoked, you are placed into the editor specified by the EDITOR environment variable, or in the vi editor if the EDITOR variable is not set, to allow you to edit the quota limits. &prompt.root; edquota -u test Quotas for user test: /usr: blocks in use: 65, limits (soft = 50, hard = 75) inodes in use: 7, limits (soft = 50, hard = 60) /usr/var: blocks in use: 0, limits (soft = 50, hard = 75) inodes in use: 0, limits (soft = 50, hard = 60) You will normally see two lines for each file system that has quotas enabled. One line for the block limits, and one line for inode limits. Simply change the value you want updated to modify the quota limit. For example, to raise this users block limit from a soft limit of 50 and a hard limit of 75 to a soft limit of 500 and a hard limit of 600, change: /usr: blocks in use: 65, limits (soft = 50, hard = 75) to: /usr: blocks in use: 65, limits (soft = 500, hard = 600) The new quota limits will be in place when you exit the editor. Sometimes it is desirable to set quota limits on a range of uids. This can be done by use of the option on the edquota command. First, assign the desired quota limit to a user, and then run edquota -p protouser startuid-enduid. For example, if user test has the desired quota limits, the following command can be used to duplicate those quota limits for uids 10,000 through 19,999: &prompt.root; edquota -p test 10000-19999 See man edquota for more detailed information. Checking Quota Limits and Disk Usage disk quotaschecking You can use either the quota or the repquota commands to check quota limits and disk usage. The quota command can be used to check individual user and group quotas and disk usage. Only the super-user may examine quotas and usage for other users, or for groups that they are not a member of. The repquota command can be used to get a summary of all quotas and disk usage for file systems with quotas enabled. The following is some sample output from the quota -v command for a user that has quota limits on two file systems. Disk quotas for user test (uid 1002): Filesystem blocks quota limit grace files quota limit grace /usr 65* 50 75 5days 7 50 60 /usr/var 0 50 75 0 50 60 grace period On the /usr file system in the above example this user is currently 15 blocks over the soft limit of 50 blocks and has 5 days of the grace period left. Note the asterisk * which indicates that the user is currently over his quota limit. Normally file systems that the user is not using any disk space on will not show up in the output from the quota command, even if he has a quota limit assigned for that file system. The option will display those file systems, such as the /usr/var file system in the above example. Quotas over NFS NFS Quotas are enforced by the quota subsystem on the NFS server. The &man.rpc.rquotad.8; daemon makes quota information available to the &man.quota.1; command on NFS clients, allowing users on those machines to see their quota statistics. Enable rpc.rquotad in /etc/inetd.conf like so: rquotad/1 dgram rpc/udp wait root /usr/libexec/rpc.rquotad rpc.rquotad Now restart inetd: &prompt.root; kill -HUP `cat /var/run/inetd.pid` Creating CDs CDROMscreating Contributed by Mike Meyer mwm@mired.org, April 2001. Introduction CDs have a number of features that differentiate them from conventional disks. Initially, they weren't writable by the user. They are designed so that they can be read continuously without delays to move the head between tracks. They are also much easier to transport between systems than similarly sized media were at the time. CDs do have tracks, but this refers to a section of data to be read continuously and not a physical property of the disk. To produce a CD on FreeBSD, you prepare the data files that are going to make up the tracks on the CD, then write the tracks to the CD. ISO 9660 filesystemsISO-9660 The ISO 9660 file system was designed to deal with these differences. It unfortunately codifies file system limits that were common then. Fortunately, it provides an extension mechanism that allows properly written CDs to exceed those limits while still working with systems that do not support those extensions. mkisofs The mkisofs program is used to produce a data file containing an ISO 9660 file system. It has options that support various extensions, and is described below. You can install it with the /usr/ports/sysutils/mkisofs port. CD burnerATAPI Which tool to use to burn the CD depends on whether your CD burner is ATAPI or something else. ATAPI CD burners use the burncd program that is part of the base system. SCSI and USB CD burners should use the cdrecord from the /usr/ports/sysutils/cdrecord port. mkisofs mkisofs produces an ISO 9660 file system that is an image of a directory tree in the Unix file system name space. The simplest usage is: &prompt.root; mkisofs imagefile.iso /path/to/tree filesystemsISO-9660 This command will create an imagefile containing an ISO 9660 file system that is a copy of the tree at /path/to/tree. In the process, it will map the file names to names that fit the limitations of the standard ISO 9660 file system, and will exclude files that have names uncharacteristic of ISO file systems. Read &man.mkisofs.8; for details of this process, and options that can be used to control it. filesystemsHFS filesystemsJoliet A number of options are available to overcome those restrictions. In particular, enables the Rock Ridge extensions common to Unix systems, enables Joliet extensions used by Microsoft systems, and can be used to create HFS file systems used by Macs. Read &man.mkisofs.8; for more information on the last two. For CDs that are going to be used only on FreeBSD systems, can be used to disable all filename restrictions. When used with , it produces a file system image that is identical to the FreeBSD tree you started from, though it may violate the ISO 9660 standard in a number of ways. CDROMscreating bootable The last option of general use is . This is used to specify the location of the boot image for use in producing an El Torito bootable CD. This option takes an argument which is the path to a boot image from the top of the tree being written to the CD. So, given that /tmp/myboot holds a bootable FreeBSD system with the boot image in /tmp/myboot/boot/cdboot, you could produce the image of an ISO 9660 file system in /tmp/bootable.iso like so: &prompt.root; mkisofs boot/cdboot /tmp/bootable.iso /tmp/myboot Having done that, if you have vn configured in your kernel, you can mount the file system with: &prompt.root; vnconfig vn0c /tmp/bootable.iso &prompt.root; mount cd9660 /dev/vn0c /mnt At which point you can verify that /mnt and /tmp/myboot are identical. There are many other options you can use with mkisofs to fine-tune its behavior. See &man.mkisofs.8; for details. burncd CDROMsburning If you have an ATAPI CD burner, you can use the burncd command to burn an ISO image onto a CD. burncd is part of the base system, installed as /usr/sbin/burncd. Usage is very simple, as it has few options: &prompt.root; burncd cddevice data imagefile.iso fixate Will burn a copy of imagefile.iso on cddevice. The default device is /dev/acd0. See &man.burncd.8; for options to set the write speed, eject the CD after burning, and write audio data. cdrecord If you do not have an ATAPI CD burner, you will have to use cdrecord to burn your CDs. cdrecord is not part of the base system; you must install it from either the port at /usr/ports/sysutils/cdrecord or the appropriate package. Changes to the base system can cause binary versions of this program to fail, possibly resulting in a coaster. You should therefore either upgrade the port when you upgrade your system, or if you are tracking -stable, upgrade the port when a new version becomes available. While cdrecord has many options, basic usage is even simpler than burncd. Burning an ISO 9660 image is done with: &prompt.root; cdrecord device imagefile.iso The tricky part of using cdrecord is finding the to use. To find the proper setting, use the flag of cdrecord, which might produce results like this: CDROMsburning &prompt.root; cdrecord Cdrecord 1.9 (i386-unknown-freebsd4.2) Copyright (C) 1995-2000 Jörg Schilling Using libscg version 'schily-0.1' scsibus0: 0,0,0 0) 'SEAGATE ' 'ST39236LW ' '0004' Disk 0,1,0 1) 'SEAGATE ' 'ST39173W ' '5958' Disk 0,2,0 2) * 0,3,0 3) 'iomega ' 'jaz 1GB ' 'J.86' Removable Disk 0,4,0 4) 'NEC ' 'CD-ROM DRIVE:466' '1.26' Removable CD-ROM 0,5,0 5) * 0,6,0 6) * 0,7,0 7) * scsibus1: 1,0,0 100) * 1,1,0 101) * 1,2,0 102) * 1,3,0 103) * 1,4,0 104) * 1,5,0 105) 'YAMAHA ' 'CRW4260 ' '1.0q' Removable CD-ROM 1,6,0 106) 'ARTEC ' 'AM12S ' '1.06' Scanner - 1,7,0 107) * - + 1,7,0 107) * This lists the appropriate value for the devices on the list. Locate your CD burner, and use the three numbers separated by commas as the value for . In this case, the CRW device is 1,5,0, so the appropriate input would be =1,5,0. There are easier ways to specify this value; see &man.cdrecord.1; for details. That is also the place to look for information on writing audio tracks, controlling the speed, and other things. diff --git a/en_US.ISO8859-1/books/handbook/kerneldebug/chapter.sgml b/en_US.ISO8859-1/books/handbook/kerneldebug/chapter.sgml index b08a119cb9..d56fa5aaba 100644 --- a/en_US.ISO8859-1/books/handbook/kerneldebug/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/kerneldebug/chapter.sgml @@ -1,639 +1,638 @@ Kernel Debugging Contributed by &a.paul; and &a.joerg; Debugging a Kernel Crash Dump with <command>gdb</command> Here are some instructions for getting kernel debugging working on a crash dump. They assume that you have enough swap space for a crash dump. If you have multiple swap partitions and the first one is too small to hold the dump, you can configure your kernel to use an alternate dump device (in the config kernel line), or you can specify an alternate using the &man.dumpon.8; command. The best way to use &man.dumpon.8; is to set the dumpdev variable in /etc/rc.conf. Typically you want to specify one of the swap devices specified in /etc/fstab. Dumps to non-swap devices, tapes for example, are currently not supported. Config your kernel using config . See Kernel Configuration for details on configuring the FreeBSD kernel. Use the &man.dumpon.8; command to tell the kernel where to dump to (note that this will have to be done after configuring the partition in question as swap space via &man.swapon.8;). This is normally arranged via /etc/rc.conf and /etc/rc. Alternatively, you can hard-code the dump device via the dump clause in the config line of your kernel config file. This is deprecated and should be used only if you want a crash dump from a kernel that crashes during booting. In the following, the term gdb refers to the debugger gdb run in kernel debug mode. This can be accomplished by starting the gdb with the option . In kernel debug mode, gdb changes its prompt to (kgdb). If you are using FreeBSD 3 or earlier, you should make a stripped copy of the debug kernel, rather than installing the large debug kernel itself: &prompt.root; cp kernel kernel.debug &prompt.root; strip -g kernel This stage isn't necessary, but it is recommended. (In FreeBSD 4 and later releases this step is performed automatically at the end of the kernel make process.) When the kernel has been stripped, either automatically or by using the commands above, you may install it as usual by typing make install. Note that older releases of FreeBSD (up to but not including 3.1) used a.out kernels by default, which must have their symbol tables permanently resident in physical memory. With the larger symbol table in an unstripped debug kernel, this is wasteful. Recent FreeBSD releases use ELF kernels where this is no longer a problem. If you are testing a new kernel, for example by typing the new kernel's name at the boot prompt, but need to boot a different one in order to get your system up and running again, boot it only into single user state using the flag at the boot prompt, and then perform the following steps: &prompt.root; fsck -p &prompt.root; mount -a -t ufs # so your file system for /var/crash is writable &prompt.root; savecore -N /kernel.panicked /var/crash &prompt.root; exit # ...to multi-user This instructs &man.savecore.8; to use another kernel for symbol name extraction. It would otherwise default to the currently running kernel and most likely not do anything at all since the crash dump and the kernel symbols differ. Now, after a crash dump, go to /sys/compile/WHATEVER and run gdb . From gdb do: symbol-file kernel.debug exec-file /var/crash/kernel.0 core-file /var/crash/vmcore.0 and voila, you can debug the crash dump using the kernel sources just like you can for any other program. Here is a script log of a gdb session illustrating the procedure. Long lines have been folded to improve readability, and the lines are numbered for reference. Despite this, it is a real-world error trace taken during the development of the pcvt console driver. 1:Script started on Fri Dec 30 23:15:22 1994 2:&prompt.root; cd /sys/compile/URIAH 3:&prompt.root; gdb -k kernel /var/crash/vmcore.1 4:Reading symbol data from /usr/src/sys/compile/URIAH/kernel ...done. 5:IdlePTD 1f3000 6:panic: because you said to! 7:current pcb at 1e3f70 8:Reading in symbols for ../../i386/i386/machdep.c...done. 9:(kgdb) where 10:#0 boot (arghowto=256) (../../i386/i386/machdep.c line 767) 11:#1 0xf0115159 in panic () 12:#2 0xf01955bd in diediedie () (../../i386/i386/machdep.c line 698) 13:#3 0xf010185e in db_fncall () 14:#4 0xf0101586 in db_command (-266509132, -266509516, -267381073) 15:#5 0xf0101711 in db_command_loop () 16:#6 0xf01040a0 in db_trap () 17:#7 0xf0192976 in kdb_trap (12, 0, -272630436, -266743723) 18:#8 0xf019d2eb in trap_fatal (...) 19:#9 0xf019ce60 in trap_pfault (...) 20:#10 0xf019cb2f in trap (...) 21:#11 0xf01932a1 in exception:calltrap () 22:#12 0xf0191503 in cnopen (...) 23:#13 0xf0132c34 in spec_open () 24:#14 0xf012d014 in vn_open () 25:#15 0xf012a183 in open () 26:#16 0xf019d4eb in syscall (...) 27:(kgdb) up 10 28:Reading in symbols for ../../i386/i386/trap.c...done. 29:#10 0xf019cb2f in trap (frame={tf_es = -260440048, tf_ds = 16, tf_\ 30:edi = 3072, tf_esi = -266445372, tf_ebp = -272630356, tf_isp = -27\ 31:2630396, tf_ebx = -266427884, tf_edx = 12, tf_ecx = -266427884, tf\ 32:_eax = 64772224, tf_trapno = 12, tf_err = -272695296, tf_eip = -26\ 33:6672343, tf_cs = -266469368, tf_eflags = 66066, tf_esp = 3072, tf_\ 34:ss = -266427884}) (../../i386/i386/trap.c line 283) 35:283 (void) trap_pfault(&frame, FALSE); 36:(kgdb) frame frame->tf_ebp frame->tf_eip 37:Reading in symbols for ../../i386/isa/pcvt/pcvt_drv.c...done. 38:#0 0xf01ae729 in pcopen (dev=3072, flag=3, mode=8192, p=(struct p\ 39:roc *) 0xf07c0c00) (../../i386/isa/pcvt/pcvt_drv.c line 403) 40:403 return ((*linesw[tp->t_line].l_open)(dev, tp)); 41:(kgdb) list 42:398 43:399 tp->t_state |= TS_CARR_ON; 44:400 tp->t_cflag |= CLOCAL; /* cannot be a modem (:-) */ 45:401 46:402 #if PCVT_NETBSD || (PCVT_FREEBSD >= 200) 47:403 return ((*linesw[tp->t_line].l_open)(dev, tp)); 48:404 #else 49:405 return ((*linesw[tp->t_line].l_open)(dev, tp, flag)); 50:406 #endif /* PCVT_NETBSD || (PCVT_FREEBSD >= 200) */ 51:407 } 52:(kgdb) print tp 53:Reading in symbols for ../../i386/i386/cons.c...done. 54:$1 = (struct tty *) 0x1bae 55:(kgdb) print tp->t_line 56:$2 = 1767990816 57:(kgdb) up 58:#1 0xf0191503 in cnopen (dev=0x00000000, flag=3, mode=8192, p=(st\ 59:ruct proc *) 0xf07c0c00) (../../i386/i386/cons.c line 126) 60: return ((*cdevsw[major(dev)].d_open)(dev, flag, mode, p)); 61:(kgdb) up 62:#2 0xf0132c34 in spec_open () 63:(kgdb) up 64:#3 0xf012d014 in vn_open () 65:(kgdb) up 66:#4 0xf012a183 in open () 67:(kgdb) up 68:#5 0xf019d4eb in syscall (frame={tf_es = 39, tf_ds = 39, tf_edi =\ 69: 2158592, tf_esi = 0, tf_ebp = -272638436, tf_isp = -272629788, tf\ 70:_ebx = 7086, tf_edx = 1, tf_ecx = 0, tf_eax = 5, tf_trapno = 582, \ 71:tf_err = 582, tf_eip = 75749, tf_cs = 31, tf_eflags = 582, tf_esp \ 72:= -272638456, tf_ss = 39}) (../../i386/i386/trap.c line 673) 73:673 error = (*callp->sy_call)(p, args, rval); 74:(kgdb) up 75:Initial frame selected; you cannot go up. 76:(kgdb) quit 77:&prompt.root; exit 78:exit 79: 80:Script done on Fri Dec 30 23:18:04 1994 Comments to the above script: line 6: This is a dump taken from within DDB (see below), hence the panic comment because you said to!, and a rather long stack trace; the initial reason for going into DDB has been a page fault trap though. line 20: This is the location of function trap() in the stack trace. line 36: Force usage of a new stack frame; this is no longer necessary now. The stack frames are supposed to point to the right locations now, even in case of a trap. From looking at the code in source line 403, there is a high probability that either the pointer access for tp was messed up, or the array access was out of bounds. line 52: The pointer looks suspicious, but happens to be a valid address. line 56: However, it obviously points to garbage, so we have found our error! (For those unfamiliar with that particular piece of code: tp->t_line refers to the line discipline of the console device here, which must be a rather small integer number.) Debugging a Crash Dump with DDD Examining a kernel crash dump with a graphical debugger like ddd is also possible. Add the option to the ddd command line you would use normally. For example; &prompt.root; ddd -k /var/crash/kernel.0 /var/crash/vmcore.0 You should then be able to go about looking at the crash dump using ddd's graphical interface. Post-Mortem Analysis of a Dump What do you do if a kernel dumped core but you did not expect it, and it is therefore not compiled using config -g? Not everything is lost here. Do not panic! Of course, you still need to enable crash dumps. See above on the options you have to specify in order to do this. Go to your kernel config directory (/usr/src/sys/arch/conf) and edit your configuration file. Uncomment (or add, if it does not exist) the following line makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Rebuild the kernel. Due to the time stamp change on the Makefile, there will be some other object files rebuild, for example trap.o. With a bit of luck, the added option will not change anything for the generated code, so you will finally get a new kernel with similar code to the faulting one but some debugging symbols. You should at least verify the old and new sizes with the &man.size.1; command. If there is a mismatch, you probably need to give up here. Go and examine the dump as described above. The debugging symbols might be incomplete for some places, as can be seen in the stack trace in the example above where some functions are displayed without line numbers and argument lists. If you need more debugging symbols, remove the appropriate object files and repeat the gdb session until you know enough. All this is not guaranteed to work, but it will do it fine in most cases. On-Line Kernel Debugging Using DDB While gdb as an off-line debugger provides a very high level of user interface, there are some things it cannot do. The most important ones being breakpointing and single-stepping kernel code. If you need to do low-level debugging on your kernel, there is an on-line debugger available called DDB. It allows to setting breakpoints, single-stepping kernel functions, examining and changing kernel variables, etc. However, it cannot access kernel source files, and only has access to the global and static symbols, not to the full debug information like gdb. To configure your kernel to include DDB, add the option line options DDB to your config file, and rebuild. (See Kernel Configuration for details on configuring the FreeBSD kernel. If you have an older version of the boot blocks, your debugger symbols might not be loaded at all. Update the boot blocks; the recent ones load the DDB symbols automagically.) Once your DDB kernel is running, there are several ways to enter DDB. The first, and earliest way is to type the boot flag right at the boot prompt. The kernel will start up in debug mode and enter DDB prior to any device probing. Hence you can even debug the device probe/attach functions. The second scenario is a hot-key on the keyboard, usually Ctrl-Alt-ESC. For syscons, this can be remapped; some of the distributed maps do this, so watch out. There is an option available for serial consoles that allows the use of a serial line BREAK on the console line to enter DDB (options BREAK_TO_DEBUGGER in the kernel config file). It is not the default since there are a lot of crappy serial adapters around that gratuitously generate a BREAK condition, for example when pulling the cable. The third way is that any panic condition will branch to DDB if the kernel is configured to use it. For this reason, it is not wise to configure a kernel with DDB for a machine running unattended. The DDB commands roughly resemble some gdb commands. The first thing you probably need to do is to set a breakpoint: b function-name b address Numbers are taken hexadecimal by default, but to make them distinct from symbol names; hexadecimal numbers starting with the letters a-f need to be preceded with 0x (this is optional for other numbers). Simple expressions are allowed, for example: function-name + 0x103. To continue the operation of an interrupted kernel, simply type: c To get a stack trace, use: trace Note that when entering DDB via a hot-key, the kernel is currently servicing an interrupt, so the stack trace might be not of much use for you. If you want to remove a breakpoint, use del del address-expression The first form will be accepted immediately after a breakpoint hit, and deletes the current breakpoint. The second form can remove any breakpoint, but you need to specify the exact address; this can be obtained from: show b To single-step the kernel, try: s This will step into functions, but you can make DDB trace them until the matching return statement is reached by: n This is different from gdb's next statement; it is like gdb's finish. To examine data from memory, use (for example): x/wx 0xf0133fe0,40 x/hd db_symtab_space x/bc termbuf,10 x/s stringbuf for word/halfword/byte access, and hexadecimal/decimal/character/ string display. The number after the comma is the object count. To display the next 0x10 items, simply use: x ,10 Similarly, use x/ia foofunc,10 to disassemble the first 0x10 instructions of foofunc, and display them along with their offset from the beginning of foofunc. To modify memory, use the write command: w/b termbuf 0xa 0xb 0 w/w 0xf0010030 0 0 The command modifier (b/h/w) specifies the size of the data to be written, the first following expression is the address to write to and the remainder is interpreted as data to write to successive memory locations. If you need to know the current registers, use: show reg Alternatively, you can display a single register value by e.g. p $eax and modify it by: set $eax new-value Should you need to call some kernel functions from DDB, simply say: call func(arg1, arg2, ...) The return value will be printed. For a &man.ps.1; style summary of all running processes, use: ps Now you have examined why your kernel failed, and you wish to reboot. Remember that, depending on the severity of previous malfunctioning, not all parts of the kernel might still be working as expected. Perform one of the following actions to shut down and reboot your system: panic This will cause your kernel to dump core and reboot, so you can later analyze the core on a higher level with gdb. This command usually must be followed by another continue statement. call boot(0) Which might be a good way to cleanly shut down the running system, sync() all disks, and finally reboot. As long as the disk and file system interfaces of the kernel are not damaged, this might be a good way for an almost clean shutdown. call cpu_reset() is the final way out of disaster and almost the same as hitting the Big Red Button. If you need a short command summary, simply type: help However, it is highly recommended to have a printed copy of the &man.ddb.4; manual page ready for a debugging session. Remember that it is hard to read the on-line manual while single-stepping the kernel. On-Line Kernel Debugging Using Remote GDB This feature has been supported since FreeBSD 2.2, and it is actually a very neat one. GDB has already supported remote debugging for a long time. This is done using a very simple protocol along a serial line. Unlike the other methods described above, you will need two machines for doing this. One is the host providing the debugging environment, including all the sources, and a copy of the kernel binary with all the symbols in it, and the other one is the target machine that simply runs a similar copy of the very same kernel (but stripped of the debugging information). You should configure the kernel in question with config -g, include into the configuration, and compile it as usual. This gives a large blurb of a binary, due to the debugging information. Copy this kernel to the target machine, strip the debugging symbols off with strip -x, and boot it using the boot option. Connect the serial line of the target machine that has "flags 080" set on its sio device to any serial line of the debugging host. Now, on the debugging machine, go to the compile directory of the target kernel, and start gdb: &prompt.user; gdb -k kernel GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.16 (i386-unknown-freebsd), Copyright 1996 Free Software Foundation, Inc... (kgdb) Initialize the remote debugging session (assuming the first serial port is being used) by: (kgdb) target remote /dev/cuaa0 Now, on the target host (the one that entered DDB right before even starting the device probe), type: Debugger("Boot flags requested debugger") Stopped at Debugger+0x35: movb $0, edata+0x51bc db> gdb DDB will respond with: Next trap will enter GDB remote protocol mode Every time you type gdb, the mode will be toggled between remote GDB and local DDB. In order to force a next trap immediately, simply type s (step). Your hosting GDB will now gain control over the target kernel: Remote debugging using /dev/cuaa0 Debugger (msg=0xf01b0383 "Boot flags requested debugger") at ../../i386/i386/db_interface.c:257 (kgdb) You can use this session almost as any other GDB session, including full access to the source, running it in gud-mode inside an Emacs window (which gives you an automatic source code display in another Emacs window) etc. Debugging Loadable Modules Using GDB When debugging a panic that occurred within a module, or using remote GDB against a machine that uses dynamic modules, you need to tell GDB how to obtain symbol information for those modules. First, you need to build the module(s) with debugging information: &prompt.root; cd /sys/modules/linux &prompt.root; make clean; make COPTS=-g If you are using remote GDB, you can run kldstat on the target machine to find out where the module was loaded: &prompt.root; kldstat Id Refs Address Size Name 1 4 0xc0100000 1c1678 kernel 2 1 0xc0a9e000 6000 linprocfs.ko 3 1 0xc0ad7000 2000 warp_saver.ko - 4 1 0xc0adc000 11000 linux.ko - + 4 1 0xc0adc000 11000 linux.ko If you are debugging a crash dump, you'll need to walk the linker_files list, starting at linker_files->tqh_first and following the link.tqe_next pointers until you find the entry with the filename you are looking for. The address member of that entry is the load address of the module. Next, you need to find out the offset of the text section within the module: &prompt.root; objdump --section-headers /sys/modules/linux/linux.ko | grep text 3 .rel.text 000016e0 000038e0 000038e0 000038e0 2**2 10 .text 00007f34 000062d0 000062d0 000062d0 2**2 The one you want is the .text section, section 10 in the above example. The fourth hexadecimal field (sixth field overall) is the offset of the text section within the file. Add this offset to the load address of the module to obtain the relocation address for the module's code. In our example, we get 0xc0adc000 + 0x62d0 = 0xc0ae22d0. Use the add-symbol-file command in GDB to tell the debugger about the module: (kgdb) add-symbol-file /sys/modules/linux/linux.ko 0xc0ae22d0 add symbol table from file "/sys/modules/linux/linux.ko" at text_addr = 0xc0ae22d0? (y or n) y Reading symbols from /sys/modules/linux/linux.ko...done. (kgdb) You should now have access to all the symbols in the module. Debugging a Console Driver Since you need a console driver to run DDB on, things are more complicated if the console driver itself is failing. You might remember the use of a serial console (either with modified boot blocks, or by specifying at the Boot: prompt), and hook up a standard terminal onto your first serial port. DDB works on any configured console driver, of course also on a serial console. diff --git a/en_US.ISO8859-1/books/handbook/kernelopts/chapter.sgml b/en_US.ISO8859-1/books/handbook/kernelopts/chapter.sgml index 1e91b5cae7..508e5e16cc 100644 --- a/en_US.ISO8859-1/books/handbook/kernelopts/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/kernelopts/chapter.sgml @@ -1,162 +1,160 @@ Adding New Kernel Configuration Options Contributed by &a.joerg; You should be familiar with the section about kernel configuration before reading here. What's a <emphasis>Kernel Option</emphasis>, Anyway? The use of kernel options is basically described in the kernel configuration section. There's also an explanation of historic and new-style options. The ultimate goal is to eventually turn all the supported options in the kernel into new-style ones, so for people who correctly did a make depend in their kernel compile directory after running &man.config.8;, the build process will automatically pick up modified options, and only recompile those files where it is necessary. Wiping out the old compile directory on each run of &man.config.8; as it is still done now can then be eliminated again. Basically, a kernel option is nothing else than the definition of a C preprocessor macro for the kernel compilation process. To make the build truly optional, the corresponding part of the kernel source (or kernel .h file) must be written with the option concept in mind, i.e., the default can be overridden by the config option. This is usually done with something like: #ifndef THIS_OPTION #define THIS_OPTION (some_default_value) #endif /* THIS_OPTION */ This way, an administrator mentioning another value for the option in his config file will take the default out of effect, and replace it with his new value. Clearly, the new value will be substituted into the source code during the preprocessor run, so it must be a valid C expression in whatever context the default value would have been used. It is also possible to create value-less options that simply enable or disable a particular piece of code by embracing it in #ifdef THAT_OPTION [your code here] #endif Simply mentioning THAT_OPTION in the config file (with or without any value) will then turn on the corresponding piece of code. People familiar with the C language will immediately recognize that everything could be counted as a config option where there is at least a single #ifdef referencing it... However, it's unlikely that many people would put options notyet,notdef in their config file, and then wonder why the kernel compilation falls over. :-) Clearly, using arbitrary names for the options makes it very hard to track their usage throughout the kernel source tree. That is the rationale behind the new-style option scheme, where each option goes into a separate .h file in the kernel compile directory, which is by convention named opt_foo.h. This way, the usual Makefile dependencies could be applied, and make can determine what needs to be recompiled once an option has been changed. The old-style option mechanism still has one advantage for local options or maybe experimental options that have a short anticipated lifetime: since it is easy to add a new #ifdef to the kernel source, this has already made it a kernel config option. In this case, the administrator using such an option is responsible himself for knowing about its implications (and maybe manually forcing the recompilation of parts of his kernel). Once the transition of all supported options has been done, &man.config.8; will warn whenever an unsupported option appears in the config file, but it will nevertheless include it into the kernel Makefile. Now What Do I Have to Do for it? First, edit sys/conf/options (or sys/<arch>/conf/options.<arch>, e. g. sys/i386/conf/options.i386), and select an opt_foo.h file where your new option would best go into. If there is already something that comes close to the purpose of the new option, pick this. For example, options modifying the overall behavior of the SCSI subsystem can go into opt_scsi.h. By default, simply mentioning an option in the appropriate option file, say FOO, implies its value will go into the corresponding file opt_foo.h. This can be overridden on the right-hand side of a rule by specifying another filename. If there is no opt_foo.h already available for the intended new option, invent a new name. Make it meaningful, and comment the new section in the options[.<arch>] file. &man.config.8; will automagically pick up the change, and create that file next time it is run. Most options should go in a header file by themselves.. Packing too many options into a single opt_foo.h will cause too many kernel files to be rebuilt when one of the options has been changed in the config file. Finally, find out which kernel files depend on the new option. Unless you have just invented your option, and it does not exist - anywhere yet, -&prompt.user; find /usr/src/sys -type f | xargs fgrep NEW_OPTION - + anywhere yet, &prompt.user; find /usr/src/sys -type f | xargs fgrep NEW_OPTION is your friend in finding them. Go and edit all those files, and add #include "opt_foo.h" on top before all the #include <xxx.h> stuff. This sequence is most important as the options could override defaults from the regular include files, if the defaults are of the form #ifndef NEW_OPTION #define NEW_OPTION (something) #endif in the regular header. Adding an option that overrides something in a system header file (i.e., a file sitting in /usr/include/sys/) is almost always a mistake. opt_foo.h cannot be included into those files since it would break the headers more seriously, but if it is not included, then places that include it may get an inconsistent value for the option. Yes, there are precedents for this right now, but that does not make them more correct. diff --git a/en_US.ISO8859-1/books/handbook/linuxemu/chapter.sgml b/en_US.ISO8859-1/books/handbook/linuxemu/chapter.sgml index daa4c89c55..0723bd83b7 100644 --- a/en_US.ISO8859-1/books/handbook/linuxemu/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/linuxemu/chapter.sgml @@ -1,2248 +1,2247 @@ Linux Binary Compatibility Restructured and parts updated by &a.jim;, 22 March 2000. Originally contributed by &a.handy; and &a.rich; Synopsis Linux binary compatibility binary compatibility Linux The following chapter will cover FreeBSD's Linux binary compatibility features, how to install it, and how it works. At this point, you may be asking yourself why exactly, does FreeBSD need to be able to run Linux binaries? The answer to that question is quite simple. Many companies and developers develop only for Linux, since it is the latest hot thing in the computing world. That leaves the rest of us FreeBSD users bugging these same companies and developers to put out native FreeBSD versions of their applications. The problem is, that most of these companies do not really realize how many people would use their product if there were FreeBSD versions too, and most continue to only develop for Linux. So what is a FreeBSD user to do? This is where the Linux binary compatibility of FreeBSD comes into play. In a nutshell, the compatibility allows FreeBSD users to run about 90% of all Linux applications without modification. This includes applications such as Star Office, the Linux version of Netscape, Adobe Acrobat, RealPlayer 5 and 7, VMWare, Oracle, WordPerfect, Doom, Quake, and more. It is also reported that in some situations, Linux binaries perform better on FreeBSD than they do under Linux. Linux /proc filesystem There are, however, some Linux-specific operating system features that are not supported under FreeBSD. Linux binaries will not work on FreeBSD if they overly use the Linux /proc filesystem (which is different from FreeBSD's /proc filesystem), or i386-specific calls, such as enabling virtual 8086 mode. For information on installing the Linux binary compatibility mode, see the next section. Installation With the advent of 3.0-RELEASE, it is no longer necessary to specify options LINUX or options COMPAT_LINUX in your kernel configuration. KLD (kernel loadable object) The Linux binary compatibility is now done via a KLD object (Kernel LoaDable object), so it can be installed on-the-fly without having to reboot. You will, however, need to have the following in /etc/rc.conf: linux_enable=YES This, in turn, triggers the following action in /etc/rc.i386: # Start the Linux binary compatibility if requested. # case ${linux_enable} in [Yy][Ee][Ss]) echo -n ' linux'; linux > /dev/null 2>&1 ;; esac If you wish to verify that the KLD is loaded, kldstat will do that: &prompt.user; kldstat Id Refs Address Size Name 1 2 0xc0100000 16bdb8 kernel 7 1 0xc24db000 d000 linux.ko kernel options LINUX If for some reason you do not want to or cannot load the KLD, then you may statically link the binary compatibility in the kernel by adding options LINUX to your kernel configuration file. Then install your new kernel as described in the kernel configuration section of this handbook. Installing Linux Runtime Libraries Linux installing Linux libraries This can be done one of two ways, either by using the linux_base port, or by installing them manually. Installing using the linux_base port ports collection This is by far the easiest method to use when installing the runtime libraries. It is just like installing any other port from the ports collection. Simply do the following: &prompt.root; cd /usr/ports/emulators/linux_base &prompt.root; make install distclean You should now have working Linux binary compatibility. Some programs may complain about incorrect minor versions of the system libraries. In general, however, this does not seem to be a problem. Installing libraries manually If you do not have the ports collection installed, you can install the libraries by hand instead. You will need the Linux shared libraries that the program depends on and the runtime linker. Also, you will need to create a shadow root directory, /compat/linux, for Linux libraries on your FreeBSD system. Any shared libraries opened by Linux programs run under FreeBSD will look in this tree first. So, if a Linux program loads, for example, /lib/libc.so, FreeBSD will first try to open /compat/linux/lib/libc.so, and if that does not exist, it will then try /lib/libc.so. Shared libraries should be installed in the shadow tree /compat/linux/lib rather than the paths that the Linux ld.so reports. Generally, you will need to look for the shared libraries that Linux binaries depend on only the first few times that you install a Linux program on your FreeBSD system. After a while, you will have a sufficient set of Linux shared libraries on your system to be able to run newly imported Linux binaries without any extra work. How to install additional shared libraries shared libraries What if you install the linux_base port and your application still complains about missing shared libraries? How do you know which shared libraries Linux binaries need, and where to get them? Basically, there are 2 possibilities (when following these instructions you will need to be root on your FreeBSD system). If you have access to a Linux system, see what shared libraries the application needs, and copy them to your FreeBSD system. Look at the following example: Let us assume you used FTP to get the Linux binary of Doom, and put it on a Linux system you have access to. You then can check which shared libraries it needs by running ldd linuxdoom, like so: &prompt.user; ldd linuxdoom libXt.so.3 (DLL Jump 3.1) => /usr/X11/lib/libXt.so.3.1.0 libX11.so.3 (DLL Jump 3.1) => /usr/X11/lib/libX11.so.3.1.0 libc.so.4 (DLL Jump 4.5pl26) => /lib/libc.so.4.6.29 symbolic links You would need to get all the files from the last column, and put them under /compat/linux, with the names in the first column as symbolic links pointing to them. This means you eventually have these files on your FreeBSD system: /compat/linux/usr/X11/lib/libXt.so.3.1.0 /compat/linux/usr/X11/lib/libXt.so.3 -> libXt.so.3.1.0 /compat/linux/usr/X11/lib/libX11.so.3.1.0 /compat/linux/usr/X11/lib/libX11.so.3 -> libX11.so.3.1.0 /compat/linux/lib/libc.so.4.6.29 /compat/linux/lib/libc.so.4 -> libc.so.4.6.29
Note that if you already have a Linux shared library with a matching major revision number to the first column of the ldd output, you will not need to copy the file named in the last column to your system, the one you already have should work. It is advisable to copy the shared library anyway if it is a newer version, though. You can remove the old one, as long as you make the symbolic link point to the new one. So, if you have these libraries on your system: /compat/linux/lib/libc.so.4.6.27 /compat/linux/lib/libc.so.4 -> libc.so.4.6.27 and you find a new binary that claims to require a later version according to the output of ldd: libc.so.4 (DLL Jump 4.5pl26) -> libc.so.4.6.29 If it is only one or two versions out of date in the in the trailing digit then do not worry about copying /lib/libc.so.4.6.29 too, because the program should work fine with the slightly older version. However, if you like, you can decide to replace the libc.so anyway, and that should leave you with: /compat/linux/lib/libc.so.4.6.29 /compat/linux/lib/libc.so.4 -> libc.so.4.6.29
The symbolic link mechanism is only needed for Linux binaries. The FreeBSD runtime linker takes care of looking for matching major revision numbers itself and you do not need to worry about it.
Installing Linux ELF binaries Linux ELF binaries ELF binaries sometimes require an extra step of branding. If you attempt to run an unbranded ELF binary, you will get an error message like the following; &prompt.user; ./my-linux-elf-binary ELF binary type not known Abort To help the FreeBSD kernel distinguish between a FreeBSD ELF binary from a Linux binary, use the &man.brandelf.1; utility. &prompt.user; brandelf -t Linux my-linux-elf-binary GNU toolchain The GNU toolchain now places the appropriate branding information into ELF binaries automatically, so you this step should become increasingly more rare in the future. Configuring the host name resolver If DNS does not work or you get this message: resolv+: "bind" is an invalid keyword resolv+: "hosts" is an invalid keyword You will need to configure a /compat/linux/etc/host.conf file containing: order hosts, bind multi on The order here specifies that /etc/hosts is searched first and DNS is searched second. When /compat/linux/etc/host.conf is not installed, linux applications find FreeBSD's /etc/host.conf and complain about the incompatible FreeBSD syntax. You should remove bind if you have not configured a name server using the /etc/resolv.conf file. Installing Mathematica Updated for Mathematica version 4.x by &a.murray and merged with work by Bojan Bistrovic bojanb@physics.odu.edu. applications Mathematica This document describes the process of installing the Linux version of Mathematica 4.X onto a FreeBSD system. The Linux version of Mathematica runs perfectly under FreeBSD however the binaries shipped by Wolfram need to be branded so that FreeBSD knows to use the Linux ABI to execute them. The Linux version of Mathematica or Mathematica for Students can be ordered directly from Wolfram at http://www.wolfram.com/. Branding the Linux binaries The Linux binaries are located in the Unix directory of the Mathematica CDROM distributed by Wolfram. You need to copy this directory tree to your local hard drive so that you can brand the Linux binaries with &man.brandelf.1; before running the installer: &prompt.root; mount /cdrom &prompt.root; cp -rp /cdrom/Unix/ /localdir/ &prompt.root; brandelf -t Linux /localdir/Files/SystemFiles/Kernel/Binaries/Linux/* &prompt.root; brandelf -t Linux /localdir/Files/SystemFiles/FrontEnd/Binaries/Linux/* &prompt.root; brandelf -t Linux /localdir/Files/SystemFiles/Installation/Binaries/Linux/* &prompt.root; brandelf -t Linux /localdir/Files/SystemFiles/Graphics/Binaries/Linux/* &prompt.root; brandelf -t Linux /localdir/Files/SystemFiles/Converters/Binaries/Linux/* &prompt.root; brandelf -t Linux /localdir/Files/SystemFiles/LicenseManager/Binaries/Linux/mathlm &prompt.root; cd /localdir/Installers/Linux/ &prompt.root; ./MathInstaller Alternatively, you can simply set the default ELF brand to Linux for all unbranded binaries with the command: - &prompt.root; sysctl -w kern.fallback_elf_brand=3 - + &prompt.root; sysctl -w kern.fallback_elf_brand=3 This will make FreeBSD assume that unbranded ELF binaries use the Linux ABI and so you should be able to run the installer straight from the CDROM. Obtaining your Mathematica Password Before you can run Mathematica you will have to obtain a password from Wolfram that corresponds to your machine ID. Ethernet MAC address Once you have installed the Linux compatibility runtime libraries and unpacked Mathematica you can obtain the machine ID by running the program mathinfo in the Install directory. This machine ID is based solely on the MAC address of your first ethernet card. &prompt.root; cd /localdir/Files/SystemFiles/Installation/Binaries/Linux &prompt.root; mathinfo disco.example.com 7115-70839-20412 When you register with Wolfram, either by email, phone or fax, you will give them the machine ID and they will respond with a corresponding password consisting of groups of numbers. You can then enter this information when you attempt to run Mathematica for the first time exactly as you would for any other Mathematica platform. Running the Mathematica front end over a network Mathematica uses some special fonts to display characters not present in any of the standard font sets (integrals, sums, greek letters, etc.). The X protocol requires these fonts to be install locally. This means you will have to copy these fonts from the CDROM or from a host with Mathematica installed to your local machine. These fonts are normally stored in /cdrom/Unix/Files/SystemFiles/Fonts on the CDROM, or /usr/local/mathematica/SystemFiles/Fonts on your hard drive. The actual fonts are in the subdirectories Type1 and X. There are several ways to use them, as described below. The first way is to copy them into one of the existing font directories in /usr/X11R6/lib/X11/fonts. This will require editing the fonts.dir file, adding the font names to it, and changing the number of fonts on the first line. Alternatively, you should also just be able to run mkfontdir in the directory you have copied them to. The second way to do this is to copy the directories to /usr/X11R6/lib/X11/fonts: &prompt.root; cd /usr/X11R6/lib/X11/fonts &prompt.root; mkdir X &prompt.root; mkdir MathType1 &prompt.root; cd /cdrom/Unix/Files/SystemFiles/Fonts &prompt.root; cp X/* /usr/X11R6/lib/X11/fonts/X &prompt.root; cp Type1/* /usr/X11R6/lib/X11/fonts/MathType1 &prompt.root; cd /usr/X11R6/lib/X11/fonts/X &prompt.root; mkfontdir &prompt.root; cd ../MathType1 &prompt.root; mkfontdir Now add the new font directories to your font path: &prompt.root; xset fp+ /usr/X11R6/lib/X11/fonts/X &prompt.root; xset fp+ /usr/X11R6/lib/X11/fonts/MathType1 &prompt.root; xset fp rehash If you are using the XFree86 server, you can have these font directories loaded automatically by adding them to your XF86Config file. fonts If you do not already have a directory called /usr/X11R6/lib/X11/fonts/Type1, you can change the name of the MathType1 directory in the example above to Type1. Installing Oracle Contributed by Marcel Moolenaar marcel@cup.hp.com applications Oracle Preface This document describes the process of installing Oracle 8.0.5 and Oracle 8.0.5.1 Enterprise Edition for Linux onto a FreeBSD machine Installing the Linux environment Make sure you have both linux_base and linux_devtools from the ports collection installed. These ports are added to the collection after the release of FreeBSD 3.2. If you are using FreeBSD 3.2 or an older version for that matter, update your ports collection. You may want to consider updating your FreeBSD version too. If you run into difficulties with linux_base-6.1 or linux_devtools-6.1 you may have to use version 5.2 of these packages. If you want to run the intelligent agent, you'll also need to install the Red Hat TCL package: tcl-8.0.3-20.i386.rpm. The general command for installing packages with the official RPM port is : &prompt.root; rpm -i --ignoreos --root /compat/linux --dbpath /var/lib/rpm package Installation of the package should not generate any errors. Creating the Oracle environment Before you can install Oracle, you need to set up a proper environment. This document only describes what to do specially to run Oracle for Linux on FreeBSD, not what has been described in the Oracle installation guide. Kernel Tuning kernel tuning As described in the Oracle installation guide, you need to set the maximum size of shared memory. Don't use SHMMAX under FreeBSD. SHMMAX is merely calculated out of SHMMAXPGS and PGSIZE. Therefore define SHMMAXPGS. All other options can be used as described in the guide. For example: options SHMMAXPGS=10000 options SHMMNI=100 options SHMSEG=10 options SEMMNS=200 options SEMMNI=70 options SEMMSL=61 Set these options to suit your intended use of Oracle. Also, make sure you have the following options in your kernel config-file: options SYSVSHM #SysV shared memory options SYSVSEM #SysV semaphores options SYSVMSG #SysV interprocess communication Oracle account Create an Oracle account just as you would create any other account. The Oracle account is special only that you need to give it a Linux shell. Add /compat/linux/bin/bash to /etc/shells and set the shell for the Oracle account to /compat/linux/bin/bash. Environment Besides the normal Oracle variables, such as ORACLE_HOME and ORACLE_SID you must set the following environment variables: Variable Value LD_LIBRARY_PATH $ORACLE_HOME/lib CLASSPATH $ORACLE_HOME/jdbc/lib/classes111.zip PATH /compat/linux/bin /compat/linux/sbin /compat/linux/usr/bin /compat/linux/usr/sbin /bin /sbin /usr/bin /usr/sbin /usr/local/bin $ORACLE_HOME/bin It is advised to set all the environment variables in .profile. A complete example is: ORACLE_BASE=/oracle; export ORACLE_BASE ORACLE_HOME=/oracle; export ORACLE_HOME LD_LIBRARY_PATH=$ORACLE_HOME/lib export LD_LIBRARY_PATH ORACLE_SID=ORCL; export ORACLE_SID ORACLE_TERM=386x; export ORACLE_TERM CLASSPATH=$ORACLE_HOME/jdbc/lib/classes111.zip export CLASSPATH PATH=/compat/linux/bin:/compat/linux/sbin:/compat/linux/usr/bin:/compat/linux/usr/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:$ORACLE_HOME/bin export PATH Installing Oracle Due to a slight inconsistency in the Linux emulator, you need to create a directory named .oracle in /var/tmp before you start the installer. Either make it world writable or let it be owner by the oracle user. You should be able to install Oracle without any problems. If you have problems, check your Oracle distribution and/or configuration first! After you have installed Oracle, apply the patches described in the next two subsections. A frequent problem is that the TCP protocol adapter is not installed right. As a consequence, you cannot start any TCP listeners. The following actions help solve this problem: &prompt.root; cd $ORACLE_HOME/network/lib &prompt.root; make -f ins_network.mk ntcontab.o &prompt.root; cd $ORACLE_HOME/lib &prompt.root; ar r libnetwork.a ntcontab.o &prompt.root; cd $ORACLE_HOME/network/lib &prompt.root; make -f ins_network.mk install Don't forget to run root.sh again! Patching root.sh When installing Oracle, some actions, which need to be performed as root, are recorded in a shell script called root.sh. root.sh is written in the orainst directory. Apply the following patch to root.sh, to have it use to proper location of chown or alternatively run the script under a Linux native shell. *** orainst/root.sh.orig Tue Oct 6 21:57:33 1998 --- orainst/root.sh Mon Dec 28 15:58:53 1998 *************** *** 31,37 **** # This is the default value for CHOWN # It will redefined later in this script for those ports # which have it conditionally defined in ss_install.h ! CHOWN=/bin/chown # # Define variables to be used in this script --- 31,37 ---- # This is the default value for CHOWN # It will redefined later in this script for those ports # which have it conditionally defined in ss_install.h ! CHOWN=/usr/sbin/chown # # Define variables to be used in this script When you don't install Oracle from CD, you can patch the source for root.sh. It is called rthd.sh and is located in the orainst directory in the source tree. Patching genclntsh The script genclntsh is used to create a single shared client library. It is used when building the demos. Apply the following patch to comment out the definition of PATH: *** bin/genclntsh.orig Wed Sep 30 07:37:19 1998 --- bin/genclntsh Tue Dec 22 15:36:49 1998 *************** *** 32,38 **** # # Explicit path to ensure that we're using the correct commands #PATH=/usr/bin:/usr/ccs/bin export PATH ! PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin export PATH # # each product MUST provide a $PRODUCT/admin/shrept.lst --- 32,38 ---- # # Explicit path to ensure that we're using the correct commands #PATH=/usr/bin:/usr/ccs/bin export PATH ! #PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin export PATH # # each product MUST provide a $PRODUCT/admin/shrept.lst Running Oracle When you have followed the instructions, you should be able to run Oracle as if it was run on Linux itself. Installing SAP R/3 (4.6B - IDES) Contributed by Holger Kipp holger.kipp@alogis.com Converted to SGML by &a.logo; applications SAP R/3 Preface This document describes a possible way of installing a SAP R/3 4.6B IDES-System with Oracle 8.0.5 for Linux onto a FreeBSD 4.3 machine, including the installation of FreeBSD 4.3 stable and Oracle 8.0.5. Even though this document tries to describe all important steps in a greater detail, it is not intended as a replacement for the Oracle and SAP R/3 installation guides. Please see the documentation that comes with the SAP R/3 Linux edition for SAP- and Oracle-specific questions, as well as resources from Oracle and SAP OSS. Software The following CD-ROMs have been used for SAP-installation: Name Number Description KERNEL 51009113 SAP Kernel Oracle / Installation / AIX, Linux, Solaris RDBMS 51007558 Oracle / RDBMS 8.0.5.X / Linux EXPORT1 51010208 IDES / DB-Export / Disc 1 of 6 EXPORT2 51010209 IDES / DB-Export / Disc 2 of 6 EXPORT3 51010210 IDES / DB-Export / Disc3 of 6 EXPORT4 51010211 IDES / DB-Export / Disc4 of 6 EXPORT5 51010212 IDES / DB-Export / Disc5 of 6 EXPORT6 51010213 IDES / DB-Export / Disc6 of 6 Additionally, I used the Oracle 8 Server (Pre-production version 8.0.5 for Linux, Kernel Version 2.0.33) CD which is not really necessary, and of course FreeBSD 4.3 stable (it was only a few days past 4.3 RELEASE). SAP-Notes The following notes should be read before installing SAP R/3 or proved to be useful during installation: Number Title 0171356 SAP Software auf Linux: grundlegenden Anmerkungen 0201147 INST: 4.6C R/3 Inst. on UNIX - Oracle 0373203 Update / Migration Oracle 8.0.5 --> 8.0.6/8.1.6 LINUX 0072984 Release of Digital UNIX 4.0B for Oracle 0130581 R3SETUP step DIPGNTAB terminates 0144978 Your system has not been installed correctly 0162266 Questions and tips for R3SETUP on Windows NT / W2K Hardware-Requirements The following equipment is sufficient for a SAP R/3 System (4.6B): Component 4.6B 4.6C Processor 2 x 800MHz Pentium III 2 x 800MHz Pentium III Memory 1GB ECC 2GB ECC Hard Disc Space 50-60GB (IDES) 50-60GB (IDES) For use in production, Xeon-Processors with large cache, high-speed disc access (SCSI, RAID hardware controller), USV and ECC-RAM is recommended. The large amount of Hard disc space is due to the preconfigured IDES System, which creates 27 GB of database files during installation. Usually after installation it is then necessary to extend some tablespaces. I used a dual processor board with 2 800MHz Pentium III processors, Adaptec 29160 Ultra160 SCSI adapter (for accessing a 40/80 GB DLT tape drive and CD-ROM), Mylex AcelleRAID (2 channels, firmware 6.00-1-00 with 32MB RAM). To the Mylex Raid-controller are attached two 17GB hard discs (mirrored) and four 36GB hard discs (RAID level 5). Installation of FreeBSD 4.3 stable First I installed FreeBSD 4.3 stable. I did the default-installation via ftp. Installation via FTP Get the diskimages kern.flp and mfsroot.flp and put them on floppy disks (I got mine from ftp7.de.freebsd.org. Please choose the appropriate mirror). &prompt.root; dd if=kern.flp of=/dev/fd0 &prompt.root; dd if=mfsroot.flp of=/dev/fd0 Don't forget to use different disks for the two images :-), then boot from the floppy with the kern.flp-image on it and follow instructions. I used the following disk layout: Filesystem Size (1k-blocks) Size (GB) Mounted on /dev/da0s1a 1.016.303 1 / /dev/da0s1b 6 <swap> /dev/da0s1e 2.032.623 2 /var /dev/da0s1f 8.205.339 8 /usr /dev/da1s1e 45.734.361 45 /compat/linux/oracle /dev/da1s1f 2.032.623 2 /compat/linux/sapmnt /dev/da1s1g 2.032.623 2 /compat/linux/usr/sap I had to configure and initialise the two logical drives with the Mylex software beforehand. It is located on the board itself and can be started during the boot phase of the pc. Please note that this disk layout differs slightly from the SAP recommendations, as SAP suggests mounting the oracle-subdirectories (and some others) separately - I decided to just create them as real subdirectories for simplicity. Get the latest stable-sources For FreeBSD 4.3 stable onwards, it is quite easy to get the latest stable sources. With the older versions of FreeBSD, I had my own script located in /etc/cvsup. Setting up cvsup for FreeBSD 4.3 is quite easy. As user root do the following: &prompt.root; cp /etc/defaults/make.conf /etc/make.conf &prompt.root; vi /etc/make.conf The file /etc/make.conf requires the following entries to be active: SUP_UPDATE= yes SUP= /usr/local/bin/cvsup SUPFLAGS= -g -L 2 SUPHOST= cvsup8.FreeBSD.org SUPFILE= /usr/share/examples/cvsup/stable-supfile PORTSSUPFILE= /usr/share/examples/cvsup/ports-supfile DOCSUPFILE= /usr/share/examples/cvsup/doc-supfile Change the SUPHOST-value appropriately. The supfiles in /usr/share/examples/cvsup should be fine. If you don't want to load all the docfiles, leave the corresponding DOCSUPFILE-entry inactive. Starting cvsup to get the latest stable-sources is then very easy: &prompt.root; cd /usr/src &prompt.root; make update Make world and a new kernel The first thing to do is to install the sources. As user root, do the following: &prompt.root; cd /usr/src &prompt.root; make world If this goes through, one can then continue creating and configuring the new kernel. Usually this is where to customize the kernel configuration file. As the computer is named troubadix, the natural name for the config file also is troubadix: &prompt.root; cd /usr/src/sys/i386/conf &prompt.root; cp GENERIC TROUBADIX &prompt.root; vi TROUBADIX At this stage one can define the drivers to use and not to use, etc. See the appropriate documentation or have a look at file LINT for some additional explanations. One can then also include the parameters as described below Creating the new kernel then requires: &prompt.root; cd /usr/src/sys/i386/conf &prompt.root; config TROUBADIX &prompt.root; cd /usr/src/sys/compile/TROUBADIX &prompt.root; make depend &prompt.root; make &prompt.root; make install After make install finished successfully, one should reboot the computer to have the new kernel available. Installing the Linux environment I had some trouble downloading the required RPM-files (for 4.3 stable, 2nd May 2001), so you might try one of the following locations (if all the others fail and the following aren't out of date): ftp7.de.freebsd.org/pub/FreeBSD/distfiles/rpm ftp.redhat.com/pub/redhat/linux/6.1/en/os/i386/RedHat/RPMS Installing Linux base-system First the linux base-system needs to be installed (as root): &prompt.root; cd /usr/ports/emulators/linux_base &prompt.root; make package Installing Linux development Next, the linux development is needed: &prompt.root; cd /usr/ports/devel/linux_devtools &prompt.root; make package Installing necessary RPMs RPMs To start the R3SETUP-Program, pam support is needed. As this also requires some other packages, I ended up installing several packages. After that, pam still complained about a missing package, so I forced the installation and it worked. I wonder if the other packages are really needed or if it would have been sufficient to install the pam-package. Anyway, here is the list of packages I installed: cracklib-2.7-5.i386.rpm cracklib-dicts-2.7-5.i386.rpm pwdb-0.60-1.i386.rpm pam-0.68-7.i386.rpm I installed these packages with the following command: &prompt.root; rpm -i --ignoreos --root /compat/linux --dbpath /var/lib/rpm <package_name> except for the pam package, which I forced with &prompt.root; rpm -i --ignoreos --nodeps --root /compat/linux --dbpath /var/lib/rpm pam-0.68-7.i386.rpm For Oracle to run the intelligent agent, I also had to install the following RedHat TCL package (as is stated in the FreeBSD Handbook): tcl-8.0.5-30.i386.rpm (otherwise the relinking during Oracle install won't work). There are some other issues regarding relinking of Oracle, but that is a Oracle-Linux issue, not FreeBSD specific as far as I understand it. Creating the SAP/R3 environment Creating the necessary filesystems and mountpoints For a simple installation, it is sufficient to create the following filesystems: mountpoint size in GB /compat/linux/oracle 45 GB /compat/linux/sapmnt 2 GB /compat/linux/usr/sap 2 GB I also created some links, so FreeBSD will also find the correct path: &prompt.root; ln -s /compat/linux/oracle /oracle &prompt.root; ln -s /compat/linux/sapmnt /sapmnt &prompt.root; ln -s /compat/linux/usr/sap /usr/sap Creating users and directories SAP R/3 needs two users and three groups. The usernames depend on the SAP system id (SID) which consists of three letters. Some of these SIDs are reserved by SAP (for example SAP and NIX. For a complete list please see the SAP documentation). For the IDES installation I used IDS. We have therefore the following groups (group ids might differ, these are just the values I used with my installation): group id group name description 100 dba Data Base Administrator 101 sapsys SAP System 102 oper Data Base Operator For a default Oracle-Installation, only group dba is used. As oper-group, one also uses group dba (see Oracle- and SAP-documentation for further information). We also need the following users: user id username generic name group additional groups description 1000 idsadm <sid>adm sapsys oper SAP Administrator 1002 oraids ora<sid> dba oper DB Administrator Adding the users with adduser requires the following (please note shell and home directory) entries for SAP-Administrator: Name: idsadm <sid>adm Password: ****** Fullname: SAP IDES Administrator Uid: 1000 Gid: 101 (sapsys) Class: Groups: sapsys dba HOME: /home/idsadm /home/<sid>adm Shell: /bin/sh and for Database-Administrator: Name: oraids ora<sid> Password: ****** Fullname: Oracle IDES Administrator Uid: 1002 Gid: 100 (dba) Class: Groups: dba HOME: /oracle/IDS /oracle/<sid> Shell: /bin/sh This should also include group oper in case you are using both groups dba and oper. Creating directories These directories are usually created as separate filesystems. This depends entirely on your requirements. I choose to create them as simple directories, as they are all located on the same RAID 5 anyway: First we'll set owners and right of some directories (as user root): &prompt.root; chmod 775 /oracle &prompt.root; chmod 777 /sapmnt &prompt.root; chown root:dba /oracle &prompt.root; chown idsadm:sapsys /compat/linux/usr/sap &prompt.root; chmow 775 /compat/linux/usr/sap Second we'll create directories as user ora<sid>. These will all be subdirectories of /oracle/IDS: &prompt.root; su - oraids &prompt.root; mkdir mirrlogA mirrlogB origlogA origlogB &prompt.root; mkdir sapdata1 sapdata2 sapdata3 sapdata4 sapdata5 sapdata6 &prompt.root; mkdir saparch sapreorg &prompt.root; exit In the third step we create directories as user idsadm (<sid>adm): &prompt.root; su - idsadm &prompt.root; cd /usr/sap &prompt.root; mkdir IDS &prompt.root; mkdir trans &prompt.root; exit Entries in /etc/services SAP R/3 requires some entries in file /etc/services , which will not be set correctly during installation under FreeBSD. Please add the following entries (you need at least those entries corresponding to the instance number - in this case, 00. It'll do no harm adding all entries from 00 to 99 for dp, gw, sp and ms); sapdp00 3200/tcp # SAP Dispatcher. 3200 + Instance-Number sapgw00 3300/tcp # SAP Gateway. 3300 + Instance-Number sapsp00 3400/tcp # 3400 + Instance-Number sapms00 3500/tcp # 3500 + Instance-Number sapmsIDS 3600/tcp # SAP Message Server. 3600 + Instance-Number Necessary locales locale SAP requires at least two locales that aren't part of the default RedHat installation. SAP offers the required RPMs as download from their ftp-server (which is only accessible if you are a customer with OSS-access). See note 0171356 for a list of RPMs you need. It is also possible to just create appropriate links (for example from de_DE and en_US ), but I wouldn't recommend this for a production system (so far it worked with the IDES system without any problems, though). The following locales are needed: de_DE.ISO-8859-1 en_US.ISO-8859-1 If they are not present, there will be some problems during the installation. If these are then subsequently ignored (eg by setting the status of the offending steps to OK in file CENTRDB.R3S), it will be impossible to log onto the SAP-system without some additional effort. Kernel Tuning kernel tuning SAP R/3 Systems need a lot of resources. I therefore added the following parameters to my kernel config-file: # Set these for memory pigs (SAP and Oracle): options MAXDSIZ="(1024*1024*1024)" options DFLDSIZ="(1024*1024*1024)" # System V options needed. options SYSVSHM #SYSV-style shared memory options SHMMAXPGS=262144 #max amount of shared mem. pages options SHMMNI=256 #max number of shared memory ident if. options SHMSEG=100 #max shared mem.segs per process options SYSVMSG #SYSV-style message queues options MSGSEG=32767 #max num. of mes.segments in system options MSGSSZ=32 #size of msg-seg. MUST be power of 2 options MSGMNB=65535 #max char. per message queue options MSGTQL=2046 #max amount of msgs in system options SYSVSEM #SYSV-style semaphores options SEMMNU=256 #number of semaphore UNDO structures options SEMMNS=1024 #number of semaphores in system options SEMMNI=520 #number of semaphore indentifiers options SEMUME=100 #number of UNDO keys The minimum values are specified in the documentation that comes from SAP. As there is no description for Linux, see the HP-UX-section (32-bit) for further information. Installing SAP R/3 Preparing SAP CD-ROMs There are lots of CD-ROMs to mount and unmount during installation. Assuming you have enough CD-ROM-drives, you can just mount them all. I decided to copy the CD-ROM contents to corresponding directories: /oracle/IDS/sapreorg/<cd-name> where <cd-name> was one of KERNEL, RDBMS, EXPORT1, EXPORT2, EXPORT3, EXPORT4, EXPORT5 and EXPORT6. All the filenames should be in capital letters, otherwise use the -g option for mounting. So use the following commands: &prompt.root; mount_cd9660 -g /dev/cd0a /mnt &prompt.root; cp -R /mnt/* /oracle/IDS/sapreorg/<cd-name> &prompt.root; umount /mnt Running the install-script First we need to prepare an install-directory: &prompt.root; cd /oracle/IDS/sapreorg &prompt.root; mkdir install &prompt.root; cd install Then the install-script is started, which will copy nearly all the relevant files into the install-directory: /oracle/IDS/sapreorg/KERNEL/UNIX/INSTTOOL.SH As this is an IDES-Installation with a fully customized SAP R/3 Demo-System, we have six instead of just three EXPORT-CDs. At this point the installation template CENTRDB.R3S is for installing a standard central instance (R/3 and Database), not an IDES central instance, so copy the corresponding CENTRDB.R3S from the EXPORT1 directory, otherwise R3SETUP will only ask for three EXPORT-CDs. Start R3SETUP Make sure LD_LIBRARY_PATH is set correctly: &prompt.root; export LD_LIBRARY_PATH=/oracle/IDS/lib:/sapmnt/IDS/exe:/oracle/805_32/lib Start R3SETUP as user root from installation directory: &prompt.root; cd /oracle/IDS/sapreorg/install &prompt.root; ./R3SETUP -f CENTRDB.R3S The script then asks some questions (defaults in brackets, followed by actual input): Question Default Input Enter SAP System ID [C11] IDS<ret> Enter SAP Instance Number [00] <ret> Enter SAPMOUNT Directory [/sapmnt] <ret> Enter name of SAP central host [troubadix.domain.de] <ret> Enter name of SAP db host [troubadix] <ret> Select character set [1] (WE8DEC) <ret> Enter Oracle server version (1) Oracle 8.0.5, (2) Oracle 8.0.6, (3) Oracle 8.1.5, (4) Oracle 8.1.6 1<ret> Extract Oracle Client archive [1] (Yes, extract) <ret> Enter path to KERNEL CD [/sapcd] /oracle/IDS/sapreorg/KERNEL Enter path to RDBMS CD [/sapcd] /oracle/IDS/sapreorg/RDBMS Enter path to EXPORT1 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT1 Directory to copy EXPORT1 CD [/oracle/IDS/sapreorg/CD4_DIR] <ret> Enter path to EXPORT2 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT2 Directory to copy EXPORT2 CD [/oracle/IDS/sapreorg/CD5_DIR] <ret> Enter path to EXPORT3 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT3 Directory to copy EXPORT3 CD [/oracle/IDS/sapreorg/CD6_DIR] <ret> Enter path to EXPORT4 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT4 Directory to copy EXPORT4 CD [/oracle/IDS/sapreorg/CD7_DIR] <ret> Enter path to EXPORT5 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT5 Directory to copy EXPORT5 CD [/oracle/IDS/sapreorg/CD8_DIR] <ret> Enter path to EXPORT6 CD [/sapcd] /oracle/IDS/sapreorg/EXPORT6 Directory to copy EXPORT6 CD [/oracle/IDS/sapreorg/CD9_DIR] <ret> Enter amount of RAM for SAP + DB 850<ret> (in Megabytes) Service Entry Message Server [3600] <ret> Enter Group-ID of sapsys [101] <ret> Enter Group-ID of oper [102] <ret> Enter Group-ID of dba [100] <ret> Enter User-ID of <sid>adm [1000] <ret> Enter User-ID of ora<sid> [1002] <ret> Number of parallel procs [2] <ret> If I had not copied the CDs to the different locations, then the SAP-Installer can't find the CD needed (identified by the LABEL.ASC-File on CD) and would then ask you to insert / mount the CD and confirm or enter the mountpath. The CENTRDB.R3S might not be error-free. In my case, it requested EXPORT4 again (but indicated the correct key (6_LOCATI ON, then 7_LOCATION etc.), so one can just continue with entering the correct values. Don't get irritated. Apart from some problems mentioned below, everything should go straight throught up to the point where the Oracle database software needs to be installed. Installing Oracle 8.0.5 Please see the corresponding SAP-Notes and Oracle Readmes regarding Linux and Oracle DB for possible problems. Most if not all problems stem from incompatible libraries For more information on installing Oracle, refer to the Installing Oracle chapter. Installing the Oracle 8.0.5 with orainst If Oracle 8.0.5 is to be used, some additional libraries are needed for successfully relinking, as Oracle 8.0.5 was linked with an old glibc (RedHat 6.0), but RedHat 6.1 already uses a new glibc. So you have to install the following additional packages to ensure that linking will work: compat-libs-5.2-2.i386.rpm compat-glibc-5.2-2.0.7.2.i386.rpm compat-egcs-5.2-1.0.3a.1.i386.rpm compat-egcs-c++-5.2-1.0.3a.1.i386.rpm compat-binutils-5.2-2.9.1.0.23.1.i386.rpm See the corresponding SAP-Notes or Oracle Readmes for further information. If this is no option (at the time of installation I didn't have enough time to check this), one could use the original binaries, or use the relinked binaries from an original RedHat System. For compiling the intelligent agent, the RedHat TCL package must be installed. If you can't get tcl-8.0.3-20.i386.rpm, a newer one like tcl-8.0.5-30.i386.rpm for RedHat 6.1 should also do. Apart from relinking, the installation is straightforward: &prompt.root; su - oraids &prompt.root; export TERM=xterm &prompt.root; export ORACLE_TERM=xterm &prompt.root; export ORACLE_HOME=/oracle/IDS &prompt.root; cd /ORACLE_HOME/orainst_sap &prompt.root; ./orainst Confirm all Screens with Enter until the software is installed, except that one has to deselect the Oracle On-Line Text Viewer , as this is not currently available for Linux. Oracle then wants to relink with i386-glibc20-linux-gcc instead of the available gcc, egcs or i386-redhat-linux-gcc . Due to time constrains I decided to use the binaries from an Oracle 8.0.5 PreProduction release, after the first attempt at getting the version from the RDBMS-CD working, failed, and finding / accessing the correct RPMs was a nightmare at that time. Installing the Oracle 8.0.5 Pre-Production release for Linux (Kernel 2.0.33) This installation is quite easy. Mount the CD, start the installer. It will then ask for the location of the Oracle home directory, and copy all binaries there. I did not delete the remains of my previous RDBMS-installation tries, though. Afterwards, Oracle Database could be started with no problems. Continue with SAP R/3 installation First check the environment settings of users idsamd (<sid>adm) and oraids (ora<sid>). They should now both have the files .profile , .login and .cshrc which are all using hostname. In case the system's hostname is the fully qualified name, you need to change hostname to hostname -s within all three files. Database load Afterwards, R3SETUP can either be restarted or continued (depending on whether exit was chosen or not). R3SETUP then creates the tablespaces and loads the data from EXPORT1 to EXPORT6 (remember, it is an IDES system, otherwise it would only be EXPORT1 to EXPORT3) with R3load into the database. When the database load is finished (might take a few hours), some passwords are requested. For test installations, one can use the well known default passwords (use different ones if security is an issue!): Question Input Enter Password for sapr3 sap<ret> Confirum Password for sapr3 sap<ret> Enter Password for sys change_on_install<ret> Confirm Password for sys change_on_install<ret> Enter Password for system manager<ret> Confirm Password for system manager<ret> At this point I had a few problems with dipgntab. Listener Start the Oracle-Listener as user oraids (ora<sid>) as follows: umask 0; lsnrctl start Otherwise you might get ORA-12546 as the sockets won't have the correct permissions. See SAP note 072984. Post-installation steps Request SAP R/3 license key This is needed, as the temporary license is only valid for four weeks. Don't forget to enter the correct Operating System: (X) Other: FreeBSD 4.3 Stable. First get the hardware key. Log on as user idsadm and call saplicense: &prompt.root; /sapmnt/IDS/exe/saplicense -get Calling saplicense without options gives a list of options. Upon receiving the license key, it can be installed using &prompt.root; /sapmnt/IDS/exe/saplicense -install You are then required to enter the following values: SAP SYSTEM ID = <SID, 3 chars> CUSTOMER KEY = <hardware key, 11 chars> INSTALLATION NO = <installation, 10 digits> EXPIRATION DATE = <yyyymmdd, usually "99991231"> LICENSE KEY = <license key, 24 chars> Creating Users Create a user within client 000 (for some tasks required to be done within client 000, but with a user different from users sap* and ddic). As a username, I usually choose wartung (or service in English). Profiles required are sap_new and sap_all. For additional safety the passwords of default users within all clients should be changed (this includes users sap* and ddic). Configure Transport System, Profile, Operation Modes, etc. Within client 000, user different from ddic and sap*, do at least the following: Task Transaction Configure Transport System, eg as Stand-Alone Transport Domain Entity STMS Create / Edit Profile for System RZ10 Maintain Operation Modes and Instances RZ04 These and all the other post-installation steps are thoroughly described in SAP installation guides. Edit init<sid>.sap (initIDS.sap) The file /oracle/IDS/dbs/initIDS.sap contains the SAP backup profile. Here the size of the tape to be used, type of compression and so on need to be defined. To get this running with sapdba / brbackup, I changed the following values: compress = hardware archive_function = copy_delete_save cpio_flags = "-ov --format=newc --block-size=128 --quiet" cpio_in_flags = "-iuv --block-size=128 --quiet" tape_size = 38000M tape_address = /dev/nsa0 tape_address_rew = /dev/sa0 Explanations: compress The tape I use is a HP DLT1 which does hardware compression. archive_function This defines the default behaviour for saving Oracle archive logs: New logfiles are saved to tape, already saved logfiles are saved again and are then deleted. This prevents lots of trouble if one needs to recover the database, and one of the archive-tapes has gone bad. cpio_flags Default is to use -B which sets blocksize to 5120 Bytes. For DLT-Tapes, HP recommends at least 32K blocksize, so I used --block-size=128 for 64K. --format=newc is needed I have inode numbers greater than 65535. The last option --quiet is needed as otherwise brbackup complains as soon as cpio outputs the numbers of blocks saved. cpio_in_flags Flags needed for loading data back from tape. Format is reckognized automagically. tape_size This usually gives the raw storage capability of the tape. For security reason (we use hardware compression), the value is slightly lower than the actual value. tape_address The non-rewindable device to be used with cpio. tape_address_rew The rewindable device to be used with cpio. Problems during installation OSUSERSIDADM_IND_ORA during R3SETUP If R3SETUP complains at this stage, edit file CENTRDB.R3S. Locate [OSUSERSIDADM_IND_ORA] and edit the following values: HOME=/home/idsadm (was empty) STATUS=OK (had status ERROR) Then you can restart R3SETUP with: &prompt.root; ./R3SETUP -f CENTRDB.R3S OSUSERDBSID_IND_ORA during R3SETUP Possibly R3SETUP also complains at this stage. Just edit CENTRDB.R3S. Locate [OSUSERDBSID_IND_ORA] and edit the following value in that section: STATUS=OK Then just restart R3SETUP again: &prompt.root; ./R3SETUP -f CENTRDB.R3S oraview.vrf FILE NOT FOUND during Oracle installation You haven't deselected Oracle On-Line Text Viewer before starting the installation. This is marked for installation even though this option is currently not available for Linux. Deselect this product inside the Oracle installation menu and restart installation. TEXTENV_INVALID during R3SETUP, RFC or SAPGUI start If this error is encountered, the correct locale is missing. SAP note 0171356 lists the necessary RPMs that need be installed (eg saplocales-1.0-3, saposcheck-1.0-1 for RedHat 6.1). In case you ignored all the related errors and set the corresponding status from ERROR to OK (in CENTRDB.R3S) every time R3SETUP complained and just restarted R3SETUP, the SAP-System will not be properly configured and you will then not be able to connect to the system with a sapgui, even though the system can be started. Trying to connect with the old Linux sapgui gave the following messages: Sat May 5 14:23:14 2001 *** ERROR => no valid userarea given [trgmsgo. 0401] Sat May 5 14:23:22 2001 *** ERROR => ERROR NR 24 occured [trgmsgi. 0410] *** ERROR => Error when generating text environment. [trgmsgi. 0435] *** ERROR => function failed [trgmsgi. 0447] *** ERROR => no socket operation allowed [trxio.c 3363] Speicherzugriffsfehler This behaviour is due to SAP R/3 being unable to correctly assign a locale and also not being properly configured itself (missing entries in some database tables). To be able to connect to SAP, add the following entries to file DEFAULT.PFL (see note 0043288): abap/set_etct_env_at_new_mode =0 install/collate/active =0 rscp/TCP0B =TCP0B Restart the SAP system. Now one can connect to the system, even though country-specific language settings might not work as expected. After correcting country-settings (and providing the correct locales), these entries can be removed from DEFAULT.PFL and the SAP system can be restarted. ORA-12546. Start Listener with correct permissions Start the Oracle Listener as user oraids with the following commands: &prompt.root; umask 0; lsnrctl start Otherwise one might get ORA-12546 as the sockets won't have the correct permissions. See SAP note 0072984. [DIPGNTAB_IND_IND] during R3SETUP In general, see SAP note 0130581 (R3SETUP step DIPGNTAB terminates). During this specific installation, for some reasons the installation process was not using the proper SAP system name "IDS", but the empty string "" instead. This lead to some minor problems with accessing directories, as the paths are generated dynamically using <sid> (in this case IDS). So instead of accessing: /usr/sap/IDS/SYS/... /usr/sap/IDS/DVMGS00 the following path were used: /usr/sap//SYS/... /usr/sap/D00i To continue with the installation, I created a link and an additional directory: &prompt.root; pwd /compat/linux/usr/sap &prompt.root; ls -l total 4 drwxr-xr-x 3 idsadm sapsys 512 May 5 11:20 D00 drwxr-x--x 5 idsadm sapsys 512 May 5 11:35 IDS lrwxr-xr-x 1 root sapsys 7 May 5 11:35 SYS -> IDS/SYS drwxrwxr-x 2 idsadm sapsys 512 May 5 13:00 tmp drwxrwxr-x 11 idsadm sapsys 512 May 4 14:20 trans I also found SAP notes (0029227 and 0008401) describing this behaviour. [RFCRSWBOINI_IND_IND] during R3SETUP Set STATUS of the offending step from ERROR to OK (file CENTRDB.R3S) and restart R3SETUP. After installation, you have to execute the report RSWBOINS from transaction SE38. See SAP note 0162266 for additional information about phase RFCRSWBOINI and RFCRADDBDIF. [RFCRADDBDIF_IND_IND] during R3SETUP Set STATUS of the offending step from ERROR to OK (file CENTRDB.R3S) and restart R3SETUP. After installation, you have to execute the report RADDBDIF from transaction SE38. See SAP note 0162266 for further information. Advanced Topics If you are curious as to how the Linux binary compatibility works, this is the section you want to read. Most of what follows is based heavily on an email written to &a.chat; by Terry Lambert tlambert@primenet.com (Message ID: <199906020108.SAA07001@usr09.primenet.com>). How Does It Work? execution class loader FreeBSD has an abstraction called an execution class loader. This is a wedge into the &man.execve.2; system call. What happens is that FreeBSD has a list of loaders, instead of a single loader with a fallback to the #! loader for running any shell interpreters or shell scripts. Historically, the only loader on the UNIX platform examined the magic number (generally the first 4 or 8 bytes of the file) to see if it was a binary known to the system, and if so, invoked the binary loader. If it was not the binary type for the system, the &man.execve.2; call returned a failure, and the shell attempted to start executing it as shell commands. The assumption was a default of whatever the current shell is. Later, a hack was made for &man.sh.1; to examine the first two characters, and if they were :\n, then it invoked the &man.csh.1; shell instead (we believe SCO first made this hack). What FreeBSD does now is go through a list of loaders, with a generic #! loader that knows about interpreters as the characters which follow to the next whitespace next to last, followed by a fallback to /bin/sh. ELF For the Linux ABI support, FreeBSD sees the magic number as an ELF binary (it makes no distinction between FreeBSD, Solaris, Linux, or any other OS which has an ELF image type, at this point). Solaris The ELF loader looks for a specialized brand, which is a comment section in the ELF image, and which is not present on SVR4/Solaris ELF binaries. For Linux binaries to function, they must be branded as type Linux; from &man.brandelf.1;: &prompt.root; brandelf -t Linux file When this is done, the ELF loader will see the Linux brand on the file. ELF branding When the ELF loader sees the Linux brand, the loader replaces a pointer in the proc structure. All system calls are indexed through this pointer (in a traditional UNIX system, this would be the sysent[] structure array, containing the system calls). In addition, the process flagged for special handling of the trap vector for the signal trampoline code, and sever other (minor) fix-ups that are handled by the Linux kernel module. The Linux system call vector contains, among other things, a list of sysent[] entries whose addresses reside in the kernel module. When a system call is called by the Linux binary, the trap code dereferences the system call function pointer off the proc structure, and gets the Linux, not the FreeBSD, system call entry points. In addition, the Linux mode dynamically reroots lookups; this is, in effect, what the union option to FS mounts (not the unionfs!) does. First, an attempt is made to lookup the file in the /compat/linux/original-path directory, then only if that fails, the lookup is done in the /original-path directory. This makes sure that binaries that require other binaries can run (e.g., the Linux toolchain can all run under Linux ABI support). It also means that the Linux binaries can load and exec FreeBSD binaries, if there are no corresponding Linux binaries present, and that you could place a &man.uname.1; command in the /compat/linux directory tree to ensure that the Linux binaries could not tell they were not running on Linux. In effect, there is a Linux kernel in the FreeBSD kernel; the various underlying functions that implement all of the services provided by the kernel are identical to both the FreeBSD system call table entries, and the Linux system call table entries: file system operations, virtual memory operations, signal delivery, System V IPC, etc… The only difference is that FreeBSD binaries get the FreeBSD glue functions, and Linux binaries get the Linux glue functions (most older OS's only had their own glue functions: addresses of functions in a static global sysent[] structure array, instead of addresses of functions dereferenced off a dynamically initialized pointer in the proc structure of the process making the call). Which one is the native FreeBSD ABI? It does not matter. Basically the only difference is that (currently; this could easily be changed in a future release, and probably will be after this) the FreeBSD glue functions are statically linked into the kernel, and the Linux glue functions can be statically linked, or they can be accessed via a kernel module. Yeah, but is this really emulation? No. It is an ABI implementation, not an emulation. There is no emulator (or simulator, to cut off the next question) involved. So why is it sometimes called Linux emulation? To make it hard to sell FreeBSD! 8-). Really, it is because the historical implementation was done at a time when there was really no word other than that to describe what was going on; saying that FreeBSD ran Linux binaries was not true, if you did not compile the code in or load a module, and there needed to be a word to describe what was being loaded—hence the Linux emulator. diff --git a/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml b/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml index 2d7a9ca646..909ec5f3b5 100644 --- a/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/mirrors/chapter.sgml @@ -1,3861 +1,3856 @@ Obtaining FreeBSD CDROM Publishers FreeBSD is available on CDROM from several retailers:
Daemon News 2680 Bayshore Parkway, Suite 307 Mountain View, CA 94043 USA Phone: +1 650 694-4949 Email: sales@daemonnews.org WWW: http://www.bsdmall.com/
Wind River Systems 4041 Pike Lane, Suite F Concord, CA 94520 USA Phone: +1 925 691-2800 Fax: +1 925 674-0821 Email: info@osd.bsdi.com WWW: http://www.freebsdmall.com/
If you are a reseller and want to carry FreeBSD CDROM products, please contact the relevant department at Wind River Systems or:
Cylogistics 2680 Bayshore Parkway, Suite 307 Mountain View, CA 94043 USA Phone: +1 650 694-4949 Fax: +1 650 694-4953 Email: sales@cylogistics.com WWW: http://www.cylogistics.com/
DVD Publishers FreeBSD is available on DVD from:
FreeBSD Services Ltd 11 Lapwing Close Bicester OX26 6XR United Kingdom WWW: http://www.freebsd-services.com/
FTP Sites The official sources for FreeBSD are available via anonymous FTP from:
ftp://ftp.FreeBSD.org/pub/FreeBSD/.
The FreeBSD mirror sites database is more accurate than the mirror listing in the handbook, as it gets its information from the DNS rather than relying on static lists of hosts. Additionally, FreeBSD is available via anonymous FTP from the following mirror sites. If you choose to obtain FreeBSD via anonymous FTP, please try to use a site near you. Argentina, Australia, Brazil, Canada, China, Czech Republic, Denmark, Estonia, Finland, France, Germany, Hong Kong, Hungary, Ireland, Israel, Japan, Korea, Lithuania, Netherlands, New Zealand, Poland, Portugal, Russia, Saudi Arabia, South Africa, Spain, Slovak Republic, Slovenia, Sweden, Taiwan, Thailand, UK, Ukraine, USA. Argentina In case of problems, please contact the hostmaster hostmaster@ar.FreeBSD.org for this domain. ftp://ftp.ar.FreeBSD.org/pub/FreeBSD/ Australia In case of problems, please contact the hostmaster hostmaster@au.FreeBSD.org for this domain. ftp://ftp.au.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.au.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.au.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.au.FreeBSD.org/pub/FreeBSD/ Brazil In case of problems, please contact the hostmaster hostmaster@br.FreeBSD.org for this domain. ftp://ftp.br.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.br.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.br.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.br.FreeBSD.org/pub/FreeBSD/ ftp://ftp5.br.FreeBSD.org/pub/FreeBSD/ ftp://ftp6.br.FreeBSD.org/pub/FreeBSD/ ftp://ftp7.br.FreeBSD.org/pub/FreeBSD/ Canada In case of problems, please contact the hostmaster hostmaster@ca.FreeBSD.org for this domain. ftp://ftp.ca.FreeBSD.org/pub/FreeBSD/ China In case of problems, please contact the hostmaster phj@cn.FreeBSD.org for this domain. ftp://ftp.cn.FreeBSD.org/pub/FreeBSD/ Czech Republic In case of problems, please contact the hostmaster hostmaster@cz.FreeBSD.org for this domain. ftp://ftp.cz.FreeBSD.org/pub/FreeBSD/ Contact: calda@dzungle.ms.mff.cuni.cz Denmark In case of problems, please contact the hostmaster hostmaster@dk.FreeBSD.org for this domain. ftp://ftp.dk.FreeBSD.org/pub/FreeBSD/ Estonia In case of problems, please contact the hostmaster hostmaster@ee.FreeBSD.org for this domain. ftp://ftp.ee.FreeBSD.org/pub/FreeBSD/ Finland In case of problems, please contact the hostmaster hostmaster@fi.FreeBSD.org for this domain. ftp://ftp.fi.FreeBSD.org/pub/FreeBSD/ France In case of problems, please contact the hostmaster hostmaster@fr.FreeBSD.org for this domain. ftp://ftp.fr.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.fr.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.fr.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.fr.FreeBSD.org/pub/FreeBSD/ ftp://ftp5.fr.FreeBSD.org/pub/FreeBSD/ ftp://ftp6.fr.FreeBSD.org/pub/FreeBSD/ Germany In case of problems, please contact the mirror admins de-bsd-hubs@de.FreeBSD.org for this domain. ftp://ftp.de.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.de.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.de.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.de.FreeBSD.org/pub/FreeBSD/ ftp://ftp5.de.FreeBSD.org/pub/FreeBSD/ ftp://ftp6.de.FreeBSD.org/pub/FreeBSD/ ftp://ftp7.de.FreeBSD.org/pub/FreeBSD/ Hong Kong ftp://ftp.hk.super.net/pub/FreeBSD/ Contact: ftp-admin@HK.Super.NET. Hungary In case of problems, please contact the hostmaster mohacsi@ik.bme.hu for this domain. ftp://ftp.hu.FreeBSD.org/pub/FreeBSD/ Ireland In case of problems, please contact the hostmaster hostmaster@ie.FreeBSD.org for this domain. ftp://ftp.ie.FreeBSD.org/pub/FreeBSD/ Israel In case of problems, please contact the hostmaster hostmaster@il.FreeBSD.org for this domain. ftp://ftp.il.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.il.FreeBSD.org/pub/FreeBSD/ Japan In case of problems, please contact the hostmaster hostmaster@jp.FreeBSD.org for this domain. ftp://ftp.jp.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.jp.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.jp.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.jp.FreeBSD.org/pub/FreeBSD/ ftp://ftp5.jp.FreeBSD.org/pub/FreeBSD/ ftp://ftp6.jp.FreeBSD.org/pub/FreeBSD/ Korea In case of problems, please contact the hostmaster hostmaster@kr.FreeBSD.org for this domain. ftp://ftp.kr.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.kr.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.kr.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.kr.FreeBSD.org/pub/FreeBSD/ ftp://ftp5.kr.FreeBSD.org/pub/FreeBSD/ ftp://ftp6.kr.FreeBSD.org/pub/FreeBSD/ Lithuania In case of problems, please contact the hostmaster hostmaster@lt.FreeBSD.org for this domain. ftp://ftp.lt.FreeBSD.org/pub/FreeBSD/ Netherlands In case of problems, please contact the hostmaster hostmaster@nl.FreeBSD.org for this domain. ftp://ftp.nl.FreeBSD.org/pub/FreeBSD/ New Zealand In case of problems, please contact the hostmaster hostmaster@nz.FreeBSD.org for this domain. ftp://ftp.nz.FreeBSD.org/pub/FreeBSD/ Poland In case of problems, please contact the hostmaster hostmaster@pl.FreeBSD.org for this domain. ftp://ftp.pl.FreeBSD.org/pub/FreeBSD/ Portugal In case of problems, please contact the hostmaster hostmaster@pt.FreeBSD.org for this domain. ftp://ftp.pt.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.pt.FreeBSD.org/pub/FreeBSD/ Romania In case of problems, please contact the hostmaster hostmaster@ro.FreeBSD.org for this domain. ftp://ftp.ro.FreeBSD.org/pub/FreeBSD/ Russia In case of problems, please contact the hostmaster hostmaster@ru.FreeBSD.org for this domain. ftp://ftp.ru.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.ru.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.ru.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.ru.FreeBSD.org/pub/FreeBSD/ Saudi Arabia In case of problems, please contact ftpadmin@isu.net.sa ftp://ftp.isu.net.sa/pub/mirrors/ftp.freebsd.org/ South Africa In case of problems, please contact the hostmaster hostmaster@za.FreeBSD.org for this domain. ftp://ftp.za.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.za.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.za.FreeBSD.org/FreeBSD/ Slovak Republic In case of problems, please contact the hostmaster hostmaster@sk.FreeBSD.org for this domain. ftp://ftp.sk.FreeBSD.org/pub/FreeBSD/ Slovenia In case of problems, please contact the hostmaster hostmaster@si.FreeBSD.org for this domain. ftp://ftp.si.FreeBSD.org/pub/FreeBSD/ Spain In case of problems, please contact the hostmaster hostmaster@es.FreeBSD.org for this domain. ftp://ftp.es.FreeBSD.org/pub/FreeBSD/ Sweden In case of problems, please contact the hostmaster hostmaster@se.FreeBSD.org for this domain. ftp://ftp.se.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.se.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.se.FreeBSD.org/pub/FreeBSD/ Taiwan In case of problems, please contact the hostmaster hostmaster@tw.FreeBSD.org for this domain. ftp://ftp.tw.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.tw.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.tw.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.tw.FreeBSD.org/pub/FreeBSD/ Thailand ftp://ftp.nectec.or.th/pub/FreeBSD/ Contact: ftpadmin@ftp.nectec.or.th. Ukraine ftp://ftp.ua.FreeBSD.org/pub/FreeBSD/ Contact: freebsd-mnt@lucky.net. UK In case of problems, please contact the hostmaster hostmaster@uk.FreeBSD.org for this domain. ftp://ftp.uk.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.uk.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.uk.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.uk.FreeBSD.org/pub/FreeBSD/ ftp://ftp5.uk.FreeBSD.org/pub/FreeBSD/ USA In case of problems, please contact the hostmaster hostmaster@FreeBSD.org for this domain. ftp://ftp.FreeBSD.org/pub/FreeBSD/ ftp://ftp2.FreeBSD.org/pub/FreeBSD/ ftp://ftp3.FreeBSD.org/pub/FreeBSD/ ftp://ftp4.FreeBSD.org/pub/FreeBSD/ ftp://ftp5.FreeBSD.org/pub/FreeBSD/ ftp://ftp6.FreeBSD.org/pub/FreeBSD/ ftp://ftp7.FreeBSD.org/pub/FreeBSD/ ftp://ftp8.FreeBSD.org/pub/FreeBSD/ ftp://ftp9.FreeBSD.org/pub/os/FreeBSD/ ftp://ftp10.FreeBSD.org/pub/FreeBSD/ Anonymous CVS <anchor id="anoncvs-intro">Introduction Anonymous CVS (or, as it is otherwise known, anoncvs) is a feature provided by the CVS utilities bundled with FreeBSD for synchronizing with a remote CVS repository. Among other things, it allows users of FreeBSD to perform, with no special privileges, read-only CVS operations against one of the FreeBSD project's official anoncvs servers. To use it, one simply sets the CVSROOT environment variable to point at the appropriate anoncvs server, provides the well-known password anoncvs with the cvs login command, and then uses the &man.cvs.1; command to access it like any local repository. While it can also be said that the CVSup and anoncvs services both perform essentially the same function, there are various trade-offs which can influence the user's choice of synchronization methods. In a nutshell, CVSup is much more efficient in its usage of network resources and is by far the most technically sophisticated of the two, but at a price. To use CVSup, a special client must first be installed and configured before any bits can be grabbed, and then only in the fairly large chunks which CVSup calls collections. Anoncvs, by contrast, can be used to examine anything from an individual file to a specific program (like ls or grep) by referencing the CVS module name. Of course, anoncvs is also only good for read-only operations on the CVS repository, so if it's your intention to support local development in one repository shared with the FreeBSD project bits then CVSup is really your only option. <anchor id="anoncvs-usage">Using Anonymous CVS Configuring &man.cvs.1; to use an Anonymous CVS repository is a simple matter of setting the CVSROOT environment variable to point to one of the FreeBSD project's anoncvs servers. At the time of this writing, the following servers are available: USA: :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs (Use cvs login and enter the password anoncvs when prompted.) Since CVS allows one to check out virtually any version of the FreeBSD sources that ever existed (or, in some cases, will exist :-), you need to be familiar with the revision () flag to &man.cvs.1; and what some of the permissible values for it in the FreeBSD Project repository are. There are two kinds of tags, revision tags and branch tags. A revision tag refers to a specific revision. Its meaning stays the same from day to day. A branch tag, on the other hand, refers to the latest revision on a given line of development, at any given time. Because a branch tag does not refer to a specific revision, it may mean something different tomorrow than it means today. Here are the branch tags that users might be interested in (keep in mind that the only tags valid for the ports collection is HEAD). HEAD Symbolic name for the main line, or FreeBSD-CURRENT. Also the default when no revision is specified. RELENG_4 The line of development for FreeBSD-4.X, also known as FreeBSD-STABLE. RELENG_4_3 The release branch for FreeBSD-4.3, used only for security advisories and other seriously critical fixes. RELENG_3 The line of development for FreeBSD-3.X, also known as 3.X-STABLE. RELENG_2_2 The line of development for FreeBSD-2.2.X, also known as 2.2-STABLE. This branch is mostly obsolete. Here are the revision tags that users might be interested in. Again, none of these are valid for the ports collection since the ports collection does not have multiple revisions. RELENG_4_3_0_RELEASE FreeBSD 4.3. RELENG_4_2_0_RELEASE FreeBSD 4.2. RELENG_4_1_1_RELEASE FreeBSD 4.1.1. RELENG_4_1_0_RELEASE FreeBSD 4.1. RELENG_4_0_0_RELEASE FreeBSD 4.0. RELENG_3_5_0_RELEASE FreeBSD-3.5. RELENG_3_4_0_RELEASE FreeBSD-3.4. RELENG_3_3_0_RELEASE FreeBSD-3.3. RELENG_3_2_0_RELEASE FreeBSD-3.2. RELENG_3_1_0_RELEASE FreeBSD-3.1. RELENG_3_0_0_RELEASE FreeBSD-3.0. RELENG_2_2_8_RELEASE FreeBSD-2.2.8. RELENG_2_2_7_RELEASE FreeBSD-2.2.7. RELENG_2_2_6_RELEASE FreeBSD-2.2.6. RELENG_2_2_5_RELEASE FreeBSD-2.2.5. RELENG_2_2_2_RELEASE FreeBSD-2.2.2. RELENG_2_2_1_RELEASE FreeBSD-2.2.1. RELENG_2_2_0_RELEASE FreeBSD-2.2.0. When you specify a branch tag, you normally receive the latest versions of the files on that line of development. If you wish to receive some past version, you can do so by specifying a date with the flag. See the &man.cvs.1; man page for more details. Examples While it really is recommended that you read the manual page for &man.cvs.1; thoroughly before doing anything, here are some quick examples which essentially show how to use Anonymous CVS: Checking out something from -CURRENT (&man.ls.1;) and deleting it again: - -&prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs + &prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs &prompt.user; cvs login At the prompt, enter the password anoncvs. &prompt.user; cvs co ls &prompt.user; cvs release -d ls &prompt.user; cvs logout Checking out the version of &man.ls.1; in the 3.X-STABLE branch: - -&prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs + &prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs &prompt.user; cvs login At the prompt, enter the password anoncvs. &prompt.user; cvs co -rRELENG_3 ls &prompt.user; cvs release -d ls &prompt.user; cvs logout Creating a list of changes (as unified diffs) to &man.ls.1; - -&prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs + &prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs &prompt.user; cvs login At the prompt, enter the password anoncvs. &prompt.user; cvs rdiff -u -rRELENG_3_0_0_RELEASE -rRELENG_3_4_0_RELEASE ls &prompt.user; cvs logout Finding out what other module names can be used: - -&prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs + &prompt.user; setenv CVSROOT :pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs &prompt.user; cvs login At the prompt, enter the password anoncvs. &prompt.user; cvs co modules &prompt.user; more modules/modules &prompt.user; cvs release -d modules &prompt.user; cvs logout Other Resources The following additional resources may be helpful in learning CVS: CVS Tutorial from Cal Poly. Cyclic Software, commercial maintainers of CVS. CVSWeb is the FreeBSD Project web interface for CVS. Using CTM CTM is a method for keeping a remote directory tree in sync with a central one. It has been developed for usage with FreeBSD's source trees, though other people may find it useful for other purposes as time goes by. Little, if any, documentation currently exists at this time on the process of creating deltas, so talk to &a.phk; for more information should you wish to use CTM for other things. Why should I use <application>CTM</application>? CTM will give you a local copy of the FreeBSD source trees. There are a number of “flavors” of the tree available. Whether you wish to track the entire CVS tree or just one of the branches, CTM can provide you the information. If you are an active developer on FreeBSD, but have lousy or non-existent TCP/IP connectivity, or simply wish to have the changes automatically sent to you, CTM was made for you. You will need to obtain up to three deltas per day for the most active branches. However, you should consider having them sent by automatic email. The sizes of the updates are always kept as small as possible. This is typically less than 5K, with an occasional (one in ten) being 10-50K and every now and then a biggie of 100K+ or more coming around. You will also need to make yourself aware of the various caveats related to working directly from the development sources rather than a pre-packaged release. This is particularly true if you choose the “current” sources. It is recommended that you read Staying current with FreeBSD. What do I need to use <application>CTM</application>? You will need two things: The CTM program, and the initial deltas to feed it (to get up to “current” levels). The CTM program has been part of FreeBSD ever since version 2.0 was released, and lives in /usr/src/usr.sbin/CTM if you have a copy of the source available. If you are running a pre-2.0 version of FreeBSD, you can fetch the current CTM sources directly from: ftp://ftp.FreeBSD.org/pub/FreeBSD/FreeBSD-current/src/usr.sbin/ctm/ The “deltas” you feed CTM can be had two ways, FTP or email. If you have general FTP access to the Internet then the following FTP sites support access to CTM: ftp://ftp.FreeBSD.org/pub/FreeBSD/CTM/ or see section mirrors. FTP the relevant directory and fetch the README file, starting from there. If you wish to get your deltas via email: Send email to &a.majordomo; to subscribe to one of the CTM distribution lists. “ctm-cvs-cur” supports the entire cvs tree. “ctm-src-cur” supports the head of the development branch. “ctm-src-2_2” supports the 2.2 release branch, etc.. (If you do not know how to subscribe yourself using majordomo, send a message first containing the word help — it will send you back usage instructions.) When you begin receiving your CTM updates in the mail, you may use the ctm_rmail program to unpack and apply them. You can actually use the ctm_rmail program directly from a entry in /etc/aliases if you want to have the process run in a fully automated fashion. Check the ctm_rmail man page for more details. No matter what method you use to get the CTM deltas, you should subscribe to the ctm-announce@FreeBSD.org mailing list. In the future, this will be the only place where announcements concerning the operations of the CTM system will be posted. Send an email to &a.majordomo; with a single line of subscribe ctm-announce to get added to the list. Using <application>CTM</application> for the first time Before you can start using CTM deltas, you will need to get to a starting point for the deltas produced subsequently to it. First you should determine what you already have. Everyone can start from an “empty” directory. You must use an initial “Empty” delta to start off your CTM supported tree. At some point it is intended that one of these “started” deltas be distributed on the CD for your convenience, however, this does not currently happen. Since the trees are many tens of megabytes, you should prefer to start from something already at hand. If you have a -RELEASE CD, you can copy or extract an initial source from it. This will save a significant transfer of data. You can recognize these “starter” deltas by the X appended to the number (src-cur.3210XEmpty.gz for instance). The designation following the X corresponds to the origin of your initial “seed”. Empty is an empty directory. As a rule a base transition from Empty is produced every 100 deltas. By the way, they are large! 25 to 30 Megabytes of gzip'd data is common for the XEmpty deltas. Once you've picked a base delta to start from, you will also need all deltas with higher numbers following it. Using <application>CTM</application> in your daily life To apply the deltas, simply say: &prompt.root; cd /where/ever/you/want/the/stuff &prompt.root; ctm -v -v /where/you/store/your/deltas/src-xxx.* CTM understands deltas which have been put through gzip, so you do not need to gunzip them first, this saves disk space. Unless it feels very secure about the entire process, CTM will not touch your tree. To verify a delta you can also use the flag and CTM will not actually touch your tree; it will merely verify the integrity of the delta and see if it would apply cleanly to your current tree. There are other options to CTM as well, see the manual pages or look in the sources for more information. That is really all there is to it. Every time you get a new delta, just run it through CTM to keep your sources up to date. Do not remove the deltas if they are hard to download again. You just might want to keep them around in case something bad happens. Even if you only have floppy disks, consider using fdwrite to make a copy. Keeping your local changes As a developer one would like to experiment with and change files in the source tree. CTM supports local modifications in a limited way: before checking for the presence of a file foo, it first looks for foo.ctm. If this file exists, CTM will operate on it instead of foo. This behavior gives us a simple way to maintain local changes: simply copy the files you plan to modify to the corresponding file names with a .ctm suffix. Then you can freely hack the code, while CTM keeps the .ctm file up-to-date. Other interesting <application>CTM</application> options Finding out exactly what would be touched by an update You can determine the list of changes that CTM will make on your source repository using the option to CTM. This is useful if you would like to keep logs of the changes, pre- or post- process the modified files in any manner, or just are feeling a tad paranoid :-). Making backups before updating Sometimes you may want to backup all the files that would be changed by a CTM update. Specifying the option causes CTM to backup all files that would be touched by a given CTM delta to backup-file. Restricting the files touched by an update Sometimes you would be interested in restricting the scope of a given CTM update, or may be interested in extracting just a few files from a sequence of deltas. You can control the list of files that CTM would operate on by specifying filtering regular expressions using the and options. For example, to extract an up-to-date copy of lib/libc/Makefile from your collection of saved CTM deltas, run the commands: &prompt.root; cd /where/ever/you/want/to/extract/it/ &prompt.root; ctm -e '^lib/libc/Makefile' ~ctm/src-xxx.* For every file specified in a CTM delta, the and options are applied in the order given on the command line. The file is processed by CTM only if it is marked as eligible after all the and options are applied to it. Future plans for <application>CTM</application> Tons of them: Use some kind of authentication into the CTM system, so as to allow detection of spoofed CTM updates. Clean up the options to CTM, they became confusing and counter intuitive. Miscellaneous stuff There is a sequence of deltas for the ports collection too, but interest has not been all that high yet. Tell me if you want an email list for that too and we will consider setting it up. CTM mirrors CTM/FreeBSD is available via anonymous FTP from the following mirror sites. If you choose to obtain CTM via anonymous FTP, please try to use a site near you. In case of problems, please contact &a.phk;. California, Bay Area, official source ftp://ftp.FreeBSD.org/pub/FreeBSD/development/CTM/ Germany, Trier ftp://ftp.uni-trier.de/pub/unix/systems/BSD/FreeBSD/CTM/ South Africa, backup server for old deltas ftp://ftp.za.FreeBSD.org/pub/FreeBSD/CTM/ Taiwan/R.O.C, Chiayi ftp://ctm.tw.FreeBSD.org/pub/FreeBSD/CTM/ ftp://ctm2.tw.FreeBSD.org/pub/FreeBSD/CTM/ ftp://ctm3.tw.FreeBSD.org/pub/freebsd/CTM/ If you did not find a mirror near to you or the mirror is incomplete, try FTP search at http://ftpsearch.ntnu.no/ftpsearch. FTP search is a great free archie server in Trondheim, Norway. Using CVSup Introduction CVSup is a software package for distributing and updating source trees from a master CVS repository on a remote server host. The FreeBSD sources are maintained in a CVS repository on a central development machine in California. With CVSup, FreeBSD users can easily keep their own source trees up to date. CVSup uses the so-called pull model of updating. Under the pull model, each client asks the server for updates, if and when they are wanted. The server waits passively for update requests from its clients. Thus all updates are instigated by the client. The server never sends unsolicited updates. Users must either run the CVSup client manually to get an update, or they must set up a cron job to run it automatically on a regular basis. The term CVSup, capitalized just so, refers to the entire software package. Its main components are the client cvsup which runs on each user's machine, and the server cvsupd which runs at each of the FreeBSD mirror sites. As you read the FreeBSD documentation and mailing lists, you may see references to sup. Sup was the predecessor of CVSup, and it served a similar purpose.CVSup is in used in much the same way as sup and, in fact, uses configuration files which are backward-compatible with sup's. Sup is no longer used in the FreeBSD project, because CVSup is both faster and more flexible. Installation The easiest way to install CVSup is to use the net/cvsup-bin port from the FreeBSD ports collection. If you prefer to build CVSup from source, you can use the net/cvsup port instead. But be forewarned: the net/cvsup port depends on the Modula-3 system, which takes a substantial amount of time, memory, and disk space to build. If you do not know anything about cvsup at all and want a single package which will install it, set up the configuration file and start the transfer via a pointy-clicky type of interface, then get the cvsupit package. Just hand it to &man.pkg.add.1; and it will lead you through the configuration process in a menu-oriented fashion. CVSup Configuration CVSup's operation is controlled by a configuration file called the supfile. There are some sample supfiles in the directory /usr/share/examples/cvsup/. The information in a supfile answers the following questions for cvsup: Which files do you want to receive? Which versions of them do you want? Where do you want to get them from? Where do you want to put them on your own machine? Where do you want to put your status files? In the following sections, we will construct a typical supfile by answering each of these questions in turn. First, we describe the overall structure of a supfile. A supfile is a text file. Comments begin with # and extend to the end of the line. Lines that are blank and lines that contain only comments are ignored. Each remaining line describes a set of files that the user wishes to receive. The line begins with the name of a collection, a logical grouping of files defined by the server. The name of the collection tells the server which files you want. After the collection name come zero or more fields, separated by white space. These fields answer the questions listed above. There are two types of fields: flag fields and value fields. A flag field consists of a keyword standing alone, e.g., delete or compress. A value field also begins with a keyword, but the keyword is followed without intervening white space by = and a second word. For example, release=cvs is a value field. A supfile typically specifies more than one collection to receive. One way to structure a supfile is to specify all of the relevant fields explicitly for each collection. However, that tends to make the supfile lines quite long, and it is inconvenient because most fields are the same for all of the collections in a supfile. CVSup provides a defaulting mechanism to avoid these problems. Lines beginning with the special pseudo-collection name *default can be used to set flags and values which will be used as defaults for the subsequent collections in the supfile. A default value can be overridden for an individual collection, by specifying a different value with the collection itself. Defaults can also be changed or augmented in mid-supfile by additional *default lines. With this background, we will now proceed to construct a supfile for receiving and updating the main source tree of FreeBSD-CURRENT. Which files do you want to receive? The files available via CVSup are organized into named groups called collections. The collections that are available are described here. In this example, we wish to receive the entire main source tree for the FreeBSD system. There is a single large collection src-all which will give us all of that. As a first step toward constructing our supfile, we simply list the collections, one per line (in this case, only one line): src-all Which version(s) of them do you want? With CVSup, you can receive virtually any version of the sources that ever existed. That is possible because the cvsupd server works directly from the CVS repository, which contains all of the versions. You specify which one of them you want using the tag= and value fields. Be very careful to specify any tag= fields correctly. Some tags are valid only for certain collections of files. If you specify an incorrect or misspelled tag, CVSup will delete files which you probably do not want deleted. In particular, use only tag=. for the ports-* collections. The tag= field names a symbolic tag in the repository. There are two kinds of tags, revision tags and branch tags. A revision tag refers to a specific revision. Its meaning stays the same from day to day. A branch tag, on the other hand, refers to the latest revision on a given line of development, at any given time. Because a branch tag does not refer to a specific revision, it may mean something different tomorrow than it means today. Here are the branch tags that users might be interested in. Keep in mind that only the tag=. is relevant for the ports collection. tag=. The main line of development, also known as FreeBSD-CURRENT. The . is not punctuation; it is the name of the tag. Valid for all collections. tag=RELENG_4 The line of development for FreeBSD-4.X, also known as FreeBSD-STABLE. tag=RELENG_3 The line of development for FreeBSD-3.X tag=RELENG_2_2 The line of development for FreeBSD-2.2.X, also known as 2.2-STABLE. Here are the revision tags that users might be interested in. Again, these are not valid for the ports collection. tag=RELENG_4_2_0_RELEASE FreeBSD-4.2. tag=RELENG_4_1_1_RELEASE FreeBSD-4.1.1. tag=RELENG_4_1_0_RELEASE FreeBSD-4.1. tag=RELENG_4_0_0_RELEASE FreeBSD-4.0. tag=RELENG_3_5_0_RELEASE FreeBSD-3.5. tag=RELENG_3_4_0_RELEASE FreeBSD-3.4. tag=RELENG_3_3_0_RELEASE FreeBSD-3.3. tag=RELENG_3_2_0_RELEASE FreeBSD-3.2. tag=RELENG_3_1_0_RELEASE FreeBSD-3.1. tag=RELENG_3_0_0_RELEASE FreeBSD-3.0. tag=RELENG_2_2_8_RELEASE FreeBSD-2.2.8. tag=RELENG_2_2_7_RELEASE FreeBSD-2.2.7. tag=RELENG_2_2_6_RELEASE FreeBSD-2.2.6. tag=RELENG_2_2_5_RELEASE FreeBSD-2.2.5. tag=RELENG_2_2_2_RELEASE FreeBSD-2.2.2. tag=RELENG_2_2_1_RELEASE FreeBSD-2.2.1. tag=RELENG_2_2_0_RELEASE FreeBSD-2.2.0. Be very careful to type the tag name exactly as shown. CVSup cannot distinguish between valid and invalid tags. If you misspell the tag, CVSup will behave as though you had specified a valid tag which happens to refer to no files at all. It will delete your existing sources in that case. When you specify a branch tag, you normally receive the latest versions of the files on that line of development. If you wish to receive some past version, you can do so by specifying a date with the value field. The &man.cvsup.1; manual page explains how to do that. For our example, we wish to receive FreeBSD-CURRENT. We add this line at the beginning of our supfile: *default tag=. There is an important special case that comes into play if you specify neither a tag= field nor a date= field. In that case, you receive the actual RCS files directly from the server's CVS repository, rather than receiving a particular version. Developers generally prefer this mode of operation. By maintaining a copy of the repository itself on their systems, they gain the ability to browse the revision histories and examine past versions of files. This gain is achieved at a large cost in terms of disk space, however. Where do you want to get them from? We use the host= field to tell cvsup where to obtain its updates. Any of the CVSup mirror sites will do, though you should try to select one that is close to you in cyberspace. In this example we will use a fictional FreeBSD distribution site, cvsup666.FreeBSD.org: *default host=cvsup666.FreeBSD.org You will need to change the host to one that actually exists before running CVSup. On any particular run of cvsup, you can override the host setting on the command line, with . Where do you want to put them on your own machine? The prefix= field tells cvsup where to put the files it receives. In this example, we will put the source files directly into our main source tree, /usr/src. The src directory is already implicit in the collections we have chosen to receive, so this is the correct specification: *default prefix=/usr Where should cvsup maintain its status files? The cvsup client maintains certain status files in what is called the base directory. These files help CVSup to work more efficiently, by keeping track of which updates you have already received. We will use the standard base directory, /usr/local/etc/cvsup: *default base=/usr/local/etc/cvsup This setting is used by default if it is not specified in the supfile, so we actually do not need the above line. If your base directory does not already exist, now would be a good time to create it. The cvsup client will refuse to run if the base directory does not exist. Miscellaneous supfile settings: There is one more line of boiler plate that normally needs to be present in the supfile: *default release=cvs delete use-rel-suffix compress release=cvs indicates that the server should get its information out of the main FreeBSD CVS repository. This is virtually always the case, but there are other possibilities which are beyond the scope of this discussion. delete gives CVSup permission to delete files. You should always specify this, so that CVSup can keep your source tree fully up-to-date. CVSup is careful to delete only those files for which it is responsible. Any extra files you happen to have will be left strictly alone. use-rel-suffix is ... arcane. If you really want to know about it, see the &man.cvsup.1; manual page. Otherwise, just specify it and do not worry about it. compress enables the use of gzip-style compression on the communication channel. If your network link is T1 speed or faster, you probably should not use compression. Otherwise, it helps substantially. Putting it all together: Here is the entire supfile for our example: *default tag=. *default host=cvsup666.FreeBSD.org *default prefix=/usr *default base=/usr/local/etc/cvsup *default release=cvs delete use-rel-suffix compress src-all The refuse file As mentioned above, CVSup uses a pull method. Basically, this means that you connect to the CVSup server, and it says, Here's what you can download from me..., and your client responds OK, I'll take this, this, this, and this. In the default configuration, the CVSup client will take every file associated with the collection and tag you chose in the configuration file. However, this is not always what you want, especially if you are synching the doc, ports, or www trees — most people can't read four or five languages, and therefore they don't need to download the language-specific files. If you are CVSuping the ports collection, you can get around this by specifying each collection individually (e.g., ports-astrology, ports-biology, etc instead of simply saying ports-all). However, since the doc and www trees do not have language-specific collections, you must use one of CVSup's many nifty features; the refuse file. The refuse file essentially tells CVSup that it should not take every single file from a collection; in other words, it tells the client to refuse certain files from the server. The refuse file can be found (or, if you do not yet have one, should be placed) in base/sup/refuse. base is defined in your supfile; by default, base is /usr/local/etc/cvsup, which means that by default the refuse file is in /usr/local/etc/cvsup/sup/refuse. The refuse file has a very simple format; it simply contains the names of files or directories that you do not wish to download. For example, if you cannot speak any languages other than English and some German, and you do not feel the need to use the German applications, you can put the following in your refuse file: - - ports/chinese + ports/chinese ports/german ports/japanese ports/korean ports/russian ports/vietnamese doc/es_ES.ISO8859-1 doc/ja_JP.eucJP and so forth for the other languages. Note that the name of the repository is the first directory in the refuse file. With this very useful feature, those users who are on slow links or pay by the minute for their Internet connection will be able to save valuable time as they will no longer need to download files that they will never use. For more information on refuse files and other neat features of CVSup, please view its man page. Running <application>CVSup</application> You are now ready to try an update. The command line for doing this is quite simple: &prompt.root; cvsup supfile where supfile is of course the name of the supfile you have just created. Assuming you are running under X11, cvsup will display a GUI window with some buttons to do the usual things. Press the go button, and watch it run. Since you are updating your actual /usr/src tree in this example, you will need to run the program as root so that cvsup has the permissions it needs to update your files. Having just created your configuration file, and having never used this program before, that might understandably make you nervous. There is an easy way to do a trial run without touching your precious files. Just create an empty directory somewhere convenient, and name it as an extra argument on the command line: &prompt.root; mkdir /var/tmp/dest &prompt.root; cvsup supfile /var/tmp/dest The directory you specify will be used as the destination directory for all file updates. CVSup will examine your usual files in /usr/src, but it will not modify or delete any of them. Any file updates will instead land in /var/tmp/dest/usr/src. CVSup will also leave its base directory status files untouched when run this way. The new versions of those files will be written into the specified directory. As long as you have read access to /usr/src, you do not even need to be root to perform this kind of trial run. If you are not running X11 or if you just do not like GUIs, you should add a couple of options to the command line when you run cvsup: &prompt.root; cvsup -g -L 2 supfile The tells cvsup not to use its GUI. This is automatic if you are not running X11, but otherwise you have to specify it. The tells cvsup to print out the details of all the file updates it is doing. There are three levels of verbosity, from to . The default is 0, which means total silence except for error messages. There are plenty of other options available. For a brief list of them, type cvsup -H. For more detailed descriptions, see the manual page. Once you are satisfied with the way updates are working, you can arrange for regular runs of cvsup using &man.cron.8;. Obviously, you should not let cvsup use its GUI when running it from cron. <application>CVSup</application> File Collections The file collections available via CVSup are organized hierarchically. There are a few large collections, and they are divided into smaller sub-collections. Receiving a large collection is equivalent to receiving each of its sub-collections. The hierarchical relationships among collections are reflected by the use of indentation in the list below. The most commonly used collections are src-all, and ports-all. The other collections are used only by small groups of people for specialized purposes, and some mirror sites may not carry all of them. cvs-all release=cvs The main FreeBSD CVS repository, including the cryptography code. distrib release=cvs Files related to the distribution and mirroring of FreeBSD. doc-all release=cvs Sources for the FreeBSD handbook and other documentation. ports-all release=cvs The FreeBSD ports collection. ports-archivers release=cvs Archiving tools. ports-astro release=cvs Astronomical ports. ports-audio release=cvs Sound support. ports-base release=cvs Miscellaneous files at the top of /usr/ports. ports-benchmarks release=cvs Benchmarks. ports-biology release=cvs Biology. ports-cad release=cvs Computer aided design tools. ports-chinese release=cvs Chinese language support. ports-comms release=cvs Communication software. ports-converters release=cvs character code converters. ports-databases release=cvs Databases. ports-deskutils release=cvs Things that used to be on the desktop before computers were invented. ports-devel release=cvs Development utilities. ports-editors release=cvs Editors. ports-emulators release=cvs Emulators for other operating systems. ports-ftp release=cvs FTP client and server utilities. ports-games release=cvs Games. ports-german release=cvs German language support. ports-graphics release=cvs Graphics utilities. ports-irc release=cvs Internet Relay Chat utilities. ports-japanese release=cvs Japanese language support. ports-java release=cvs Java utilities. ports-korean release=cvs Korean language support. ports-lang release=cvs Programming languages. ports-mail release=cvs Mail software. ports-math release=cvs Numerical computation software. ports-mbone release=cvs MBone applications. ports-misc release=cvs Miscellaneous utilities. ports-net release=cvs Networking software. ports-news release=cvs USENET news software. ports-palm release=cvs Software support for 3Com Palm(tm) series. ports-print release=cvs Printing software. ports-russian release=cvs Russian language support. ports-security release=cvs Security utilities. ports-shells release=cvs Command line shells. ports-sysutils release=cvs System utilities. ports-textproc release=cvs text processing utilities (does not include desktop publishing). ports-vietnamese release=cvs Vietnamese language support. ports-www release=cvs Software related to the World Wide Web. ports-x11 release=cvs Ports to support the X window system. ports-x11-clocks release=cvs X11 clocks. ports-x11-fm release=cvs X11 file managers. ports-x11-fonts release=cvs X11 fonts and font utilities. ports-x11-toolkits release=cvs X11 toolkits. ports-x11-servers X11 servers. ports-x11-wm X11 window managers. src-all release=cvs The main FreeBSD sources, including the cryptography code. src-base release=cvs Miscellaneous files at the top of /usr/src. src-bin release=cvs User utilities that may be needed in single-user mode (/usr/src/bin). src-contrib release=cvs Utilities and libraries from outside the FreeBSD project, used relatively unmodified (/usr/src/contrib). src-crypto release=cvs Cryptography utilities and libraries from outside the FreeBSD project, used relatively unmodified (/usr/src/crypto). src-eBones release=cvs Kerberos and DES (/usr/src/eBones). Not used in current releases of FreeBSD. src-etc release=cvs System configuration files (/usr/src/etc). src-games release=cvs Games (/usr/src/games). src-gnu release=cvs Utilities covered by the GNU Public License (/usr/src/gnu). src-include release=cvs Header files (/usr/src/include). src-kerberos5 release=cvs Kerberos5 security package (/usr/src/kerberos5). src-kerberosIV release=cvs KerberosIV security package (/usr/src/kerberosIV). src-lib release=cvs Libraries (/usr/src/lib). src-libexec release=cvs System programs normally executed by other programs (/usr/src/libexec). src-release release=cvs Files required to produce a FreeBSD release (/usr/src/release). src-secure release=cvs DES (/usr/src/secure). src-sbin release=cvs System utilities for single-user mode (/usr/src/sbin). src-share release=cvs Files that can be shared across multiple systems (/usr/src/share). src-sys release=cvs The kernel (/usr/src/sys). src-sys-crypto release=cvs Kernel cryptography code (/usr/src/sys/crypto). src-tools release=cvs Various tools for the maintenance of FreeBSD (/usr/src/tools). src-usrbin release=cvs User utilities (/usr/src/usr.bin). src-usrsbin release=cvs System utilities (/usr/src/usr.sbin). www release=cvs The sources for the World Wide Web data. distrib release=self The CVSup server's own configuration files. Used by CVSup mirror sites. gnats release=current The GNATS bug-tracking database. mail-archive release=current FreeBSD mailing list archive. www release=current The installed World Wide Web data. Used by WWW mirror sites. For more information For the CVSup FAQ and other information about CVSup, see The CVSup Home Page. Most FreeBSD-related discussion of CVSup takes place on the &a.hackers;. New versions of the software are announced there, as well as on the &a.announce;. Questions and bug reports should be addressed to the author of the program at cvsup-bugs@polstra.com. CVSup Sites CVSup servers for FreeBSD are running at the following sites: Argentina cvsup.ar.FreeBSD.org (maintainer msagre@cactus.fi.uba.ar) Australia cvsup.au.FreeBSD.org (maintainer dawes@xfree86.org) cvsup3.au.FreeBSD.org (maintainer FreeBSD@admin.gil.com.au) Austria cvsup.at.FreeBSD.org (maintainer postmaster@wu-wien.ac.at) Brazil cvsup.br.FreeBSD.org (maintainer cvsup@cvsup.br.FreeBSD.org) cvsup2.br.FreeBSD.org (maintainer tps@ti.sk) cvsup3.br.FreeBSD.org (maintainer camposr@matrix.com.br) Canada cvsup.ca.FreeBSD.org (maintainer dan@jaded.net) cvsup2.ca.FreeBSD.org (maintainer hostmaster@ca.freebsd.org) China cvsup.cn.FreeBSD.org (maintainer phj@cn.FreeBSD.org) Czech Republic cvsup.cz.FreeBSD.org (maintainer cejkar@dcse.fee.vutbr.cz) Denmark cvsup.dk.FreeBSD.org (maintainer jesper@skriver.dk) Estonia cvsup.ee.FreeBSD.org (maintainer taavi@uninet.ee) Finland cvsup.fi.FreeBSD.org (maintainer count@key.sms.fi) cvsup2.fi.FreeBSD.org (maintainer count@key.sms.fi) France cvsup.fr.FreeBSD.org (maintainer hostmaster@fr.FreeBSD.org) cvsup2.fr.FreeBSD.org (maintainer ftpmaint@uvsq.fr) Germany cvsup.de.FreeBSD.org (maintainer rse@freebsd.org) cvsup1.de.FreeBSD.org (maintainer wosch@FreeBSD.org) cvsup2.de.FreeBSD.org (maintainer cvsup@nikoma.de) cvsup3.de.FreeBSD.org (maintainer ag@leo.org) cvsup4.de.FreeBSD.org (maintainer cvsup@cosmo-project.de) cvsup5.de.FreeBSD.org (maintainer rse@freebsd.org) Greece cvsup.gr.FreeBSD.org (maintainer ftpadm@duth.gr) cvsup2.gr.FreeBSD.org (maintainer paschos@cs.uoi.gr) Iceland cvsup.is.FreeBSD.org (maintainer adam@veda.is) Ireland cvsup.ie.FreeBSD.org (maintainer dwmalone@maths.tcd.ie), Trinity College, Dublin. Japan cvsup.jp.FreeBSD.org (maintainer cvsupadm@jp.FreeBSD.org) cvsup2.jp.FreeBSD.org (maintainer max@FreeBSD.org) cvsup3.jp.FreeBSD.org (maintainer shige@cin.nihon-u.ac.jp) cvsup4.jp.FreeBSD.org (maintainer cvsup-admin@ftp.media.kyoto-u.ac.jp) cvsup5.jp.FreeBSD.org (maintainer cvsup@imasy.or.jp) cvsup6.jp.FreeBSD.org (maintainer cvsupadm@jp.FreeBSD.org) Korea cvsup.kr.FreeBSD.org (maintainer cjh@kr.FreeBSD.org) cvsup2.kr.FreeBSD.org (maintainer holywar@mail.holywar.net) Lithuania cvsup.lt.FreeBSD.org (maintainer domas.mituzas@delfi.lt) Netherlands cvsup.nl.FreeBSD.org (maintainer xaa@xaa.iae.nl) cvsup2.nl.FreeBSD.org (maintainer cvsup@nl.uu.net) Norway cvsup.no.FreeBSD.org (maintainer Per.Hove@math.ntnu.no) Poland cvsup.pl.FreeBSD.org (maintainer Mariusz@kam.pl) Portugal cvsup.pt.FreeBSD.org (maintainer jpedras@webvolution.net) Russia cvsup.ru.FreeBSD.org (maintainer ache@nagual.pp.ru) cvsup2.ru.FreeBSD.org (maintainer dv@dv.ru) cvsup3.ru.FreeBSD.org (maintainer fjoe@iclub.nsu.ru) cvsup4.ru.FreeBSD.org (maintainer zhecka@klondike.ru) cvsup5.ru.FreeBSD.org (maintainer maxim@macomnet.ru) cvsup6.ru.FreeBSD.org (maintainer pvr@corbina.net) Slovak Republic cvsup.sk.FreeBSD.org (maintainer tps@tps.sk) cvsup2.sk.FreeBSD.org (maintainer tps@tps.sk) Slovenia cvsup.si.FreeBSD.org (maintainer blaz@si.FreeBSD.org) South Africa cvsup.za.FreeBSD.org (maintainer markm@FreeBSD.org) cvsup2.za.FreeBSD.org (maintainer markm@FreeBSD.org) Spain cvsup.es.FreeBSD.org (maintainer jesusr@FreeBSD.org) cvsup2.es.FreeBSD.org (maintainer jesusr@FreeBSD.org) cvsup3.es.FreeBSD.org (maintainer jose@we.lc.ehu.es) Sweden cvsup.se.FreeBSD.org (maintainer pantzer@ludd.luth.se) cvsup2.se.FreeBSD.org (maintainer cvsup@dataphone.net) Taiwan cvsup.tw.FreeBSD.org (maintainer jdli@freebsd.csie.nctu.edu.tw) cvsup2.tw.FreeBSD.org (maintainer ycheng@sinica.edu.tw) cvsup3.tw.FreeBSD.org (maintainer foxfair@FreeBSD.org) Ukraine cvsup2.ua.FreeBSD.org (maintainer freebsd-mnt@lucky.net) cvsup3.ua.FreeBSD.org (maintainer ftpmaster@ukr.net), Kiev cvsup4.ua.FreeBSD.org (maintainer phantom@cris.net) United Kingdom cvsup.uk.FreeBSD.org (maintainer joe@pavilion.net) cvsup2.uk.FreeBSD.org (maintainer brian@FreeBSD.org) cvsup3.uk.FreeBSD.org (maintainer ftp-admin@plig.net) USA cvsup1.FreeBSD.org (maintainer skynyrd@opus.cts.cwu.edu), Washington state cvsup2.FreeBSD.org (maintainer jdp@FreeBSD.org), California cvsup3.FreeBSD.org (maintainer wollman@FreeBSD.org), Massachusetts cvsup4.FreeBSD.org (maintainer rgrimes@FreeBSD.org), Oregon cvsup5.FreeBSD.org (maintainer mjr@blackened.com), Arizona cvsup6.FreeBSD.org (maintainer jdp@FreeBSD.org), Florida cvsup7.FreeBSD.org (maintainer jdp@FreeBSD.org), Washington state cvsup8.FreeBSD.org (maintainer hostmaster@bigmirror.com), Washington state cvsup9.FreeBSD.org (maintainer qbsd@uswest.net), Minnesota cvsup10.FreeBSD.org (maintainer jdp@FreeBSD.org), California cvsup11.FreeBSD.org (maintainer cvsup@research.uu.net), Virginia cvsup12.FreeBSD.org (maintainer will@FreeBSD.org), Indiana cvsup13.FreeBSD.org (maintainer dima@valueclick.com), California cvsup14.FreeBSD.org (maintainer freebsd-cvsup@mfnx.net), California cvsup15.FreeBSD.org (maintainer cvsup@math.uic.edu), Illinois cvsup16.FreeBSD.org (maintainer pth3k@virginia.edu), Virginia cvsup17.FreeBSD.org (maintainer cvsup@mirrortree.com), Washington state AFS Sites AFS servers for FreeBSD are running at the following sites; Sweden The path to the files are: /afs/stacken.kth.se/ftp/pub/FreeBSD/ stacken.kth.se # Stacken Computer Club, KTH, Sweden 130.237.234.43 #hot.stacken.kth.se 130.237.237.230 #fishburger.stacken.kth.se 130.237.234.3 #milko.stacken.kth.se Maintainer ftp@stacken.kth.se diff --git a/en_US.ISO8859-1/books/handbook/multimedia/chapter.sgml b/en_US.ISO8859-1/books/handbook/multimedia/chapter.sgml index 12485dcf49..06d6817232 100644 --- a/en_US.ISO8859-1/books/handbook/multimedia/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/multimedia/chapter.sgml @@ -1,334 +1,332 @@ Sound Contributed by Moses Moore jm-moses@home.com, 20 November 2000. Synopsis This chapter of the handbook deals with setting up sound on a FreeBSD system. Locating the Correct Device PCI ISA sound cards Before you begin, you should know the model of the card you have, the chip it uses, and whether it is a PCI or ISA card. FreeBSD supports a wide variety of both PCI and ISA cards. If you do not see your card in the following list, check the &man.pcm.4; manual page. This is not a complete list; however, it does list some of the most common cards. Crystal 4237, 4236, 4232, 4231 Yamaha OPL-SAx OPTi931 Ensoniq AudioPCI 1370/1371 ESS Solo-1/1E NeoMagic 256AV/ZX Sound Blaster Pro, 16, 32, AWE64, AWE128, Live Creative ViBRA16 Advanced Asound 100, 110, and Logic ALS120 ES 1868, 1869, 1879, 1888 Gravis UltraSound Aureal Vortex 1 or 2 kernelconfiguration The driver you use in your kernel depends on the kind of card you have. The sections below provide more information and what you will need to add to your kernel configuration. Creative, Advance, and ESS Sound Cards If you have one of the above cards, you will need to add device pcm to your kernel. If you have a PnP ISA card, you will also need to add device sbc to your kernel. For a non-PnP ISA card, add device pcm and device sbc0 at isa? port0x220 irq 5 drq 1 flags 0x15 to your kernel. Those are the default settings. You may need to change the IRQ, etc. See the &man.sbc.4; man page for more information. The Sound Blaster Live is not supported under FreeBSD 4.0 without a patch, which this document will not cover. It is recommended that you update to the latest -STABLE before trying to use this card. Gravis UltraSound Cards For a PnP ISA card, you will need to add device pcm and device gusc to your kernel. If you have a non-PnP ISA card, you will need to add device pcm and device gus0 at isa? port 0x220 irq 5 drq 1 flags 0x13 to your kernel. You may need to change the IRQ, etc. See the &man.gusc.4; man page for more information. Crystal Sound Cards For Crystal cards, you will need both device pcm and device csa in your kernel. Generic Support For PnP ISA or PCI cards, you will need to add device pcm to your kernel configuration. If you have a non-PnP ISA sound card that does not have a bridge driver, you will need to add device pcm0 at isa? irq 10 drq 1 flags 0x0 to your kernel configuration. You may need to change the IRQ, etc., to match your hardware configuration. Recompiling the Kernel After adding the driver(s) you need to your kernel configuration, you will need to recompile your kernel. Please see of the handbook for more information. Creating and Testing the Device Nodes device nodes After you reboot, log in and run cat /dev/sndstat. You should see output similar to the following: FreeBSD Audio Driver (newpcm) Sep 21 2000 18:29:53 Installed devices: pcm0: <Aureal Vortex 8830> at memory 0xfeb40000 irq 5 (4p/1r +channels duplex) If you see an error message, something went wrong earlier. If that happens, go through your kernel configuration file again and make sure you chose the correct device. If it reported no errors and returned pcm0, su to root and do the following: - -&prompt.root; cd /dev + &prompt.root; cd /dev &prompt.root; sh MAKEDEV snd0 If it reported no errors and returned pcm1, su to root and do the following: - -&prompt.root; cd /dev + &prompt.root; cd /dev &prompt.root; sh MAKEDEV snd1 Please note that either of the above commands will not create a /dev/snd device! Instead it creates a group of device nodes including: Device Description /dev/audio SPARC-compatible audio device /dev/dsp Digitized voice device /dev/dspW Like /dev/dsp, but 16 bits per sample /dev/midi Raw midi access device /dev/mixer Control port mixer device /dev/music Level 2 sequencer interface /dev/sequencer Sequencer device /dev/pss Programmable device interface If all goes well, you should now have a functioning sound card. If you do not, see the next section. Common Problems device node I get an unsupported subdevice XX error! One or more of the device nodes wasn't created correctly. Repeat the steps above. I/O port I get a sb_dspwr(XX) timed out error! The I/O port is not set correctly. IRQ I get a bad irq XX error! The IRQ is set incorrectly. Make sure that the set IRQ and the sound IRQ are the same. I get a "xxx: gus pcm not attached, out of memory" error. What causes that? If this happens, it is because there is not enough available memory to use the device. diff --git a/en_US.ISO8859-1/books/handbook/ports/chapter.sgml b/en_US.ISO8859-1/books/handbook/ports/chapter.sgml index f00b8540ce..5720fb0559 100644 --- a/en_US.ISO8859-1/books/handbook/ports/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/ports/chapter.sgml @@ -1,1254 +1,1249 @@ Installing Applications: Packages and Ports Synopsis There is only so much you can do with FreeBSD. If you are an operating systems developer then the base system likely contains everything you need. If that is not what you are planning to do with FreeBSD then you will probably want to install additional software—perhaps a web server, or a mail reader, or a graphical environment such as KDE or GNOME. If you have used a Unix system before you will know that the typical procedure for installing third party software goes something like this: Download the software, which might be distributed in source code format, or as a binary. Unpack the software from its distribution format (typically a tarball compressed with either &man.compress.1; or &man.gzip.1;). Locate the documentation (perhaps a README file, or some files in a doc/ subdirectory) and read up on how to install the software. If the software was distributed in source format, compile it. This may involve editing a Makefile, or running a configure script, and other work. Test and install the software. And that is only if everything goes well. If you are installing a software package that was not deliberately ported to FreeBSD you may even have to go in and edit the code to make it work properly. Should you want to, you can continue to install software the traditional way with FreeBSD. However, FreeBSD provides two technologies which can save you a lot of effort; packages and ports. At the time of writing, over 4,000 third party applications have been made available in this way. For any given application, the FreeBSD package for that application is a single file which you must download. The package contains pre-compiled copies of all the commands for the application, as well as any configuration files or documentation. A downloaded package file can be manipulated with FreeBSD pkg_* commands, such as &man.pkg.add.1; &man.pkg.delete.1;, &man.pkg.info.1;, and so on. Installing a new application can be carried out with a single command. A FreeBSD port for an application is a collection of files designed to automate the process of compiling an application from source code. Remember that there are a number of steps you would normally carry out if you compiled a program yourself (unpacking, patching, compiling, installing). The files that make up a port contain all the necessary information to alllow the system to do this for you. You run a handful of simple commands and the source code for the application is automatically downloaded, extracted, patched, compiled, and installed for you. In fact, the ports system can also be used to generate packages which can later be manipulated with the pkg_* commands. Both packages and ports understand dependencies. Suppose you want to install an application that depends on a specific library being installed. Both the application and the library have been made available as FreeBSD ports and packages. If you use the pkg_add command or the ports system to add the application, both will notice that the library has not been installed, and the commands will install the library first. Given that the two technologies are quite similar, you might be wondering why FreeBSD bothers with both. Packages and ports both have their own strengths, and which one you use will depend on your own preference. Package benefits A compressed package tarball is typically smaller than the compressed tarball containing the source code for the application. Packages do not require any additional compilation. For large applications, such as Mozilla, KDE, or GNOME this can be important, particularly if you are on a slow system. Packages do not require you to understand any of the process involved in compiling software on FreeBSD. Ports benefits Packages are normally compiled with conservative options, because they have to run on the maximum number of systems. By installing from the port, you can tweak the compilation options to (for example) generate code that is specific to a 686 processor. Some packages have compile time options relating to what they can and can't do. For example, Apache can be configured with a wide variety of different builtin options. By building from the port you do not have to accept the default options, and can set them yourself. In some cases, multiple packages will exist for the same application to specify certain settings. For example, Ghostscript is available as a ghostscript package and a ghostscript-nox11 package, depending on whether or not you have installed an X11 server. This sort of rough tweaking is possible with packages, but rapidly becomes impossible if an application has more than one or two different compile time options. The licensing conditions of some software distributions forbid binary distribution. They must be distributed as source code. Some people do not trust binary distributions. At least with source code, you can (in theory) read through it and look for potential problems yourself. If you have local patches, you will need the source in order to apply them. Some people like having code around, so they can read it if they get bored, hack it, borrow from it (license permitting, of course), and so on. To keep track of updated ports, subscribe to freebsd-ports. The remainder of this chapter will explain how to use packages and ports to install and manage third party software on FreeBSD. Finding your application Before you can install any applications you need to know what you want, and what the application is called. FreeBSD's list of available applications is growing all the time. Currently there are over 4,000 applications available as packages or ports. There are a number of ways to find what you want. The FreeBSD web site maintains an up-to-date searchable list of all the available applications, at http://www.FreeBSD.org/ports/. The name space is divided in to categories, and you may either search for an application by name (if you know it), or you can list all the applications available in a category. Dan Langille maintains FreshPorts, at http://www.freshports.org/. FreshPorts tracks changes to the applications in the ports tree as they happen, and allows you to watch one or more ports, and will send you an e-mail when they are updated. If you do not know the name of the application you want, try using a site like FreshMeat (http://www.freshmeat.net/) or AppWatch (http://www.appwatch.com/) to find an application, then check back at the FreeBSD site to see if the application has been ported yet. Using the Packages System Contributed by &a.chern;, April 30, 2001. Installing a Package You can use the &man.pkg.add.1; utility to install a FreeBSD software package from a local file or from a server on the network. Downloading a package and then installing it locally &prompt.root; ftp ftp2.freebsd.org Connected to ftp2.freebsd.org. 220 ftp2.freebsd.org FTP server (Version 6.00LS) ready. 331 Guest login ok, send your email address as password. 230- 230- This machine is in Vienna, VA, USA, hosted by Verio. 230- Questions? E-mail freebsd@vienna.verio.net. 230- 230- 230 Guest login ok, access restrictions apply. Remote system type is UNIX. Using binary mode to transfer files. ftp> cd /pub/FreeBSD/ports/packages/irc 250 CWD command successful. ftp> get xchat-1.7.1.tgz local: xchat-1.7.1.tgz remote: xchat-1.7.1.tgz 150 Opening BINARY mode data connection for 'xchat-1.7.1.tgz' (471488 bytes). 100% |**************************************************| 460 KB 00:00 ETA 226 Transfer complete. 471488 bytes received in 5.37 seconds (85.70 KB/s) ftp> exit &prompt.root; pkg_add xchat-1.7.1.tgz -&prompt.root; - +&prompt.root; If you don't have a source of local packages (such as a FreeBSD CD-ROM set) then it will probably be easier to use the -r option to &man.pkg.add.1;. This will cause the utility to automatically determine the correct object format and release and then to fetch and install the package from an FTP site. - &prompt.root; pkg_add -r xchat-1.7.1 - + &prompt.root; pkg_add -r xchat-1.7.1 This would download the correct package and add it without any further user intervention. Package files are distributed in .tgz format. You can find them at ftp://ftp.freebsd.org/ports/packages, or on the FreeBSD CD-ROM distribution. Every CD on the FreeBSD 4-CD set (and PowerPak, etc) contains packages in the /packages directory. The layout of the packages is similar to that of the /usr/ports tree. Each category has its own directory, and every package can be found within the All directory. The directory structure of the package system is homologous to that of the ports; they work with each other to form the entire package/port system. Deleting a Package &prompt.root pkg_delete xchat-1.7.1 -&prompt.root - +&prompt.root &man.pkg.delete.1; is the utility for removing previously installed software package distributions. Managing packages &man.pkg.info.1; a utility that lists and describes the various packages installed. &prompt.root pkg_info cvsup-bin-16.1 A general network file distribution system optimized for CV docbook-1.2 Meta-port for the different versions of the DocBook DTD -... - +... &man.pkg.version.1; a utility that summarizes the versions of all installed packages. It compares the package version to the current version found in the ports tree. &prompt.root pkg_version cvsup-bin = docbook = -... - +... The symbols in the second column indicate the relative age of the installed version and the version available in the local ports tree. = The version of the installed package matches that of the one found in the local ports tree. < The installed version is older then the one available in the ports tree. >The installed version is newer than the one found in the local ports tree. (local ports tree is probably out of date) ?The installed package cannot be found in the ports index. *There are multiple versions of the package. Miscellaneous &man.pkg.add.1; &man.pkg.delete.1; &man.pkg.info.1; &man.pkg.version.1; &man.pkg.create.1; All package information is stored within the /var/db/pkg directory. The listing of contents and descriptions of each package can be found within files in this directory. Using the Ports Collection The following sections provide basic instructions on using the ports collection to install or remove programs from your system. Installing Ports The first thing that should be explained when it comes to the Ports collection is what is actually meant by a skeleton. In a nutshell, a port skeleton is a minimal set of files that are needed for a program to compile and install cleanly on FreeBSD. Each port skeleton includes: A Makefile. The Makefile contains various statements that specify how the application should be compiled and where it should be installed on your system A distinfo file. This file contains information about the files that must be downloaded to build the port, and checksums, to ensure that those files have not been corrupted during the download. A files directory. This directory contains patches to make the program compile and install on your FreeBSD system. Patches are basically small files that specify changes to particular files. They are in plain text format, and basically say Remove line 10 or Change line 26 to this .... Patches are also known as diffs because they are generated by the diff program. This directory may also contain other files used in building the port. A pkg-comment file. This is a one-line description of the program. A pkg-descr file. This is a more detailed, often multiple-line, description of the program. A pkg-plist file. This is a list of all the files that will be installed by the port. It also tells the ports system what files to remove upon deinstallation. Now that you have enough background information to know what the Ports collection is used for, you are ready to install your first port. There are two ways this can be done, and each is explained below. Before we get into that however, you will need to choose a port to install. There are a few ways to do this, with the easiest method being the ports listing on the FreeBSD web site. You can browse through the ports listed there or use the search function on the site. Each port also includes a description so you can read a bit about each port before deciding to install it. Another method is to use the whereis command. To use whereis, simply type whereis <program you want to install> at the prompt, and if it is found on your system, you will be told where it is, like so: &prompt.root; whereis xchat xchat: /usr/ports/irc/xchat &prompt.root; This tells us that xchat (an irc client) can be found in the /usr/ports/irc/xchat directory. Yet another way of finding a particular port is by using the Ports collection's built-in search mechanism. To use the search feature, you will need to be in the /usr/ports directory. Once in that directory, run make search key=program-name where program-name is the name of the program you want to find. For example, if you were looking for xchat: &prompt.root; cd /usr/ports &prompt.root; make search key=xchat Port: xchat-1.3.8 Path: /usr/ports/irc/xchat Info: An X11 IRC client using the GTK+ toolkit, and optionally, GNOME Maint: jim@FreeBSD.org Index: irc B-deps: XFree86-3.3.5 bzip2-0.9.5d gettext-0.10.35 giflib-4.1.0 glib-1.2.6 gmake-3.77 gtk-1.2.6 imlib-1.9.8 jpeg-6b png-1.0.3 tiff-3.5.1 R-deps: XFree86-3.3.5 gettext-0.10.35 giflib-4.1.0 glib-1.2.6 gtk-1.2.6 imlib-1.9.8 jpeg-6b png-1.0.3 tiff-3.5.1 The part of the output you want to pay particular attention to is the Path: line, since that tells you where to find it. The other information provided is not needed in order to install the port directly, so it will not be covered here. You must be the root user to install ports. Now that you have found a port you would like to install, you are ready to do the actual installation. Installing ports from a CDROM As you may have guessed from the title, everything described in this section assumes you have a FreeBSD CDROM set. If you do not, you can order one from the FreeBSD Mall. Assuming that your FreeBSD CDROM is in the drive and is mounted on /cdrom (and the mount point must be /cdrom), you are ready to install the port. To begin, change directories to the directory where the port you want to install lives: &prompt.root; cd /usr/ports/irc/xchat Once inside the xchat directory, you will see the port skeleton. The next step is to compile (also called build) the port. This is done by simply typing make at the prompt. Once you have done so, you should see something like this: &prompt.root; make >> xchat-1.3.8.tar.bz2 doesn't seem to exist on this system. >> Attempting to fetch from file:/cdrom/ports/distfiles/. ===> Extracting for xchat-1.3.8 >> Checksum OK for xchat-1.3.8.tar.bz2. ===> xchat-1.3.8 depends on executable: bzip2 - found ===> xchat-1.3.8 depends on executable: gmake - found ===> xchat-1.3.8 depends on shared library: gtk12.2 - found ===> xchat-1.3.8 depends on shared library: Imlib.5 - found ===> xchat-1.3.8 depends on shared library: X11.6 - found ===> Patching for xchat-1.3.8 ===> Applying FreeBSD patches for xchat-1.3.8 ===> Configuring for xchat-1.3.8 ... [configure output snipped] ... ===> Building for xchat-1.3.8 ... [compilation snipped] ... &prompt.root; Take notice that once the compile is complete you are returned to your prompt. The next step is to install the port. In order to install it, you simply need to tack one word onto the make command, and that word is install: &prompt.root; make install ===> Installing for xchat-1.3.8 ===> xchat-1.3.8 depends on shared library: gtk12.2 - found ===> xchat-1.3.8 depends on shared library: Imlib.5 - found ===> xchat-1.3.8 depends on shared library: X11.6 - found ... [install routines snipped] ... ===> Generating temporary packing list ===> Installing xchat docs in /usr/X11R6/share/doc/xchat ===> Registering installation for xchat-1.3.8 &prompt.root; Once you are returned to your prompt, you should be able to run the application you just installed. You can save an extra step by just running make install instead of make and make install as two separate steps. Please be aware that the licenses of a few ports do not allow for inclusion on the CDROM. This could be for various reasons, including things such as registration form needs to be filled out before downloading, if redistribution is not allowed, and so on. If you wish to install a port not included on the CDROM, you will need to be online in order to do so (see the next section). Installing ports from the Internet As with the last section, this section makes an assumption that you have a working Internet connection. If you do not, you will need to do the CDROM installation. Installing a port from the Internet is done exactly the same way as it would be if you were installing from a CDROM. The only difference between the two is that the program's source code is downloaded from the Internet instead of pulled from the CDROM. The steps involved are identical: &prompt.root; make install >> xchat-1.3.8.tar.bz2 doesn't seem to exist on this system. >> Attempting to fetch from http://xchat.org/files/v1.3/. Receiving xchat-1.3.8.tar.bz2 (305543 bytes): 100% 305543 bytes transferred in 2.9 seconds (102.81 Kbytes/s) ===> Extracting for xchat-1.3.8 >> Checksum OK for xchat-1.3.8.tar.bz2. ===> xchat-1.3.8 depends on executable: bzip2 - found ===> xchat-1.3.8 depends on executable: gmake - found ===> xchat-1.3.8 depends on shared library: gtk12.2 - found ===> xchat-1.3.8 depends on shared library: Imlib.5 - found ===> xchat-1.3.8 depends on shared library: X11.6 - found ===> Patching for xchat-1.3.8 ===> Applying FreeBSD patches for xchat-1.3.8 ===> Configuring for xchat-1.3.8 ... [configure output snipped] ... ===> Building for xchat-1.3.8 ... [compilation snipped] ... ===> Installing for xchat-1.3.8 ===> xchat-1.3.8 depends on shared library: gtk12.2 - found ===> xchat-1.3.8 depends on shared library: Imlib.5 - found ===> xchat-1.3.8 depends on shared library: X11.6 - found ... [install routines snipped] ... ===> Generating temporary packing list ===> Installing xchat docs in /usr/X11R6/share/doc/xchat ===> Registering installation for xchat-1.3.8 &prompt.root; As you can see, the only difference is the line that tells you where the system is fetching the port from. That about does it for installing ports onto your system. In the section you will learn how to remove a port from your system. Removing Installed Ports Now that you know how to install ports, you are probably wondering how to remove them, just in case you install one and later on you decide that you installed the wrong port. The next few paragraphs will cover just that. Now we will remove our previous example (which was xchat for those of you not paying attention). As with installing ports, the first thing you must do is change to the port directory, which if you remember was /usr/ports/irc/xchat. After you change directories, you are ready to uninstall xchat. This is done with the make deinstall command (makes sense right?): &prompt.root; cd /usr/ports/irc/xchat &prompt.root; make deinstall ===> Deinstalling for xchat-1.3.8 &prompt.root; That was easy enough. You have now managed to remove xchat from your system. If you would like to reinstall it, you can do so by running make reinstall from the /usr/ports/irc/xchat directory. Troubleshooting The following sections cover some of the more frequently asked questions about the Ports collection and some basic troubleshooting techniques, and what do to if a port is broken. Some Questions and Answers I thought this was going to be a discussion about modems??! Ah, you must be thinking of the serial ports on the back of your computer. We are using port here to mean the result of porting a program from one version of UNIX to another. What is a patch? A patch is a small file that specifies how to go from one version of a file to another. It contains plain text, and basically says things like delete line 23, add these two lines after line 468, or change line 197 to this. They are also known as diffs because they are generated by the diff program. What is all this about tarballs? It is a file ending in .tar, or with variations such as .tar.gz, .tar.Z, .tar.bz2, and even .tgz. Basically, it is a directory tree that has been archived into a single file (.tar) and optionally compressed (.gz). This technique was originally used for Tape ARchives (hence the name tar), but it is a widely used way of distributing program source code around the Internet. You can see what files are in them, or even extract them yourself by using the standard UNIX tar program, which comes with the base FreeBSD system, like this: &prompt.user; tar tvzf foobar.tar.gz &prompt.user; tar xzvf foobar.tar.gz &prompt.user; tar tvf foobar.tar &prompt.user; tar xvf foobar.tar And a checksum? It is a number generated by adding up all the data in the file you want to check. If any of the characters change, the checksum will no longer be equal to the total, so a simple comparison will allow you to spot the difference. I did what you said for compiling ports from a CDROM and it worked great until I tried to install the kermit port. &prompt.root; make install >> cku190.tar.gz doesn't seem to exist on this system. >> Attempting to fetch from ftp://kermit.columbia.edu/kermit/archives/. Why can it not be found? Have I got a dud CDROM? As was explained in the compiling ports from CDROM section, some ports cannot be put on the CDROM set due to licensing restrictions. Kermit is an example of that. The licensing terms for kermit do not allow us to put the tarball for it on the CDROM, so you will have to fetch it by hand—sorry! The reason why you got all those error messages was because you were not connected to the Internet at the time. Once you have downloaded it from any of the MASTER_SITES (listed in the Makefile), you can restart the install process. I did that, but when I tried to put it into /usr/ports/distfiles I got some error about not having permission. The ports mechanism looks for the tarball in /usr/ports/distfiles, but you will not be able to copy anything there because it is symlinked to the CDROM, which is read-only. You can tell it to look somewhere else by doing: &prompt.root; make DISTDIR=/where/you/put/it install Does the ports scheme only work if you have everything in /usr/ports? My system administrator says I must put everything under /u/people/guests/wurzburger, but it does not seem to work. You can use the PORTSDIR and PREFIX variables to tell the ports mechanism to use different directories. For instance, &prompt.root; make PORTSDIR=/u/people/guests/wurzburger/ports install will compile the port in /u/people/guests/wurzburger/ports and install everything under /usr/local. &prompt.root; make PREFIX=/u/people/guests/wurzburger/local install will compile it in /usr/ports and install it in /u/people/guests/wurzburger/local. And of course, &prompt.root; make PORTSDIR=../ports PREFIX=../local install will combine the two (it is too long to write fully on the page, but it should give you the general idea). Some ports that use &man.imake.1; (a part of the X Windows System) don't work well with PREFIX, and will insist on installing under /usr/X11R6. Similarly, some Perl ports ignore PREFIX and install in the Perl tree. Making these ports respect PREFIX is a difficult or impossible job. If you do not fancy typing all that in every time you install a port, it is a good idea to put these variables into your environment. Read the man page for your shell for instructions on doing so. I do not have a FreeBSD CDROM, but I would like to have all the tarballs handy on my system so I do not have to wait for a download every time I install a port. Is there any way to get them all at once? To get every single tarball for the Ports collection, do: &prompt.root; cd /usr/ports &prompt.root; make fetch For all the tarballs for a single ports directory, do: &prompt.root; cd /usr/ports/directory &prompt.root; make fetch and for just one port—well, you have probably guessed already. I know it is probably faster to fetch the tarballs from one of the FreeBSD mirror sites close by. Is there any way to tell the port to fetch them from servers other than the ones listed in the MASTER_SITES? Yes. If you know, for example, that ftp.FreeBSD.org is much closer to you than the sites listed in MASTER_SITES, do as follows: &prompt.root; cd /usr/ports/directory &prompt.root; make MASTER_SITE_OVERRIDE= \ ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/ fetch I want to know what files make is going to need before it tries to pull them down. make fetch-list will display a list of the files needed for a port. Is there any way to stop the port from compiling? I want to do some hacking on the source before I install it, but it is a bit tiresome to watch it and hit control-C every time. Doing make extract will stop it after it has fetched and extracted the source code. I am trying to make my own port and I want to be able to stop it compiling until I have had a chance to see if my patches worked properly. Is there something like make extract, but for patches? Yep, make patch is what you want. You will probably find the PATCH_DEBUG option useful as well. And by the way, thank you for your efforts! I have heard that some compiler options can cause bugs. Is this true? How can I make sure that I compile ports with the right settings? Yes, with version 2.6.3 of gcc (the version shipped with FreeBSD 2.1.0 and 2.1.5), the option could result in buggy code unless you used the option as well. (Most of the ports do not use ). You should be able to specify the compiler options used by something like: &prompt.root; make CFLAGS='-O2 -fno-strength-reduce' install or by editing /etc/make.conf, but unfortunately not all ports respect this. The surest way is to do make configure, then go into the source directory and inspect the Makefiles by hand, but this can get tedious if the source has lots of sub-directories, each with their own Makefiles. The default FreeBSD compiler options are quite conservative, so if you have not changed them you should not have any problems. There are so many ports it is hard to find the one I want. Is there a list anywhere of what ports are available? Look in the INDEX file in /usr/ports. If you would like to search the ports collection for a keyword, you can do that too. For example, you can find ports relevant to the LISP programming language using: &prompt.user; cd /usr/ports &prompt.user; make search key=lisp I went to install the foo port but the system suddenly stopped compiling it and starting compiling the bar port. What is going on? The foo port needs something that is supplied with bar — for instance, if foo uses graphics, bar might have a library with useful graphics processing routines. Or bar might be a tool that is needed to compile the foo port. I installed the grizzle program from the ports and frankly it is a complete waste of disk space. I want to delete it but I do not know where it put all the files. Any clues? No problem, just do: &prompt.root; pkg_delete grizzle-6.5 Alternatively, you can do: &prompt.root; cd /usr/ports/somewhere/grizzle &prompt.root; make deinstall Hang on a minute, you have to know the version number to use that command. You do not seriously expect me to remember that, do you?? Not at all, you can find it out by doing: &prompt.root; pkg_info -I 'grizzle*' Information for grizzle-6.5: grizzle-6.5 - the combined piano tutorial, LOGO interpreter and shoot 'em up arcade game. Talking of disk space, the ports directory seems to be taking up an awful lot of room. Is it safe to go in there and delete things? Yes, if you have installed the program and are fairly certain you will not need the source again, there is no point in keeping it hanging around. The best way to do this is: &prompt.root; cd /usr/ports &prompt.root; make clean which will go through all the ports subdirectories and delete everything except the skeletons for each port. I tried that and it still left all those tarballs or whatever you called them in the distfiles directory. Can I delete those as well? Yes, if you are sure you have finished with them, those can go as well. They can be removed manually, or by using make distclean. I like having lots and lots of programs to play with. Is there any way of installing all the ports in one go? Just do: &prompt.root; cd /usr/ports &prompt.root; make install Be careful, as some ports may install files with the same name. If you install two graphics ports and they both install /usr/local/bin/plot then you will obviously have problems. OK, I tried that, but I thought it would take a very long time so I went to bed and left it to get on with it. When I looked at the computer this morning, it had only done three and a half ports. Did something go wrong? No, the problem is that some of the ports need to ask you questions that we cannot answer for you (e.g., Do you want to print on A4 or US letter sized paper?) and they need to have someone on hand to answer them. I really do not want to spend all day staring at the monitor. Any better ideas? OK, do this before you go to bed/work/the local park: &prompt.root cd /usr/ports &prompt.root; make -DBATCH install This will install every port that does not require user input. Then, when you come back, do: &prompt.root; cd /usr/ports &prompt.root; make -DIS_INTERACTIVE install to finish the job. At work, we are using frobble, which is in your Ports collection, but we have altered it quite a bit to get it to do what we need. Is there any way of making our own packages, so we can distribute it more easily around our sites? No problem, assuming you know how to make patches for your changes: &prompt.root; cd /usr/ports/somewhere/frobble &prompt.root; make extract &prompt.root; cd work/frobble-2.8 [Apply your patches] &prompt.root; cd ../.. &prompt.root; make package This ports stuff is really clever. I am desperate to find out how you did it. What is the secret? Nothing secret about it at all, just look at the bsd.port.mk and bsd.port.subdir.mk files in your makefiles directory. (Readers with an aversion to intricate shell-scripts are advised not to follow this link...) Help! This port is broken! If you come across a port that doesn't work for you, there are a few things you can do, including: Fix it! The how to make a port section should help you do this. Gripe—by email only! Send email to the maintainer of the port first. Type make maintainer or read the Makefile to find the maintainer's email address. Remember to include the name and version of the port (send the $FreeBSD: line from the Makefile) and the output leading up to the error when you email the maintainer. If you do not get a response from the maintainer, you can use send-pr to submit a bug report. Forget about it. This is the easiest route—very few ports can be classified as essential. There's also a good chance any problems will be fixed in the next version when the port is updated. Grab the package from an ftp site near you. The master package collection is on ftp.FreeBSD.org in the packages directory, but be sure to check your local mirror first! These are more likely to work than trying to compile from source and are a lot faster as well. Use the &man.pkg.add.1; program to install the package on your system. Advanced Topics The documentation that was here has been moved to its own Porter's Handbook for ease of reference. Please go there if you wish to create and submit your own ports. diff --git a/en_US.ISO8859-1/books/handbook/security/chapter.sgml b/en_US.ISO8859-1/books/handbook/security/chapter.sgml index 3aa8343290..a207a7cbec 100644 --- a/en_US.ISO8859-1/books/handbook/security/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/security/chapter.sgml @@ -1,3064 +1,3033 @@ Security security Much of this chapter has been taken from the &man.security.7; man page, originally written by &a.dillon;. Synopsis The following chapter will provide a basic introduction to system security concepts, some general good rules of thumb, and some advanced topics such as S/Key, OpenSSL, Kerberos, and others. Introduction Security is a function that begins and ends with the system administrator. While all BSD UNIX multi-user systems have some inherent security, the job of building and maintaining additional security mechanisms to keep those users honest is probably one of the single largest undertakings of the sysadmin. Machines are only as secure as you make them, and security concerns are ever competing with the human necessity for convenience. UNIX systems, in general, are capable of running a huge number of simultaneous processes and many of these processes operate as servers – meaning that external entities can connect and talk to them. As yesterday's mini-computers and mainframes become today's desktops, and as computers become networked and internetworked, security becomes an ever bigger issue. Security is best implemented through a layered onion approach. In a nutshell, what you want to do is to create as many layers of security as are convenient and then carefully monitor the system for intrusions. You do not want to overbuild your security or you will interfere with the detection side, and detection is one of the single most important aspects of any security mechanism. For example, it makes little sense to set the schg flags (see &man.chflags.1;) on every system binary because while this may temporarily protect the binaries, it prevents an attacker who has broken in from making an easily detectable change that may result in your security mechanisms not detecting the attacker at all. System security also pertains to dealing with various forms of attack, including attacks that attempt to crash or otherwise make a system unusable but do not attempt to break root. Security concerns can be split up into several categories: Denial of service attacks. User account compromises. Root compromise through accessible servers. Root compromise via user accounts. Backdoor creation. DOS attacks security DOS attacks Denial of Service A denial of service attack is an action that deprives the machine of needed resources. Typically, D.O.S. attacks are brute-force mechanisms that attempt to crash or otherwise make a machine unusable by overwhelming its servers or network stack. Some D.O.S. attacks try to take advantages of bugs in the networking stack to crash a machine with a single packet. The latter can only be fixed by applying a bug fix to the kernel. Attacks on servers can often be fixed by properly specifying options to limit the load the servers incur on the system under adverse conditions. Brute-force network attacks are harder to deal with. A spoofed-packet attack, for example, is nearly impossible to stop short of cutting your system off from the Internet. It may not be able to take your machine down, but it can saturate your Internet connection. security account compromises A user account compromise is even more common than a D.O.S. attack. Many sysadmins still run standard telnetd, rlogind, rshd, and ftpd servers on their machines. These servers, by default, do not operate over encrypted connections. The result is that if you have any moderate-sized user base, one or more of your users logging into your system from a remote location (which is the most common and convenient way to login to a system) will have his or her password sniffed. The attentive system admin will analyze his remote access logs looking for suspicious source addresses even for successful logins. One must always assume that once an attacker has access to a user account, the attacker can break root. However, the reality is that in a well secured and maintained system, access to a user account does not necessarily give the attacker access to root. The distinction is important because without access to root the attacker cannot generally hide his tracks and may, at best, be able to do nothing more than mess with the user's files or crash the machine. User account compromises are very common because users tend not to take the precautions that sysadmins take. security backdoors System administrators must keep in mind that there are potentially many ways to break root on a machine. The attacker may know the root password, the attacker may find a bug in a root-run server and be able to break root over a network connection to that server, or the attacker may know of a bug in an suid-root program that allows the attacker to break root once he has broken into a user's account. If an attacker has found a a way to break root on a machine, the attacker may not have a need to install a backdoor. Many of the root holes found and closed to date involve a considerable amount of work by the attacker to cleanup after himself, so most attackers install backdoors. Backdoors provide the attacker with a way to easily regain root access to the system, but it also gives the smart system administrator a convenient way to detect the intrusion. Making it impossible for an attacker to install a backdoor may actually be detrimental to your security because it will not close off the hole the attacker found to break in the first place. Security remedies should always be implemented with a multi-layered onion peel approach and can be categorized as follows: Securing root and staff accounts. Securing root – root-run servers and suid/sgid binaries. Securing user accounts. Securing the password file. Securing the kernel core, raw devices, and filesystems. Quick detection of inappropriate changes made to the system. Paranoia. The next section of this chapter will cover the above bullet items in greater depth. security securing Securing FreeBSD The sections that follow will cover the methods of securing your FreeBSD system that were mentioned in the last section of this chapter. Securing the root account and staff accounts su First off, do not bother securing staff accounts if you have not secured the root account. Most systems have a password assigned to the root account. The first thing you do is assume that the password is always compromised. This does not mean that you should remove the password. The password is almost always necessary for console access to the machine. What it does mean is that you should not make it possible to use the password outside of the console or possibly even with the &man.su.1; command. For example, make sure that your pty's are specified as being unsecure in the /etc/ttys file so that direct root logins via telnet or rlogin are disallowed. If using other login services such as sshd, make sure that direct root logins are disabled there as well. Consider every access method – services such as FTP often fall through the cracks. Direct root logins should only be allowed via the system console. wheel Of course, as a sysadmin you have to be able to get to root, so we open up a few holes. But we make sure these holes require additional password verification to operate. One way to make root accessible is to add appropriate staff accounts to the wheel group (in /etc/group). The staff members placed in the wheel group are allowed to su to root. You should never give staff members native wheel access by putting them in the wheel group in their password entry. Staff accounts should be placed in a staff group, and then added to the wheel group via the /etc/group file. Only those staff members who actually need to have root access should be placed in the wheel group. It is also possible, when using an authentication method such as kerberos, to use kerberos' .k5login file in the root account to allow a &man.ksu.1; to root without having to place anyone at all in the wheel group. This may be the better solution since the wheel mechanism still allows an intruder to break root if the intruder has gotten hold of your password file and can break into a staff account. While having the wheel mechanism is better than having nothing at all, it is not necessarily the safest option. An indirect way to secure staff accounts, and ultimately root access is to use an alternative login access method and do what is known as *'ing out the crypted password for the staff accounts. Using the &man.vipw.8; command, one can replace each instance of a crypted password with a single * character. This command will update the /etc/master.passwd file and user/password database to disable password-authenticated logins. A staff account entry such as: foobar:R9DT/Fa1/LV9U:1000:1000::0:0:Foo Bar:/home/foobar:/usr/local/bin/tcsh Should be changed to this : foobar:*:1000:1000::0:0:Foo Bar:/home/foobar:/usr/local/bin/tcsh This change will prevent normal logins from occurring, since the encrypted password will never match *. With this done, staff members must use another mechanism to authenticate themselves such as &man.kerberos.1; or &man.ssh.1; using a public/private key pair. When using something like kerberos, one generally must secure the machines which run the kerberos servers and your desktop workstation. When using a public/private key pair with ssh, one must generally secure the machine used to login from (typically one's workstation). An additional layer of protection can be added to the key pair by password protecting the key pair when creating it with &man.ssh-keygen.1;. Being able to * out the passwords for staff accounts also guarantees that staff members can only login through secure access methods that you have setup. This forces all staff members to use secure, encrypted connections for all of their sessions which closes an important hole used by many intruders: That of sniffing the network from an unrelated, less secure machine. The more indirect security mechanisms also assume that you are logging in from a more restrictive server to a less restrictive server. For example, if your main box is running all sorts of servers, your workstation should not be running any. In order for your workstation to be reasonably secure you should run as few servers as possible, up to and including no servers at all, and you should run a password-protected screen blanker. Of course, given physical access to a workstation an attacker can break any sort of security you put on it. This is definitely a problem that you should consider but you should also consider the fact that the vast majority of break-ins occur remotely, over a network, from people who do not have physical access to your workstation or servers. Kerberos Using something like kerberos also gives you the ability to disable or change the password for a staff account in one place and have it immediately effect all the machine the staff member may have an account on. If a staff member's account gets compromised, the ability to instantly change his password on all machines should not be underrated. With discrete passwords, changing a password on N machines can be a mess. You can also impose re-passwording restrictions with kerberos: not only can a kerberos ticket be made to timeout after a while, but the kerberos system can require that the user choose a new password after a certain period of time (say, once a month). Securing Root-run Servers and SUID/SGID Binaries ntalk comsat finger sandboxes sshd telnetd rshd rlogind The prudent sysadmin only runs the servers he needs to, no more, no less. Be aware that third party servers are often the most bug-prone. For example, running an old version of imapd or popper is like giving a universal root ticket out to the entire world. Never run a server that you have not checked out carefully. Many servers do not need to be run as root. For example, the ntalk, comsat, and finger daemons can be run in special user sandboxes. A sandbox isn't perfect unless you go to a large amount of trouble, but the onion approach to security still stands: If someone is able to break in through a server running in a sandbox, they still have to break out of the sandbox. The more layers the attacker must break through, the lower the likelihood of his success. Root holes have historically been found in virtually every server ever run as root, including basic system servers. If you are running a machine through which people only login via sshd and never login via telnetd or rshd or rlogind, then turn off those services! FreeBSD now defaults to running ntalkd, comsat, and finger in a sandbox. Another program which may be a candidate for running in a sandbox is &man.named.8;. /etc/defaults/rc.conf includes the arguments necessary to run named in a sandbox in a commented-out form. Depending on whether you are installing a new system or upgrading an existing system, the special user accounts used by these sandboxes may not be installed. The prudent sysadmin would research and implement sandboxes for servers whenever possible. sendmail There are a number of other servers that typically do not run in sandboxes: sendmail, popper, imapd, ftpd, and others. There are alternatives to some of these, but installing them may require more work than you are willing to perform (the convenience factor strikes again). You may have to run these servers as root and rely on other mechanisms to detect break-ins that might occur through them. The other big potential root hole in a system are the suid-root and sgid binaries installed on the system. Most of these binaries, such as rlogin, reside in /bin, /sbin, /usr/bin, or /usr/sbin. While nothing is 100% safe, the system-default suid and sgid binaries can be considered reasonably safe. Still, root holes are occasionally found in these binaries. A root hole was found in Xlib in 1998 that made xterm (which is typically suid) vulnerable. It is better to be safe than sorry and the prudent sysadmin will restrict suid binaries that only staff should run to a special group that only staff can access, and get rid of (chmod 000) any suid binaries that nobody uses. A server with no display generally does not need an xterm binary. Sgid binaries can be almost as dangerous. If an intruder can break an sgid-kmem binary the intruder might be able to read /dev/kmem and thus read the crypted password file, potentially compromising any passworded account. Alternatively an intruder who breaks group kmem can monitor keystrokes sent through pty's, including pty's used by users who login through secure methods. An intruder that breaks the tty group can write to almost any user's tty. If a user is running a terminal program or emulator with a keyboard-simulation feature, the intruder can potentially generate a data stream that causes the user's terminal to echo a command, which is then run as that user. Securing User Accounts User accounts are usually the most difficult to secure. While you can impose Draconian access restrictions on your staff and * out their passwords, you may not be able to do so with any general user accounts you might have. If you do have sufficient control then you may win out and be able to secure the user accounts properly. If not, you simply have to be more vigilant in your monitoring of those accounts. Use of ssh and kerberos for user accounts is more problematic due to the extra administration and technical support required, but still a very good solution compared to a crypted password file. Securing the Password File The only sure fire way is to * out as many passwords as you can and use ssh or kerberos for access to those accounts. Even though the crypted password file (/etc/spwd.db) can only be read by root, it may be possible for an intruder to obtain read access to that file even if the attacker cannot obtain root-write access. Your security scripts should always check for and report changes to the password file (see Checking file integrity below). Securing the Kernel Core, Raw Devices, and Filesystems If an attacker breaks root he can do just about anything, but there are certain conveniences. For example, most modern kernels have a packet sniffing device driver built in. Under FreeBSD it is called the bpf device. An intruder will commonly attempt to run a packet sniffer on a compromised machine. You do not need to give the intruder the capability and most systems should not have the bpf device compiled in. sysctl But even if you turn off the bpf device, you still have /dev/mem and /dev/kmem to worry about. For that matter, the intruder can still write to raw disk devices. Also, there is another kernel feature called the module loader, &man.kldload.8;. An enterprising intruder can use a KLD module to install his own bpf device or other sniffing device on a running kernel. To avoid these problems you have to run the kernel at a higher secure level, at least securelevel 1. The securelevel can be set with a sysctl on the kern.securelevel variable. Once you have set the securelevel to 1, write access to raw devices will be denied and special chflags flags, such as schg, will be enforced. You must also ensure that the schg flag is set on critical startup binaries, directories, and script files – everything that gets run up to the point where the securelevel is set. This might be overdoing it, and upgrading the system is much more difficult when you operate at a higher secure level. You may compromise and run the system at a higher secure level but not set the schg flag for every system file and directory under the sun. Another possibility is to simply mount / and /usr read-only. It should be noted that being too draconian in what you attempt to protect may prevent the all-important detection of an intrusion. Checking File Integrity: Binaries, Configuration Files, Etc. When it comes right down to it, you can only protect your core system configuration and control files so much before the convenience factor rears its ugly head. For example, using chflags to set the schg bit on most of the files in / and /usr is probably counterproductive because while it may protect the files, it also closes a detection window. The last layer of your security onion is perhaps the most important – detection. The rest of your security is pretty much useless (or, worse, presents you with a false sense of safety) if you cannot detect potential incursions. Half the job of the onion is to slow down the attacker rather than stop him in order to give the detection side of the equation a chance to catch him in the act. The best way to detect an incursion is to look for modified, missing, or unexpected files. The best way to look for modified files is from another (often centralized) limited-access system. Writing your security scripts on the extra-secure limited-access system makes them mostly invisible to potential attackers, and this is important. In order to take maximum advantage you generally have to give the limited-access box significant access to the other machines in the business, usually either by doing a read-only NFS export of the other machines to the limited-access box, or by setting up ssh key-pairs to allow the limit-access box to ssh to the other machines. Except for its network traffic, NFS is the least visible method – allowing you to monitor the filesystems on each client box virtually undetected. If your limited-access server is connected to the client boxes through a switch, the NFS method is often the better choice. If your limited-access server is connected to the client boxes through a hub or through several layers of routing, the NFS method may be too insecure (network-wise) and using ssh may be the better choice even with the audit-trail tracks that ssh lays. Once you give a limit-access box at least read access to the client systems it is supposed to monitor, you must write scripts to do the actual monitoring. Given an NFS mount, you can write scripts out of simple system utilities such as &man.find.1; and &man.md5.1;. It is best to physically md5 the client-box files boxes at least once a day, and to test control files such as those found in /etc and /usr/local/etc even more often. When mismatches are found relative to the base md5 information the limited-access machine knows is valid, it should scream at a sysadmin to go check it out. A good security script will also check for inappropriate suid binaries and for new or deleted files on system partitions such as / and /usr. When using ssh rather than NFS, writing the security script is much more difficult. You essentially have to scp the scripts to the client box in order to run them, making them visible, and for safety you also need to scp the binaries (such as find) that those scripts use. The ssh daemon on the client box may already be compromised. All in all, using ssh may be necessary when running over unsecure links, but it's also a lot harder to deal with. A good security script will also check for changes to user and staff members access configuration files: .rhosts, .shosts, .ssh/authorized_keys and so forth… files that might fall outside the purview of the MD5 check. If you have a huge amount of user disk space it may take too long to run through every file on those partitions. In this case, setting mount flags to disallow suid binaries and devices on those partitions is a good idea. The nodev and nosuid options (see &man.mount.8;) are what you want to look into. You should probably scan them anyway at least once a week, since the object of this layer is to detect a break-in whether or not the break-in is effective. Process accounting (see &man.accton.8;) is a relatively low-overhead feature of the operating system which might help as a post-break-in evaluation mechanism. It is especially useful in tracking down how an intruder has actually broken into a system, assuming the file is still intact after the break-in occurs. Finally, security scripts should process the log files and the logs themselves should be generated in as secure a manner as possible – remote syslog can be very useful. An intruder tries to cover his tracks, and log files are critical to the sysadmin trying to track down the time and method of the initial break-in. One way to keep a permanent record of the log files is to run the system console to a serial port and collect the information on a continuing basis through a secure machine monitoring the consoles. Paranoia A little paranoia never hurts. As a rule, a sysadmin can add any number of security features as long as they do not effect convenience, and can add security features that do effect convenience with some added thought. Even more importantly, a security administrator should mix it up a bit – if you use recommendations such as those given by this document verbatim, you give away your methodologies to the prospective attacker who also has access to this document. Denial of Service Attacks DOS attacks This section covers Denial of Service attacks. A DOS attack is typically a packet attack. While there is not much you can do about modern spoofed packet attacks that saturate your network, you can generally limit the damage by ensuring that the attacks cannot take down your servers. Limiting server forks. Limiting springboard attacks (ICMP response attacks, ping broadcast, etc.). Kernel Route Cache. A common DOS attack is against a forking server that attempts to cause the server to eat processes, file descriptors, and memory until the machine dies. Inetd (see &man.inetd.8;) has several options to limit this sort of attack. It should be noted that while it is possible to prevent a machine from going down it is not generally possible to prevent a service from being disrupted by the attack. Read the inetd manual page carefully and pay specific attention to the , , and options. Note that spoofed-IP attacks will circumvent the option to inetd, so typically a combination of options must be used. Some standalone servers have self-fork-limitation parameters. Sendmail has its option which tends to work much better than trying to use sendmail's load limiting options due to the load lag. You should specify a MaxDaemonChildren parameter when you start sendmail high enough to handle your expected load but no so high that the computer cannot handle that number of sendmails without falling on its face. It is also prudent to run sendmail in queued mode () and to run the daemon (sendmail -bd) separate from the queue-runs (sendmail -q15m). If you still want real-time delivery you can run the queue at a much lower interval, such as , but be sure to specify a reasonable MaxDaemonChildren option for that sendmail to prevent cascade failures. Syslogd can be attacked directly and it is strongly recommended that you use the option whenever possible, and the option otherwise. You should also be fairly careful with connect-back services such as tcpwrapper's reverse-identd, which can be attacked directly. You generally do not want to use the reverse-ident feature of tcpwrappers for this reason. It is a very good idea to protect internal services from external access by firewalling them off at your border routers. The idea here is to prevent saturation attacks from outside your LAN, not so much to protect internal services from network-based root compromise. Always configure an exclusive firewall, i.e., firewall everything except ports A, B, C, D, and M-Z. This way you can firewall off all of your low ports except for certain specific services such as named (if you are primary for a zone), ntalkd, sendmail, and other Internet-accessible services. If you try to configure the firewall the other way – as an inclusive or permissive firewall, there is a good chance that you will forget to close a couple of services or that you will add a new internal service and forget to update the firewall. You can still open up the high-numbered port range on the firewall to allow permissive-like operation without compromising your low ports. Also take note that FreeBSD allows you to control the range of port numbers used for dynamic binding via the various net.inet.ip.portrange sysctl's (sysctl -a | fgrep portrange), which can also ease the complexity of your firewall's configuration. For example, you might use a normal first/last range of 4000 to 5000, and a hiport range of 49152 to 65535, then block everything under 4000 off in your firewall (except for certain specific Internet-accessible ports, of course). ICMP_BANDLIM Another common DOS attack is called a springboard attack – to attack a server in a manner that causes the server to generate responses which then overload the server, the local network, or some other machine. The most common attack of this nature is the ICMP ping broadcast attack. The attacker spoofs ping packets sent to your LAN's broadcast address with the source IP address set to the actual machine they wish to attack. If your border routers are not configured to stomp on ping's to broadcast addresses, your LAN winds up generating sufficient responses to the spoofed source address to saturate the victim, especially when the attacker uses the same trick on several dozen broadcast addresses over several dozen different networks at once. Broadcast attacks of over a hundred and twenty megabits have been measured. A second common springboard attack is against the ICMP error reporting system. By constructing packets that generate ICMP error responses, an attacker can saturate a server's incoming network and cause the server to saturate its outgoing network with ICMP responses. This type of attack can also crash the server by running it out of mbuf's, especially if the server cannot drain the ICMP responses it generates fast enough. The FreeBSD kernel has a new kernel compile option called ICMP_BANDLIM which limits the effectiveness of these sorts of attacks. The last major class of springboard attacks is related to certain internal inetd services such as the udp echo service. An attacker simply spoofs a UDP packet with the source address being server A's echo port, and the destination address being server B's echo port, where server A and B are both on your LAN. The two servers then bounce this one packet back and forth between each other. The attacker can overload both servers and their LANs simply by injecting a few packets in this manner. Similar problems exist with the internal chargen port. A competent sysadmin will turn off all of these inetd-internal test services. Spoofed packet attacks may also be used to overload the kernel route cache. Refer to the net.inet.ip.rtexpire, rtminexpire, and rtmaxcache sysctl parameters. A spoofed packet attack that uses a random source IP will cause the kernel to generate a temporary cached route in the route table, viewable with netstat -rna | fgrep W3. These routes typically timeout in 1600 seconds or so. If the kernel detects that the cached route table has gotten too big it will dynamically reduce the rtexpire but will never decrease it to less than rtminexpire. There are two problems: The kernel does not react quickly enough when a lightly loaded server is suddenly attacked. The rtminexpire is not low enough for the kernel to survive a sustained attack. If your servers are connected to the Internet via a T3 or better it may be prudent to manually override both rtexpire and rtminexpire via &man.sysctl.8;. Never set either parameter to zero (unless you want to crash the machine :-). Setting both parameters to 2 seconds should be sufficient to protect the route table from attack. Access Issues with Kerberos and SSH SSH Kerberos There are a few issues with both kerberos and ssh that need to be addressed if you intend to use them. Kerberos V is an excellent authentication protocol but there are bugs in the kerberized telnet and rlogin applications that make them unsuitable for dealing with binary streams. Also, by default kerberos does not encrypt a session unless you use the option. ssh encrypts everything by default. ssh works quite well in every respect except that it forwards encryption keys by default. What this means is that if you have a secure workstation holding keys that give you access to the rest of the system, and you ssh to an unsecure machine, your keys becomes exposed. The actual keys themselves are not exposed, but ssh installs a forwarding port for the duration of your login and if a attacker has broken root on the unsecure machine he can utilize that port to use your keys to gain access to any other machine that your keys unlock. We recommend that you use ssh in combination with kerberos whenever possible for staff logins. ssh can be compiled with kerberos support. This reduces your reliance on potentially exposable ssh keys while at the same time protecting passwords via kerberos. ssh keys should only be used for automated tasks from secure machines (something that kerberos is unsuited to). We also recommend that you either turn off key-forwarding in the ssh configuration, or that you make use of the from=IP/DOMAIN option that ssh allows in its authorized_keys file to make the key only usable to entities logging in from specific machines. DES, MD5, and Crypt security crypt crypt DES MD5 Parts rewritten and updated by &a.unfurl;, 21 March 2000. Every user on a UNIX system has a password associated with their account. It seems obvious that these passwords need to be known only to the user and the actual operating system. In order to keep these passwords secret, they are encrypted with what is known as a one-way hash, that is, they can only be easily encrypted but not decrypted. In other words, what we told you a moment ago was obvious is not even true: the operating system itself does not really know the password. It only knows the encrypted form of the password. The only way to get the plain-text password is by a brute force search of the space of possible passwords. Unfortunately the only secure way to encrypt passwords when UNIX came into being was based on DES, the Data Encryption Standard. This is not such a problem for users that live in the US, but since the source code for DES could not be exported outside the US, FreeBSD had to find a way to both comply with US law and retain compatibility with all the other UNIX variants that still use DES. The solution was to divide up the encryption libraries so that US users could install the DES libraries and use DES but international users still had an encryption method that could be exported abroad. This is how FreeBSD came to use MD5 as its default encryption method. MD5 is believed to be more secure than DES, so installing DES is offered primarily for compatibility reasons. Recognizing your crypt mechanism It is pretty easy to identify which encryption method FreeBSD is set up to use. Examining the encrypted passwords in the /etc/master.passwd file is one way. Passwords encrypted with the MD5 hash are longer than those with encrypted with the DES hash and also begin with the characters $1$. DES password strings do not have any particular identifying characteristics, but they are shorter than MD5 passwords, and are coded in a 64-character alphabet which does not include the $ character, so a relatively short string which does not begin with a dollar sign is very likely a DES password. The libraries can identify the passwords this way as well. As a result, the DES libraries are able to identify MD5 passwords, and use MD5 to check passwords that were encrypted that way, and DES for the rest. They are able to do this because the DES libraries also contain MD5. Unfortunately, the reverse is not true, so the MD5 libraries cannot authenticate passwords that were encrypted with DES. Identifying which library is being used by the programs on your system is easy as well. Any program that uses crypt is linked against libcrypt which for each type of library is a symbolic link to the appropriate implementation. For example, on a system using the DES versions: &prompt.user; ls -l /usr/lib/libcrypt* lrwxr-xr-x 1 root wheel 13 Mar 19 06:56 libcrypt.a -> libdescrypt.a lrwxr-xr-x 1 root wheel 18 Mar 19 06:56 libcrypt.so.2.0 -> libdescrypt.so.2.0 lrwxr-xr-x 1 root wheel 15 Mar 19 06:56 libcrypt_p.a -> libdescrypt_p.a On a system using the MD5-based libraries, the same links will be present, but the target will be libscrypt rather than libdescrypt. If you have installed the DES-capable crypt library libdescrypt (e.g. by installing the "crypto" distribution), then which password format will be used for new passwords is controlled by the passwd_format login capability in /etc/login.conf, which takes values of either des or md5. See the &man.login.conf.5; manpage for more information about login capabilities. S/Key S/Key security S/Key S/Key is a one-time password scheme based on a one-way hash function. FreeBSD uses the MD4 hash for compatibility but other systems have used MD5 and DES-MAC. S/Key has been part of the FreeBSD base system since version 1.1.5 and is also used on a growing number of other operating systems. S/Key is a registered trademark of Bell Communications Research, Inc. There are three different sorts of passwords which we will talk about in the discussion below. The first is your usual UNIX-style or Kerberos password; we will call this a UNIX password. The second sort is the one-time password which is generated by the S/Key key program and accepted by the keyinit program and the login prompt; we will call this a one-time password. The final sort of password is the secret password which you give to the key program (and sometimes the keyinit program) which it uses to generate one-time passwords; we will call it a secret password or just unqualified password. The secret password does not have anything to do with your UNIX password; they can be the same but this is not recommended. S/Key secret passwords are not limited to 8 characters like UNIX passwords, they can be as long as you like. Passwords of six or seven word long phrases are fairly common. For the most part, the S/Key system operates completely independently of the UNIX password system. Besides the password, there are two other pieces of data that are important to S/Key. One is what is known as the seed or key and consists of two letters and five digits. The other is what is called the iteration count and is a number between 1 and 100. S/Key creates the one-time password by concatenating the seed and the secret password, then applying the MD4 hash as many times as specified by the iteration count and turning the result into six short English words. These six English words are your one-time password. The login and su programs keep track of the last one-time password used, and the user is authenticated if the hash of the user-provided password is equal to the previous password. Because a one-way hash is used it is impossible to generate future one-time passwords if a successfully used password is captured; the iteration count is decremented after each successful login to keep the user and the login program in sync. When the iteration count gets down to 1 S/Key must be reinitialized. There are four programs involved in the S/Key system which we will discuss below. The key program accepts an iteration count, a seed, and a secret password, and generates a one-time password. The keyinit program is used to initialized S/Key, and to change passwords, iteration counts, or seeds; it takes either a secret password, or an iteration count, seed, and one-time password. The keyinfo program examines the /etc/skeykeys file and prints out the invoking user's current iteration count and seed. Finally, the login and su programs contain the necessary logic to accept S/Key one-time passwords for authentication. The login program is also capable of disallowing the use of UNIX passwords on connections coming from specified addresses. There are four different sorts of operations we will cover. The first is using the keyinit program over a secure connection to set up S/Key for the first time, or to change your password or seed. The second operation is using the keyinit program over an insecure connection, in conjunction with the key program over a secure connection, to do the same. The third is using the key program to log in over an insecure connection. The fourth is using the key program to generate a number of keys which can be written down or printed out to carry with you when going to some location without secure connections to anywhere. Secure connection initialization To initialize S/Key for the first time, change your password, or change your seed while logged in over a secure connection (e.g., on the console of a machine or via ssh), use the keyinit command without any parameters while logged in as yourself: &prompt.user; keyinit Adding unfurl: Reminder - Only use this method if you are directly connected. If you are using telnet or rlogin exit with no password and use keyinit -s. Enter secret password: Again secret password: ID unfurl s/key is 99 to17757 DEFY CLUB PRO NASH LACE SOFT At the Enter secret password: prompt you should enter a password or phrase. Remember, this is not the password that you will use to login with, this is used to generate your one-time login keys. The ID line gives the parameters of your particular S/Key instance; your login name, the iteration count, and seed. When logging in with S/Key, the system will remember these parameters and present them back to you so you do not have to remember them. The last line gives the particular one-time password which corresponds to those parameters and your secret password; if you were to re-login immediately, this one-time password is the one you would use. Insecure connection initialization To initialize S/Key or change your secret password over an insecure connection, you will need to already have a secure connection to some place where you can run the key program; this might be in the form of a desk accessory on a Macintosh, or a shell prompt on a machine you trust. You will also need to make up an iteration count (100 is probably a good value), and you may make up your own seed or use a randomly-generated one. Over on the insecure connection (to the machine you are initializing), use the keyinit -s command: &prompt.user; keyinit -s Updating unfurl: Old key: to17758 Reminder you need the 6 English words from the key command. Enter sequence count from 1 to 9999: 100 Enter new key [default to17759]: s/key 100 to 17759 s/key access password: To accept the default seed (which the keyinit program confusingly calls a key), press return. Then before entering an access password, move over to your secure connection or S/Key desk accessory, and give it the same parameters: &prompt.user; key 100 to17759 Reminder - Do not use this program while logged in via telnet or rlogin. Enter secret password: <secret password> CURE MIKE BANE HIM RACY GORE Now switch back over to the insecure connection, and copy the one-time password generated by key over to the keyinit program: s/key access password:CURE MIKE BANE HIM RACY GORE ID unfurl s/key is 100 to17759 CURE MIKE BANE HIM RACY GORE The rest of the description from the previous section applies here as well. Generating a single one-time password Once you've initialized S/Key, when you login you will be presented with a prompt like this: &prompt.user; telnet example.com Trying 10.0.0.1... Connected to example.com Escape character is '^]'. FreeBSD/i386 (example.com) (ttypa) login: <username> s/key 97 fw13894 Password: As a side note, the S/Key prompt has a useful feature (not shown here): if you press return at the password prompt, the login program will turn echo on, so you can see what you are typing. This can be extremely useful if you are attempting to type in an S/Key by hand, such as from a printout. Also, if this machine were configured to disallow UNIX passwords over a connection from the source machine, the prompt would have also included the annotation (s/key required), indicating that only S/Key one-time passwords will be accepted. At this point you need to generate your one-time password to answer this login prompt. This must be done on a trusted system that you can run the key command on. (There are versions of the key program from DOS, Windows and MacOS as well.) The key program needs both the iteration count and the seed as command line options. You can cut-and-paste these right from the login prompt on the machine that you are logging in to. On the trusted system: &prompt.user; key 97 fw13894 Reminder - Do not use this program while logged in via telnet or rlogin. Enter secret password: WELD LIP ACTS ENDS ME HAAG Now that you have your one-time password you can continue logging in: login: <username> s/key 97 fw13894 Password: <return to enable echo> s/key 97 fw13894 Password [echo on]: WELD LIP ACTS ENDS ME HAAG Last login: Tue Mar 21 11:56:41 from 10.0.0.2 ... This is the easiest mechanism if you have a trusted machine. There is a Java S/Key key applet, The Java OTP Calculator, that you can download and run locally on any Java supporting browser. Generating multiple one-time passwords Sometimes you have to go places where you do not have access to a trusted machine or secure connection. In this case, it is possible to use the key command to generate a number of one-time passwords before hand to be printed out and taken with you. For example: &prompt.user; key -n 5 30 zz99999 Reminder - Do not use this program while logged in via telnet or rlogin. Enter secret password: <secret password> 26: SODA RUDE LEA LIND BUDD SILT 27: JILT SPY DUTY GLOW COWL ROT 28: THEM OW COLA RUNT BONG SCOT 29: COT MASH BARR BRIM NAN FLAG 30: CAN KNEE CAST NAME FOLK BILK The requests five keys in sequence, the specifies what the last iteration number should be. Note that these are printed out in reverse order of eventual use. If you are really paranoid, you might want to write the results down by hand; otherwise you can cut-and-paste into lpr. Note that each line shows both the iteration count and the one-time password; you may still find it handy to scratch off passwords as you use them. Restricting use of UNIX passwords Restrictions can be placed on the use of UNIX passwords based on the host name, user name, terminal port, or IP address of a login session. These restrictions can be found in the configuration file /etc/skey.access. The &man.skey.access.5; manual page has more info on the complete format of the file and also details some security cautions to be aware of before depending on this file for security. If there is no /etc/skey.access file (this is the FreeBSD default), then all users will be allowed to use UNIX passwords. If the file exists, however, then all users will be required to use S/Key unless explicitly permitted to do otherwise by configuration statements in the skey.access file. In all cases, UNIX passwords are permitted on the console. Here is a sample configuration file which illustrates the three most common sorts of configuration statements: permit internet 192.168.0.0 255.255.0.0 permit user fnord permit port ttyd0 The first line (permit internet) allows users whose IP source address (which is vulnerable to spoofing) matches the specified value and mask, to use UNIX passwords. This should not be considered a security mechanism, but rather, a means to remind authorized users that they are using an insecure network and need to use S/Key for authentication. The second line (permit user) allows the specified username, in this case fnord, to use UNIX passwords at any time. Generally speaking, this should only be used for people who are either unable to use the key program, like those with dumb terminals, or those who are uneducable. The third line (permit port) allows all users logging in on the specified terminal line to use UNIX passwords; this would be used for dial-ups. Kerberos Kerberos Contributed by &a.markm; (based on contribution by &a.md;). Kerberos is a network add-on system/protocol that allows users to authenticate themselves through the services of a secure server. Services such as remote login, remote copy, secure inter-system file copying and other high-risk tasks are made considerably safer and more controllable. The following instructions can be used as a guide on how to set up Kerberos as distributed for FreeBSD. However, you should refer to the relevant manual pages for a complete description. 4.4BSD-Lite In FreeBSD, the Kerberos is not that from the original 4.4BSD-Lite, distribution, but eBones, which had been previously ported to FreeBSD 1.1.5.1, and was sourced from outside the USA/Canada, and was thus available to system owners outside those countries during the era of restrictive export controls on cryptographic code from the USA. Creating the initial database This is done on the Kerberos server only. First make sure that you do not have any old Kerberos databases around. You should change to the directory /etc/kerberosIV and check that only the following files are present: &prompt.root; cd /etc/kerberosIV &prompt.root; ls README krb.conf krb.realms If any additional files (such as principal.* or master_key) exist, then use the kdb_destroy command to destroy the old Kerberos database, of if Kerberos is not running, simply delete the extra files. You should now edit the krb.conf and krb.realms files to define your Kerberos realm. In this case the realm will be GRONDAR.ZA and the server is grunt.grondar.za. We edit or create the krb.conf file: &prompt.root; cat krb.conf GRONDAR.ZA GRONDAR.ZA grunt.grondar.za admin server CS.BERKELEY.EDU okeeffe.berkeley.edu ATHENA.MIT.EDU kerberos.mit.edu ATHENA.MIT.EDU kerberos-1.mit.edu ATHENA.MIT.EDU kerberos-2.mit.edu ATHENA.MIT.EDU kerberos-3.mit.edu LCS.MIT.EDU kerberos.lcs.mit.edu TELECOM.MIT.EDU bitsy.mit.edu ARC.NASA.GOV trident.arc.nasa.gov In this case, the other realms do not need to be there. They are here as an example of how a machine may be made aware of multiple realms. You may wish to not include them for simplicity. The first line names the realm in which this system works. The other lines contain realm/host entries. The first item on a line is a realm, and the second is a host in that realm that is acting as a key distribution center. The words admin server following a hosts name means that host also provides an administrative database server. For further explanation of these terms, please consult the Kerberos man pages. Now we have to add grunt.grondar.za to the GRONDAR.ZA realm and also add an entry to put all hosts in the .grondar.za domain in the GRONDAR.ZA realm. The krb.realms file would be updated as follows: &prompt.root; cat krb.realms grunt.grondar.za GRONDAR.ZA .grondar.za GRONDAR.ZA .berkeley.edu CS.BERKELEY.EDU .MIT.EDU ATHENA.MIT.EDU .mit.edu ATHENA.MIT.EDU Again, the other realms do not need to be there. They are here as an example of how a machine may be made aware of multiple realms. You may wish to remove them to simplify things. The first line puts the specific system into the named realm. The rest of the lines show how to default systems of a particular subdomain to a named realm. Now we are ready to create the database. This only needs to run on the Kerberos server (or Key Distribution Center). Issue the kdb_init command to do this: &prompt.root; kdb_init Realm name [default ATHENA.MIT.EDU ]: GRONDAR.ZA You will be prompted for the database Master Password. It is important that you NOT FORGET this password. Enter Kerberos master key: Now we have to save the key so that servers on the local machine can pick it up. Use the kstash command to do this. &prompt.root; kstash Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! This saves the encrypted master password in /etc/kerberosIV/master_key. Making it all run Two principals need to be added to the database for each system that will be secured with Kerberos. Their names are kpasswd and rcmd These two principals are made for each system, with the instance being the name of the individual system. These daemons, kpasswd and rcmd allow other systems to change Kerberos passwords and run commands like rcp, rlogin and rsh. Now let's add these entries: &prompt.root; kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: passwd Instance: grunt <Not found>, Create [y] ? y Principal: passwd, Instance: grunt, kdc_key_ver: 1 New Password: <---- enter RANDOM here Verifying password New Password: <---- enter RANDOM here Random password [y] ? y Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: rcmd Instance: grunt <Not found>, Create [y] ? Principal: rcmd, Instance: grunt, kdc_key_ver: 1 New Password: <---- enter RANDOM here Verifying password New Password: <---- enter RANDOM here Random password [y] ? Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit Creating the server file We now have to extract all the instances which define the services on each machine. For this we use the ext_srvtab command. This will create a file which must be copied or moved by secure means to each Kerberos client's /etc/kerberosIV directory. This file must be present on each server and client, and is crucial to the operation of Kerberos. &prompt.root; ext_srvtab grunt Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Generating 'grunt-new-srvtab'.... Now, this command only generates a temporary file which must be renamed to srvtab so that all the server can pick it up. Use the mv command to move it into place on the original system: &prompt.root; mv grunt-new-srvtab srvtab If the file is for a client system, and the network is not deemed safe, then copy the client-new-srvtab to removable media and transport it by secure physical means. Be sure to rename it to srvtab in the client's /etc/kerberosIV directory, and make sure it is mode 600: &prompt.root; mv grumble-new-srvtab srvtab &prompt.root; chmod 600 srvtab Populating the database We now have to add some user entries into the database. First let's create an entry for the user jane. Use the kdb_edit command to do this: &prompt.root; kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: jane Instance: <Not found>, Create [y] ? y Principal: jane, Instance: , kdc_key_ver: 1 New Password: <---- enter a secure password here Verifying password New Password: <---- re-enter the password here Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit Testing it all out First we have to start the Kerberos daemons. NOTE that if you have correctly edited your /etc/rc.conf then this will happen automatically when you reboot. This is only necessary on the Kerberos server. Kerberos clients will automagically get what they need from the /etc/kerberosIV directory. &prompt.root; kerberos & Kerberos server starting Sleep forever on error Log file is /var/log/kerberos.log Current Kerberos master key version is 1. Master key entered. BEWARE! Current Kerberos master key version is 1 Local realm: GRONDAR.ZA &prompt.root; kadmind -n & KADM Server KADM0.0A initializing Please do not use 'kill -9' to kill this job, use a regular kill instead Current Kerberos master key version is 1. Master key entered. BEWARE! Now we can try using the kinit command to get a ticket for the id jane that we created above: &prompt.user; kinit jane MIT Project Athena (grunt.grondar.za) Kerberos Initialization for "jane" Password: Try listing the tokens using klist to see if we really have them: &prompt.user; klist Ticket file: /tmp/tkt245 Principal: jane@GRONDAR.ZA Issued Expires Principal Apr 30 11:23:22 Apr 30 19:23:22 krbtgt.GRONDAR.ZA@GRONDAR.ZA Now try changing the password using passwd to check if the kpasswd daemon can get authorization to the Kerberos database: &prompt.user; passwd realm GRONDAR.ZA Old password for jane: New Password for jane: Verifying password New Password for jane: Password changed. Adding <command>su</command> privileges Kerberos allows us to give each user who needs root privileges their own separate supassword. We could now add an id which is authorized to su to root. This is controlled by having an instance of root associated with a principal. Using kdb_edit we can create the entry jane.root in the Kerberos database: &prompt.root; kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: jane Instance: root <Not found>, Create [y] ? y Principal: jane, Instance: root, kdc_key_ver: 1 New Password: <---- enter a SECURE password here Verifying password New Password: <---- re-enter the password here Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? 12 <--- Keep this short! Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit Now try getting tokens for it to make sure it works: &prompt.root; kinit jane.root MIT Project Athena (grunt.grondar.za) Kerberos Initialization for "jane.root" Password: Now we need to add the user to root's .klogin file: &prompt.root; cat /root/.klogin jane.root@GRONDAR.ZA Now try doing the su: &prompt.user; su Password: and take a look at what tokens we have: &prompt.root; klist Ticket file: /tmp/tkt_root_245 Principal: jane.root@GRONDAR.ZA Issued Expires Principal May 2 20:43:12 May 3 04:43:12 krbtgt.GRONDAR.ZA@GRONDAR.ZA Using other commands In an earlier example, we created a principal called jane with an instance root. This was based on a user with the same name as the principal, and this is a Kerberos default; that a <principal>.<instance> of the form <username>.root will allow that <username> to su to root if the necessary entries are in the .klogin file in root's home directory: &prompt.root; cat /root/.klogin jane.root@GRONDAR.ZA Likewise, if a user has in their own home directory lines of the form: &prompt.user; cat ~/.klogin jane@GRONDAR.ZA jack@GRONDAR.ZA This allows anyone in the GRONDAR.ZA realm who has authenticated themselves to jane or jack (via kinit, see above) access to rlogin to jane's account or files on this system (grunt) via rlogin, rsh or rcp. For example, Jane now logs into another system, using Kerberos: &prompt.user; kinit MIT Project Athena (grunt.grondar.za) Password: &prompt.user; rlogin grunt Last login: Mon May 1 21:14:47 from grumble Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD BUILT-19950429 (GR386) #0: Sat Apr 29 17:50:09 SAT 1995 Or Jack logs into Jane's account on the same machine (Jane having set up the .klogin file as above, and the person in charge of Kerberos having set up principal jack with a null instance: &prompt.user; kinit &prompt.user; rlogin grunt -l jane MIT Project Athena (grunt.grondar.za) Password: Last login: Mon May 1 21:16:55 from grumble Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD BUILT-19950429 (GR386) #0: Sat Apr 29 17:50:09 SAT 1995 Firewalls firewalls security firewalls Contributed by &a.gpalmer; and Alex Nash. Firewalls are an area of increasing interest for people who are connected to the Internet, and are even finding applications on private networks to provide enhanced security. This section will hopefully explain what firewalls are, how to use them, and how to use the facilities provided in the FreeBSD kernel to implement them. People often think that having a firewall between your internal network and the Big Bad Internet will solve all your security problems. It may help, but a poorly setup firewall system is more of a security risk than not having one at all. A firewall can add another layer of security to your systems, but it cannot stop a really determined cracker from penetrating your internal network. If you let internal security lapse because you believe your firewall to be impenetrable, you have just made the crackers job that much easier. What is a firewall? There are currently two distinct types of firewalls in common use on the Internet today. The first type is more properly called a packet filtering router, where the kernel on a multi-homed machine chooses whether to forward or block packets based on a set of rules. The second type, known as a proxy server, relies on daemons to provide authentication and to forward packets, possibly on a multi-homed machine which has kernel packet forwarding disabled. Sometimes sites combine the two types of firewalls, so that only a certain machine (known as a bastion host) is allowed to send packets through a packet filtering router onto an internal network. Proxy services are run on the bastion host, which are generally more secure than normal authentication mechanisms. FreeBSD comes with a kernel packet filter (known as IPFW), which is what the rest of this section will concentrate on. Proxy servers can be built on FreeBSD from third party software, but there is such a variety of proxy servers available that it would be impossible to cover them in this document. Packet filtering routers A router is a machine which forwards packets between two or more networks. A packet filtering router has an extra piece of code in its kernel which compares each packet to a list of rules before deciding if it should be forwarded or not. Most modern IP routing software has packet filtering code within it that defaults to forwarding all packets. To enable the filters, you need to define a set of rules for the filtering code so it can decide if the packet should be allowed to pass or not. To decide whether a packet should be passed on, the code looks through its set of rules for a rule which matches the contents of this packets headers. Once a match is found, the rule action is obeyed. The rule action could be to drop the packet, to forward the packet, or even to send an ICMP message back to the originator. Only the first match counts, as the rules are searched in order. Hence, the list of rules can be referred to as a rule chain. The packet matching criteria varies depending on the software used, but typically you can specify rules which depend on the source IP address of the packet, the destination IP address, the source port number, the destination port number (for protocols which support ports), or even the packet type (UDP, TCP, ICMP, etc). Proxy servers Proxy servers are machines which have had the normal system daemons (telnetd, ftpd, etc) replaced with special servers. These servers are called proxy servers as they normally only allow onward connections to be made. This enables you to run (for example) a proxy telnet server on your firewall host, and people can telnet in to your firewall from the outside, go through some authentication mechanism, and then gain access to the internal network (alternatively, proxy servers can be used for signals coming from the internal network and heading out). Proxy servers are normally more secure than normal servers, and often have a wider variety of authentication mechanisms available, including one-shot password systems so that even if someone manages to discover what password you used, they will not be able to use it to gain access to your systems as the password instantly expires. As they do not actually give users access to the host machine, it becomes a lot more difficult for someone to install backdoors around your security system. Proxy servers often have ways of restricting access further, so that only certain hosts can gain access to the servers, and often they can be set up so that you can limit which users can talk to which destination machine. Again, what facilities are available depends largely on what proxy software you choose. What does IPFW allow me to do? ipfw IPFW, the software supplied with FreeBSD, is a packet filtering and accounting system which resides in the kernel, and has a user-land control utility, &man.ipfw.8;. Together, they allow you to define and query the rules currently used by the kernel in its routing decisions. There are two related parts to IPFW. The firewall section allows you to perform packet filtering. There is also an IP accounting section which allows you to track usage of your router, based on similar rules to the firewall section. This allows you to see (for example) how much traffic your router is getting from a certain machine, or how much WWW (World Wide Web) traffic it is forwarding. As a result of the way that IPFW is designed, you can use IPFW on non-router machines to perform packet filtering on incoming and outgoing connections. This is a special case of the more general use of IPFW, and the same commands and techniques should be used in this situation. Enabling IPFW on FreeBSD ipfw enabling As the main part of the IPFW system lives in the kernel, you will need to add one or more options to your kernel configuration file, depending on what facilities you want, and recompile your kernel. See reconfiguring the kernel for more details on how to recompile your kernel. There are currently three kernel configuration options relevant to IPFW: options IPFIREWALL Compiles into the kernel the code for packet filtering. options IPFIREWALL_VERBOSE Enables code to allow logging of packets through &man.syslogd.8;. Without this option, even if you specify that packets should be logged in the filter rules, nothing will happen. options IPFIREWALL_VERBOSE_LIMIT=10 Limits the number of packets logged through &man.syslogd.8; on a per entry basis. You may wish to use this option in hostile environments in which you want to log firewall activity, but do not want to be open to a denial of service attack via syslog flooding. When a chain entry reaches the packet limit specified, logging is turned off for that particular entry. To resume logging, you will need to reset the associated counter using the &man.ipfw.8; utility: &prompt.root; ipfw zero 4500 Where 4500 is the chain entry you wish to continue logging. Previous versions of FreeBSD contained an IPFIREWALL_ACCT option. This is now obsolete as the firewall code automatically includes accounting facilities. Configuring IPFW ipfw configuring The configuration of the IPFW software is done through the &man.ipfw.8; utility. The syntax for this command looks quite complicated, but it is relatively simple once you understand its structure. There are currently four different command categories used by the utility: addition/deletion, listing, flushing, and clearing. Addition/deletion is used to build the rules that control how packets are accepted, rejected, and logged. Listing is used to examine the contents of your rule set (otherwise known as the chain) and packet counters (accounting). Flushing is used to remove all entries from the chain. Clearing is used to zero out one or more accounting entries. Altering the IPFW rules The syntax for this form of the command is: ipfw -N command index action log protocol addresses options There is one valid flag when using this form of the command: -N Resolve addresses and service names in output. The command given can be shortened to the shortest unique form. The valid commands are: add Add an entry to the firewall/accounting rule list delete Delete an entry from the firewall/accounting rule list Previous versions of IPFW used separate firewall and accounting entries. The present version provides packet accounting with each firewall entry. If an index value is supplied, it used to place the entry at a specific point in the chain. Otherwise, the entry is placed at the end of the chain at an index 100 greater than the last chain entry (this does not include the default policy, rule 65535, deny). The log option causes matching rules to be output to the system console if the kernel was compiled with IPFIREWALL_VERBOSE. Valid actions are: reject Drop the packet, and send an ICMP host or port unreachable (as appropriate) packet to the source. allow Pass the packet on as normal. (aliases: pass and accept) deny Drop the packet. The source is not notified via an ICMP message (thus it appears that the packet never arrived at the destination). count Update packet counters but do not allow/deny the packet based on this rule. The search continues with the next chain entry. Each action will be recognized by the shortest unambiguous prefix. The protocols which can be specified are: all Matches any IP packet icmp Matches ICMP packets tcp Matches TCP packets udp Matches UDP packets The address specification is: from address/maskport to address/maskport via interface You can only specify port in conjunction with protocols which support ports (UDP and TCP). The is optional and may specify the IP address or domain name of a local IP interface, or an interface name (e.g. ed0) to match only packets coming through this interface. Interface unit numbers can be specified with an optional wildcard. For example, ppp* would match all kernel PPP interfaces. The syntax used to specify an address/mask is: address or address/mask-bits or address:mask-pattern A valid hostname may be specified in place of the IP address. is a decimal number representing how many bits in the address mask should be set. e.g. specifying 192.216.222.1/24 will create a mask which will allow any address in a class C subnet (in this case, 192.216.222) to be matched. is an IP address which will be logically AND'ed with the address given. The keyword any may be used to specify any IP address. The port numbers to be blocked are specified as: port,port,port… to specify either a single port or a list of ports, or port-port to specify a range of ports. You may also combine a single range with a list, but the range must always be specified first. The options available are: frag Matches if the packet is not the first fragment of the datagram. in Matches if the packet is on the way in. out Matches if the packet is on the way out. ipoptions spec Matches if the IP header contains the comma separated list of options specified in spec. The supported list of IP options are: ssrr (strict source route), lsrr (loose source route), rr (record packet route), and ts (time stamp). The absence of a particular option may be denoted with a leading !. established Matches if the packet is part of an already established TCP connection (i.e. it has the RST or ACK bits set). You can optimize the performance of the firewall by placing established rules early in the chain. setup Matches if the packet is an attempt to establish a TCP connection (the SYN bit set is set but the ACK bit is not). tcpflags flags Matches if the TCP header contains the comma separated list of flags. The supported flags are fin, syn, rst, psh, ack, and urg. The absence of a particular flag may be indicated by a leading !. icmptypes types Matches if the ICMP type is present in the list types. The list may be specified as any combination of ranges and/or individual types separated by commas. Commonly used ICMP types are: 0 echo reply (ping reply), 3 destination unreachable, 5 redirect, 8 echo request (ping request), and 11 time exceeded (used to indicate TTL expiration as with &man.traceroute.8;). Listing the IPFW rules The syntax for this form of the command is: ipfw -a -t -N l There are three valid flags when using this form of the command: -a While listing, show counter values. This option is the only way to see accounting counters. -t Display the last match times for each chain entry. The time listing is incompatible with the input syntax used by the &man.ipfw.8; utility. -N Attempt to resolve given addresses and service names. Flushing the IPFW rules The syntax for flushing the chain is: ipfw flush This causes all entries in the firewall chain to be removed except the fixed default policy enforced by the kernel (index 65535). Use caution when flushing rules, the default deny policy will leave your system cut off from the network until allow entries are added to the chain. Clearing the IPFW packet counters The syntax for clearing one or more packet counters is: ipfw zero index When used without an index argument, all packet counters are cleared. If an index is supplied, the clearing operation only affects a specific chain entry. Example commands for ipfw This command will deny all packets from the host evil.crackers.org to the telnet port of the host nice.people.org: &prompt.root ipfw add deny tcp from evil.crackers.org to nice.people.org 23 The next example denies and logs any TCP traffic from the entire crackers.org network (a class C) to the nice.people.org machine (any port). &prompt.root; ipfw add deny log tcp from evil.crackers.org/24 to nice.people.org If you do not want people sending X sessions to your internal network (a subnet of a class C), the following command will do the necessary filtering: &prompt.root; ipfw add deny tcp from any to my.org/28 6000 setup To see the accounting records: &prompt.root; ipfw -a list or in the short form &prompt.root; ipfw -a l You can also see the last time a chain entry was matched with: &prompt.root; ipfw -at l Building a packet filtering firewall The following suggestions are just that: suggestions. The requirements of each firewall are different and we cannot tell you how to build a firewall to meet your particular requirements. When initially setting up your firewall, unless you have a test bench setup where you can configure your firewall host in a controlled environment, it is strongly recommend you use the logging version of the commands and enable logging in the kernel. This will allow you to quickly identify problem areas and cure them without too much disruption. Even after the initial setup phase is complete, I recommend using the logging for `deny' as it allows tracing of possible attacks and also modification of the firewall rules if your requirements alter. If you use the logging versions of the accept command, it can generate large amounts of log data as one log line will be generated for every packet that passes through the firewall, so large ftp/http transfers, etc, will really slow the system down. It also increases the latencies on those packets as it requires more work to be done by the kernel before the packet can be passed on. syslogd with also start using up a lot more processor time as it logs all the extra data to disk, and it could quite easily fill the partition /var/log is located on. You should enable your firewall from /etc/rc.conf.local or /etc/rc.conf. The associated man page explains which knobs to fiddle and lists some preset firewall configurations. If you do not use a preset configuration, ipfw list will output the current ruleset into a file that you can pass to rc.conf. If you do not use /etc/rc.conf.local or /etc/rc.conf to enable your firewall, it is important to make sure your firewall is enabled before any IP interfaces are configured. The next problem is what your firewall should actually do! This is largely dependent on what access to your network you want to allow from the outside, and how much access to the outside world you want to allow from the inside. Some general rules are: Block all incoming access to ports below 1024 for TCP. This is where most of the security sensitive services are, like finger, SMTP (mail) and telnet. Block all incoming UDP traffic. There are very few useful services that travel over UDP, and what useful traffic there is normally a security threat (e.g. Suns RPC and NFS protocols). This has its disadvantages also, since UDP is a connectionless protocol, denying incoming UDP traffic also blocks the replies to outgoing UDP traffic. This can cause a problem for people (on the inside) using external archie (prospero) servers. If you want to allow access to archie, you'll have to allow packets coming from ports 191 and 1525 to any internal UDP port through the firewall. ntp is another service you may consider allowing through, which comes from port 123. Block traffic to port 6000 from the outside. Port 6000 is the port used for access to X11 servers, and can be a security threat (especially if people are in the habit of doing xhost + on their workstations). X11 can actually use a range of ports starting at 6000, the upper limit being how many X displays you can run on the machine. The upper limit as defined by RFC 1700 (Assigned Numbers) is 6063. Check what ports any internal servers use (e.g. SQL servers, etc). It is probably a good idea to block those as well, as they normally fall outside the 1-1024 range specified above. Another checklist for firewall configuration is available from CERT at http://www.cert.org/tech_tips/packet_filtering.html As stated above, these are only guidelines. You will have to decide what filter rules you want to use on your firewall yourself. We cannot accept ANY responsibility if someone breaks into your network, even if you follow the advice given above. OpenSSL security OpenSSL OpenSSL As of FreeBSD 4.0, the OpenSSL toolkit is a part of the base system. OpenSSL provides a general-purpose cryptography library, as well as the Secure Sockets Layer v2/v3 (SSLv2/SSLv3) and Transport Layer Security v1 (TLSv1) network security protocols. However, one of the algorithms (specifically IDEA) included in OpenSSL is protected by patents in the USA and elsewhere, and is not available for unrestricted use. IDEA is included in the OpenSSL sources in FreeBSD, but it is not built by default. If you wish to use it, and you comply with the license terms, enable the MAKE_IDEA switch in /etc/make.conf and rebuild your sources using 'make world'. Today, the RSA algorithm is free for use in USA and other countries. In the past it was protected by a patent. OpenSSL install Source Code Installations OpenSSL is part of the src-crypto and src-secure cvsup collections. See the Obtaining FreeBSD section for more information about obtaining and updating FreeBSD source code. IPsec IPsec security IPsec Contributed by &a.shin;, 5 March 2000. The IPsec mechanism provides secure communication either for IP layer and socket layer communication. This section should explain how to use them. For implementation details, please refer to The Developers' Handbook. The current IPsec implementation supports both transport mode and tunnel mode. However, tunnel mode comes with some restrictions. http://www.kame.net/newsletter/ has more comprehensive examples. Please be aware that in order to use this functionality, you must have the following options compiled into your kernel: options IPSEC #IP security options IPSEC_ESP #IP security (crypto; define w/IPSEC) Transport mode example with IPv4 Let's setup security association to deploy a secure channel between HOST A (10.2.3.4) and HOST B (10.6.7.8). Here we show a little complicated example. From HOST A to HOST B, only old AH is used. From HOST B to HOST A, new AH and new ESP are combined. Now we should choose algorithm to be used corresponding to "AH"/"new AH"/"ESP"/"new ESP". Please refer to the &man.setkey.8; man page to know algorithm names. Our choice is MD5 for AH, new-HMAC-SHA1 for new AH, and new-DES-expIV with 8 byte IV for new ESP. Key length highly depends on each algorithm. For example, key length must be equal to 16 bytes for MD5, 20 for new-HMAC-SHA1, and 8 for new-DES-expIV. Now we choose "MYSECRETMYSECRET", "KAMEKAMEKAMEKAMEKAME", "PASSWORD", respectively. OK, let's assign SPI (Security Parameter Index) for each protocol. Please note that we need 3 SPIs for this secure channel since three security headers are produced (one for from HOST A to HOST B, two for from HOST B to HOST A). Please also note that SPI MUST be greater than or equal to 256. We choose, 1000, 2000, and 3000, respectively. - (1) HOST A ------> HOST B (1)PROTO=AH ALG=MD5(RFC1826) KEY=MYSECRETMYSECRET SPI=1000 (2.1) HOST A <------ HOST B <------ (2.2) (2.1) PROTO=AH ALG=new-HMAC-SHA1(new AH) KEY=KAMEKAMEKAMEKAMEKAME SPI=2000 (2.2) PROTO=ESP ALG=new-DES-expIV(new ESP) IV length = 8 KEY=PASSWORD SPI=3000 - - + Now, let's setup security association. Execute &man.setkey.8; on both HOST A and B: - &prompt.root; setkey -c add 10.2.3.4 10.6.7.8 ah-old 1000 -m transport -A keyed-md5 "MYSECRETMYSECRET" ; add 10.6.7.8 10.2.3.4 ah 2000 -m transport -A hmac-sha1 "KAMEKAMEKAMEKAMEKAME" ; add 10.6.7.8 10.2.3.4 esp 3000 -m transport -E des-cbc "PASSWORD" ; ^D - - + Actually, IPsec communication doesn't process until security policy entries will be defined. In this case, you must setup each host. - At A: &prompt.root; setkey -c spdadd 10.2.3.4 10.6.7.8 any -P out ipsec ah/transport/10.2.3.4-10.6.7.8/require ; ^D At B: &prompt.root; setkey -c spdadd 10.6.7.8 10.2.3.4 any -P out ipsec esp/transport/10.6.7.8-10.2.3.4/require ; spdadd 10.6.7.8 10.2.3.4 any -P out ipsec ah/transport/10.6.7.8-10.2.3.4/require ; ^D HOST A --------------------------------------> HOST E 10.2.3.4 10.6.7.8 | | ========== old AH keyed-md5 ==========> <========= new AH hmac-sha1 =========== <========= new ESP des-cbc ============ - - + Transport mode example with IPv6 Another example using IPv6. ESP transport mode is recommended for TCP port number 110 between Host-A and Host-B. - ============ ESP ============ | | Host-A Host-B fec0::10 -------------------- fec0::11 - - + Encryption algorithm is blowfish-cbc whose key is "kamekame", and authentication algorithm is hmac-sha1 whose key is "this is the test key". Configuration at Host-A: - &prompt.root; setkey -c <<EOF spdadd fec0::10[any] fec0::11[110] tcp -P out ipsec esp/transport/fec0::10-fec0::11/use ; spdadd fec0::11[110] fec0::10[any] tcp -P in ipsec esp/transport/fec0::11-fec0::10/use ; add fec0::10 fec0::11 esp 0x10001 -m transport -E blowfish-cbc "kamekame" -A hmac-sha1 "this is the test key" ; add fec0::11 fec0::10 esp 0x10002 -m transport -E blowfish-cbc "kamekame" -A hmac-sha1 "this is the test key" ; EOF - - + and at Host-B: - - &prompt.root; setkey -c <<EOF + &prompt.root; setkey -c <<EOF spdadd fec0::11[110] fec0::10[any] tcp -P out ipsec esp/transport/fec0::11-fec0::10/use ; spdadd fec0::10[any] fec0::11[110] tcp -P in ipsec esp/transport/fec0::10-fec0::11/use ; add fec0::10 fec0::11 esp 0x10001 -m transport -E blowfish-cbc "kamekame" -A hmac-sha1 "this is the test key" ; add fec0::11 fec0::10 esp 0x10002 -m transport -E blowfish-cbc "kamekame" -A hmac-sha1 "this is the test key" ; EOF - - + Note the direction of SP. Tunnel mode example with IPv4 Tunnel mode between two security gateways Security protocol is old AH tunnel mode, i.e. specified by RFC1826, with keyed-md5 whose key is "this is the test" as authentication algorithm. - ======= AH ======= | | Network-A Gateway-A Gateway-B Network-B 10.0.1.0/24 ---- 172.16.0.1 ----- 172.16.0.2 ---- 10.0.2.0/24 - - + Configuration at Gateway-A: - &prompt.root; setkey -c <<EOF spdadd 10.0.1.0/24 10.0.2.0/24 any -P out ipsec ah/tunnel/172.16.0.1-172.16.0.2/require ; spdadd 10.0.2.0/24 10.0.1.0/24 any -P in ipsec ah/tunnel/172.16.0.2-172.16.0.1/require ; add 172.16.0.1 172.16.0.2 ah-old 0x10003 -m any -A keyed-md5 "this is the test" ; add 172.16.0.2 172.16.0.1 ah-old 0x10004 -m any -A keyed-md5 "this is the test" ; EOF - - + If port number field is omitted such above then "[any]" is employed. `-m' specifies the mode of SA to be used. "-m any" means wild-card of mode of security protocol. You can use this SA for both tunnel and transport mode. and at Gateway-B: - &prompt.root; setkey -c <<EOF spdadd 10.0.2.0/24 10.0.1.0/24 any -P out ipsec ah/tunnel/172.16.0.2-172.16.0.1/require ; spdadd 10.0.1.0/24 10.0.2.0/24 any -P in ipsec ah/tunnel/172.16.0.1-172.16.0.2/require ; add 172.16.0.1 172.16.0.2 ah-old 0x10003 -m any -A keyed-md5 "this is the test" ; add 172.16.0.2 172.16.0.1 ah-old 0x10004 -m any -A keyed-md5 "this is the test" ; EOF - - + Making SA bundle between two security gateways AH transport mode and ESP tunnel mode is required between Gateway-A and Gateway-B. In this case, ESP tunnel mode is applied first, and AH transport mode is next. - ========== AH ========= | ======= ESP ===== | | | | | Network-A Gateway-A Gateway-B Network-B fec0:0:0:1::/64 --- fec0:0:0:1::1 ---- fec0:0:0:2::1 --- fec0:0:0:2::/64 - - + Tunnel mode example with IPv6 Encryption algorithm is 3des-cbc, and authentication algorithm for ESP is hmac-sha1. Authentication algorithm for AH is hmac-md5. Configuration at Gateway-A: - &prompt.root; setkey -c <<EOF spdadd fec0:0:0:1::/64 fec0:0:0:2::/64 any -P out ipsec esp/tunnel/fec0:0:0:1::1-fec0:0:0:2::1/require ah/transport/fec0:0:0:1::1-fec0:0:0:2::1/require ; spdadd fec0:0:0:2::/64 fec0:0:0:1::/64 any -P in ipsec esp/tunnel/fec0:0:0:2::1-fec0:0:0:1::1/require ah/transport/fec0:0:0:2::1-fec0:0:0:1::1/require ; add fec0:0:0:1::1 fec0:0:0:2::1 esp 0x10001 -m tunnel -E 3des-cbc "kamekame12341234kame1234" -A hmac-sha1 "this is the test key" ; add fec0:0:0:1::1 fec0:0:0:2::1 ah 0x10001 -m transport -A hmac-md5 "this is the test" ; add fec0:0:0:2::1 fec0:0:0:1::1 esp 0x10001 -m tunnel -E 3des-cbc "kamekame12341234kame1234" -A hmac-sha1 "this is the test key" ; add fec0:0:0:2::1 fec0:0:0:1::1 ah 0x10001 -m transport -A hmac-md5 "this is the test" ; EOF - - + Making SAs with the different end ESP tunnel mode is required between Host-A and Gateway-A. Encryption algorithm is cast128-cbc, and authentication algorithm for ESP is hmac-sha1. ESP transport mode is recommended between Host-A and Host-B. Encryption algorithm is rc5-cbc, and authentication algorithm for ESP is hmac-md5. - ================== ESP ================= | ======= ESP ======= | | | | | Host-A Gateway-A Host-B fec0:0:0:1::1 ---- fec0:0:0:2::1 ---- fec0:0:0:2::2 - - + Configuration at Host-A: - &prompt.root; setkey -c <<EOF spdadd fec0:0:0:1::1[any] fec0:0:0:2::2[80] tcp -P out ipsec esp/transport/fec0:0:0:1::1-fec0:0:0:2::2/use esp/tunnel/fec0:0:0:1::1-fec0:0:0:2::1/require ; spdadd fec0:0:0:2::1[80] fec0:0:0:1::1[any] tcp -P in ipsec esp/transport/fec0:0:0:2::2-fec0:0:0:l::1/use esp/tunnel/fec0:0:0:2::1-fec0:0:0:1::1/require ; add fec0:0:0:1::1 fec0:0:0:2::2 esp 0x10001 -m transport -E cast128-cbc "12341234" -A hmac-sha1 "this is the test key" ; add fec0:0:0:1::1 fec0:0:0:2::1 esp 0x10002 -E rc5-cbc "kamekame" -A hmac-md5 "this is the test" ; add fec0:0:0:2::2 fec0:0:0:1::1 esp 0x10003 -m transport -E cast128-cbc "12341234" -A hmac-sha1 "this is the test key" ; add fec0:0:0:2::1 fec0:0:0:1::1 esp 0x10004 -E rc5-cbc "kamekame" -A hmac-md5 "this is the test" ; EOF - - + OpenSSH OpenSSH security OpenSSH Contributed by &a.chern;, April 21, 2001. Secure shell is a set of network connectivity tools used to access remote machines securely. It can be used as a direct replacement for rlogin, rsh, rcp, and telnet. Additionally, any other TCP/IP connections can be tunneled/forwarded securely through ssh. ssh encrypts all traffic to effectively eliminate eavesdropping, connection hijacking, and other network-level attacks. OpenSSH is maintained by the OpenBSD project, and is based upon SSH v1.2.12 with all the recent bug fixes and updates. It is compatible with both SSH protocols 1 and 2. OpenSSH has been in the base system since FreeBSD 4.0. Advantages of using OpenSSH Normally, when using &man.telnet.1; or &man.rlogin.1;, data is sent over the network in an clear, un-encrypted form. Network sniffers anywhere in between the client and server can steal your user/password information or data transferred in your session. OpenSSH offers a variety of authentication and encryption methods to prevent this from happening. Enabling sshd OpenSSH enabling Be sure to make the following additions to your rc.conf file: sshd_enable="YES" This will load the ssh daemon the next time your system initializes. Alternatively, you can simply run the sshd daemon. SSH client OpenSSH client The &man.ssh.1; utility works similarly to &man.rlogin.1;. &prompt.root ssh user@foobardomain.com Host key not found from the list of known hosts. Are you sure you want to continue connecting (yes/no)? yes Host 'foobardomain.com' added to the list of known hosts. -user@foobardomain.com's password: ******* - +user@foobardomain.com's password: ******* The login will continue just as it would have if a session was created using rlogin or telnet. SSH utilizes a key fingerprint system for verifying the authenticity of the server when the client connects. The user is prompted to enter 'yes' only during the first time connecting. Future attempts to login are all verified against the saved fingerprint key. The SSH client will alert you if the saved fingerprint differs from the received fingerprint on future login attempts. The fingerprints are saved in ~/.ssh/known_hosts Secure copy OpenSSH secure copy scp The scp command works similarly to rcp; it copies a file to or from a remote machine, except in a secure fashion. &prompt.root scp user@foobardomain.com:/COPYRIGHT COPYRIGHT user@foobardomain.com's password: COPYRIGHT 100% |*****************************| 4735 00:00 -&prompt.root - +&prompt.root Since the fingerprint was already saved for this host in the previous example, it is verified when using scp here. Configuration OpenSSH configuration The system-wide configuration files for both the OpenSSH daemon and client reside within the /etc/ssh directory. ssh_config configures the client settings, while sshd_config configures the daemon. ssh-keygen Instead of using passwords, &man.ssh-keygen.1; can be used to generate RSA keys to authenticate a user. &prompt.user ssh-keygen Initializing random number generator... Generating p: .++ (distance 66) Generating q: ..............................++ (distance 498) Computing the keys... Key generation complete. Enter file in which to save the key (/home/user/.ssh/identity): Enter passphrase: Enter the same passphrase again: Your identification has been saved in /home/user/.ssh/identity. -... - +... &man.ssh-keygen.1; will create a public and private key pair for use in authentication. The private key is stored in ~/.ssh/identity, whereas the public key is stored in ~/.ssh/identity.pub. The public key must be placed in ~/.ssh/authorized_keys of the remote machine in order for the setup to work. This will allow connection to the remote machine based upon RSA authentication instead of passwords. If a passphrase is used in &man.ssh-keygen.1;, the user will be prompted for a password each time in order to use the private key. &man.ssh-agent.1; and &man.ssh-add.1; are utilities used in managing multiple passworded private keys. SSH Tunneling OpenSSH tunneling OpenSSH has the ability to create a tunnel to encapsulate another protocol in an encrypted session. The following command tells &man.ssh.1; to create a tunnel for telnet. &prompt.user; ssh -2 -N -f -L 5023:localhost:23 user@foo.bar.com -&prompt.user; - +&prompt.user; -2 this forces &man.ssh.1 to use version 2 of the protocol. (Do not use if you are working with older ssh servers) -N indicates no command, or tunnel only. If omitted, &man.ssh.1; would initiate a normal session. -f forces &man.ssh.1; to run in the background. -L indicates a local tunnel in localport:localhost:remoteport fashion. foo.bar.com is the remote/target SSH server. An SSH tunnel works by creating a listen socket on the specified local host and port. It then forwards any connection to the local host/port via the SSH connection to the remote machine on the specified remote port. In the example, port 5023 on localhost is being forwarded to port 23 on the remote machine. Since 23 is telnet, this would create a secure telnet session through an SSH tunnel. This can be used to wrap any number of insecure TCP protocols such as smtp, pop3, ftp, etc. A typical SSH Tunnel &prompt.user; ssh -2 -N -f -L 5025:localhost:25 user@mailserver.foobar.com user@mailserver.foobar.com's password: ***** &prompt.user; telnet localhost 5025 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. -220 mailserver.foobar.com ESMTP - +220 mailserver.foobar.com ESMTP This can be used in conjunction with an &man.ssh-keygen.1; and additional user accounts to create a more seamless/hassle-free SSH tunneling environment. Keys can be used in place of typing a password, and the tunnels can be run as a separate user. Further Reading OpenSSH &man.ssh.1; &man.scp.1; &man.ssh-keygen.1; &man.ssh-agent.1; &man.ssh-add.1; &man.sshd.8; &man.sftp-server.8; diff --git a/en_US.ISO8859-1/books/handbook/sound/chapter.sgml b/en_US.ISO8859-1/books/handbook/sound/chapter.sgml index 12485dcf49..06d6817232 100644 --- a/en_US.ISO8859-1/books/handbook/sound/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/sound/chapter.sgml @@ -1,334 +1,332 @@ Sound Contributed by Moses Moore jm-moses@home.com, 20 November 2000. Synopsis This chapter of the handbook deals with setting up sound on a FreeBSD system. Locating the Correct Device PCI ISA sound cards Before you begin, you should know the model of the card you have, the chip it uses, and whether it is a PCI or ISA card. FreeBSD supports a wide variety of both PCI and ISA cards. If you do not see your card in the following list, check the &man.pcm.4; manual page. This is not a complete list; however, it does list some of the most common cards. Crystal 4237, 4236, 4232, 4231 Yamaha OPL-SAx OPTi931 Ensoniq AudioPCI 1370/1371 ESS Solo-1/1E NeoMagic 256AV/ZX Sound Blaster Pro, 16, 32, AWE64, AWE128, Live Creative ViBRA16 Advanced Asound 100, 110, and Logic ALS120 ES 1868, 1869, 1879, 1888 Gravis UltraSound Aureal Vortex 1 or 2 kernelconfiguration The driver you use in your kernel depends on the kind of card you have. The sections below provide more information and what you will need to add to your kernel configuration. Creative, Advance, and ESS Sound Cards If you have one of the above cards, you will need to add device pcm to your kernel. If you have a PnP ISA card, you will also need to add device sbc to your kernel. For a non-PnP ISA card, add device pcm and device sbc0 at isa? port0x220 irq 5 drq 1 flags 0x15 to your kernel. Those are the default settings. You may need to change the IRQ, etc. See the &man.sbc.4; man page for more information. The Sound Blaster Live is not supported under FreeBSD 4.0 without a patch, which this document will not cover. It is recommended that you update to the latest -STABLE before trying to use this card. Gravis UltraSound Cards For a PnP ISA card, you will need to add device pcm and device gusc to your kernel. If you have a non-PnP ISA card, you will need to add device pcm and device gus0 at isa? port 0x220 irq 5 drq 1 flags 0x13 to your kernel. You may need to change the IRQ, etc. See the &man.gusc.4; man page for more information. Crystal Sound Cards For Crystal cards, you will need both device pcm and device csa in your kernel. Generic Support For PnP ISA or PCI cards, you will need to add device pcm to your kernel configuration. If you have a non-PnP ISA sound card that does not have a bridge driver, you will need to add device pcm0 at isa? irq 10 drq 1 flags 0x0 to your kernel configuration. You may need to change the IRQ, etc., to match your hardware configuration. Recompiling the Kernel After adding the driver(s) you need to your kernel configuration, you will need to recompile your kernel. Please see of the handbook for more information. Creating and Testing the Device Nodes device nodes After you reboot, log in and run cat /dev/sndstat. You should see output similar to the following: FreeBSD Audio Driver (newpcm) Sep 21 2000 18:29:53 Installed devices: pcm0: <Aureal Vortex 8830> at memory 0xfeb40000 irq 5 (4p/1r +channels duplex) If you see an error message, something went wrong earlier. If that happens, go through your kernel configuration file again and make sure you chose the correct device. If it reported no errors and returned pcm0, su to root and do the following: - -&prompt.root; cd /dev + &prompt.root; cd /dev &prompt.root; sh MAKEDEV snd0 If it reported no errors and returned pcm1, su to root and do the following: - -&prompt.root; cd /dev + &prompt.root; cd /dev &prompt.root; sh MAKEDEV snd1 Please note that either of the above commands will not create a /dev/snd device! Instead it creates a group of device nodes including: Device Description /dev/audio SPARC-compatible audio device /dev/dsp Digitized voice device /dev/dspW Like /dev/dsp, but 16 bits per sample /dev/midi Raw midi access device /dev/mixer Control port mixer device /dev/music Level 2 sequencer interface /dev/sequencer Sequencer device /dev/pss Programmable device interface If all goes well, you should now have a functioning sound card. If you do not, see the next section. Common Problems device node I get an unsupported subdevice XX error! One or more of the device nodes wasn't created correctly. Repeat the steps above. I/O port I get a sb_dspwr(XX) timed out error! The I/O port is not set correctly. IRQ I get a bad irq XX error! The IRQ is set incorrectly. Make sure that the set IRQ and the sound IRQ are the same. I get a "xxx: gus pcm not attached, out of memory" error. What causes that? If this happens, it is because there is not enough available memory to use the device. diff --git a/en_US.ISO8859-1/books/handbook/x11/chapter.sgml b/en_US.ISO8859-1/books/handbook/x11/chapter.sgml index 4ccc91dd24..304a51c752 100644 --- a/en_US.ISO8859-1/books/handbook/x11/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/x11/chapter.sgml @@ -1,1903 +1,1887 @@ The X Window System This chapter has been graciously donated by &a.grog; from his book, The Complete FreeBSD, and remains copyright of him. Modifications for the handbook made by &a.jim;. The section on fonts in XFree86 was contributed by &a.murray; and the section on XDM was contributed by &a.sethk;. Synopsis The following chapter will cover installing and configuring X11 on your system. For more information on X11 and to see whether your video card is supported, check the XFree86 web site. Overview FreeBSD comes with XFree86, a port of X11R6 that supports several versions of Intel-based UNIX. This chapter describes how to set up your XFree86 server. It is based on material supplied with the FreeBSD release, specifically the files README.FreeBSD and README.Config in the directory /usr/X11R6/lib/X11/doc. If you find any discrepancy, the material in those files will be more up-to-date than this description. In addition, the file /usr/X11R6/lib/X11/doc/RELNOTES contains OS-independent information about the current release. X uses a lot of memory. In order to run X, your system should have an absolute minimum of 8 MB of memory, but performance will be painful with so little memory. A more practical minimum is 16 MB, and you can improve performance by adding more memory. If you use X intensively, you will continue seeing performance improvement by increasing to as much as 128 MB of RAM. There is lots of useful information in the rest of this chapter, but maybe you are not interested in information right now. You just want to get your X server up and running. However, be warned: An incorrect installation can burn out your monitor or your video board. However, if you know you are in spec, and you have a standard Super VGA board and a good multi-frequency monitor, then you can probably get things up and running without reading this chapter. Installing XFree86 The easiest way to install XFree86 is with the sysinstall program, either when you are installing the system, or later by starting the program /stand/sysinstall. In the rest of this chapter, we will look at what makes up the distribution, and we will also take a look at manually installing X11. The XFree86 Distribution XFree86 is distributed as a bewildering number of archives. In the following section, we will take a look at what you should install. Do not worry too much, though; if you cannot decide what to pick and you have 200MB of disk space free, it's safe to unpack everything. At a minimum you need to unpack the archives in the following table and at least one server that matches your VGA board. You will need 10Mb for the minimum required run-time binaries only, and between 1.7 and 3 MB for the server. Below is a table of the required components. Archive Description Xbin.tgz All the executable X client applications and shared libraries. Xfnts.tgz The misc and 75 dpi fonts. Xlib.tgz Data files and libraries needed at runtime. The X Server In addition to the archives above, you need at least one server, which will take up about 3 MB of disk. The choice depends primarily on what kind of display board you have. The default server name is /usr/X11R6/bin/X, and it is a link to a specific server binary /usr/X11R6/bin/XF86_xxxx. You will find the server archives for the standard PC architecture in /cdrom/XF86336/Servers, and the servers for the Japanese PC98 architecture in /cdrom/XF86336/PC98-Servers if you have the CD set. Alternatively, they are available on our FTP site at ftp://ftp.FreeBSD.org/pub/FreeBSD/releases/i386/&rel.current;-RELEASE/XF86336/Servers/ or ftp://ftp.FreeBSD.org/pub/FreeBSD/releases/i386/&rel.current;-RELEASE/XF86336/PC98-Servers/ Available X servers for the standard PC architecture: Archive Description X8514.tgz 8-bit color for IBM 8514 and true compatibles. XAGX.tgz 8 and 16-bit color for AGX and XGA boards. XI128.tgz 8 and 16-bit color for I128 boards. XMa32.tgz 8 and 16-bit color for ATI Mach32 boards. XMa64.tgz 8, 16, and 32-bit color fot ATI Mach64 boards. XMa8.tgz 8-bit color for ATI Mach8 boards. XMono.tgz 1-bit monochrome for VGA, Super-VGA, Hercules, and others. XP9K.tgz 8, 16, and 32-bit color for Weitek P9000 boards (Diamond Viper). XS3.tgz 8, 16, and 32-bit color for S3 boards. XS3V.tgz 8 and 16-bit color for S3 ViRGE boards. XSVGA.tgz >=8-bit color for Super-VGA cards. XVG16.tgz 4-bit color for VGA and Super-VGA cards. XW32.tgz 8-bit color for ET4000/W32, /W32i, /W32p, and ET6000 cards. Available X servers for the Japanese PC98 architecture: Archive Description X9GAN.tgz 8-bit color for PC98 GA-98NB/WAP boards. X9GA9.tgz 8, 16, and 32-bit color for PC98 S3 GA-968 boards. X9480.tgz 8-bit color for PC98 PEGC X9NKV.tgz 8-bit color for PC98 NEC-CIRRUS/EPSON NKV/NKV2 boards. X9WBS.tgz 8-bit color for PC98 WAB-S boards. X9WEP.tgz 8-bit color for PC98 WAB-EP boards. X9WSN.tgz 8-bit color for PC98 WSN-A2F boards. X9EGC.tgz 4-bit color for PC98 EGC. X9TGU.tgz 8 and 16-bit color for PC98 Trident Cyber9320/9680 boards. X9NS3.tgz 8 and 16-bit color for PC98 NEC S3 boards. X9SPW.tgz 8 and 16-bit color for PC98 S3 PW/PCSKB boards. X9LPW.tgz 8 and 16-bit color for PC98 S3 PW/LB boards. Each of these servers includes a manual page which contains details of supported chipsets and server-specific configuration options. There are also a number of archives are provided for X programmers: Archive Description Xprog.tgz Config, lib*.a, and *.h files needed for compiling clients. Xctrb.tgz Contributed sources. Xlk98.tgz The link kit for building servers, Japanese PC98 version. Xlkit.tgz The link kit for building servers, normal PC architecture. Xsrc-1.tgz Part 1 of the complete sources. Xsrc-2.tgz Part 2 of the complete sources. Xsrc-3.tgz Part 3 of the complete sources. You will need Xprog.tgz if you intend to install ports of X software. XFree86 also includes a number of optional parts, such as documentation, and setup programs. Archive Description Xdoc.tgz READMEs Xjdoc.tgz READMEs in Japanese. Xps.tgz READMEs in PostScript. Xhtml.tgz READMEs in HTML. Xman.tgz Manual pages. Xcfg.tgz Customizable xinit and xdm runtime configuration files. Xset.tgz The X86Setup utility; a graphical version of the xf86config utility. Xjset.tgz The XF86Setup utility, Japanese version, for the normal PC architecture. XF86Setup is a graphical mode setup program for XFree86, and you may prefer it to the standard setup program xf86config. You do not need any special archives for xf86config; it is included in Xbin.tgz. The first time you install, you will need Xcfg.tgz to create your initial configuration files. Do not use it when upgrading; it overwrites your configuration files. There are also additional fonts that are available with XFree86: Archive Description Xf100.tgz 100 dpi fonts. Xfscl.tgz Speedo and Type1 fonts. Xfnon.tgz Japanese, Chinese, and other non-english fonts. Xfcyr.tgz Cyrillic fonts. Unlike the X servers described above, the archives for the following servers are all in the main directory. Archive Description Xfsrv.tgz The font server. Xnest.tgz A nested server running as a client window on another display. Xprt.tgz The print server. Xvfb.tgz The Virtual Framebuffer X server, which renders into memory or an mmapped file. Installing XFree86 Manually If you do not use sysinstall to install X, you need to perform a number of steps: Create the directories and unpack the required archives. Choose and install an X server. Set up the environment to be able to access X. Find a virtual terminal in which to run X. Configure X for your hardware. This sounds like a lot of work, but if you approach it methodically, it is not too bad. In the rest of this section, we will look at each step in turn. Unpacking the Archives You must unpack the archives as root, since a number of the executables are set-user-id (they run as root even when started by other users). If you unpack the server as an ordinary user, it may abort when you try to run it. You must also use a umask value of 022 (permissions rwxr-xr-x), because the X server requires special permissions. &prompt.user; su Password: &prompt.root; umask 022 If you do not have enough space in the /usr file system, create a directory on another partition and symlink it to /usr. For example, if you have a file system /home with adequate space, you could do: &prompt.root; cd /home &prompt.root; mkdir X11R6 &prompt.root; ln -s /home/X11R6 /usr/X11R6 Next, decide which archives you want to install. For a minimal installation, choose Xbin.tgz, Xfnts.tgz, Xlib.tgz, and Xcfg.tgz. If you have already configured X for your hardware, you can omit Xcfg.tgz. If you are using sh, unpack like this: &prompt.root; mkdir -p /usr/X11R6 &prompt.root; cd /usr/X11R6 &prompt.root; for i in bin fnts lib cfg; do &prompt.root; tar xzf X$i.tgz &prompt.root; done If you are using csh, enter: &prompt.root; mkdir -p /usr/X11R6 &prompt.root; cd /usr/X11R6 &prompt.root; foreach i (bin fnts lib cfg) ? tar xzf X$i.tgz ? end Installing the Server Choose a server archive corresponding to your VGA board. If the table in the section above does not give you enough information, check the server man pages, /usr/X11R6/man/man1/XF86_*, which list the VGA chipsets supported by each server. For example, if you have an ET4000 based board you will use the XF86_SVGA server. In this case you would enter: &prompt.root; cd /usr/X11R6 &prompt.root; tar xzf XSVGA.tgz [substitute your server name here] Setting up the environment Next, you may wish to create a symbolic link /usr/X11/bin/X that points to the server that matches your video board. In this example, it is the XF86_SVGA server: &prompt.root; cd /usr/X11R6/bin &prompt.root; rm X &prompt.root; ln -s XF86_SVGA X X needs this symbolic link in order to be able to work correctly, but you have the option of setting it when you run xf86config – see below. Next, check that the directory /usr/X11R6/bin is in the default path for sh in /etc/profile and for csh in /etc/csh.login, and add it if it is not. It is best to do this with an editor, but if you want to take a shortcut, you can enter: &prompt.root; echo 'PATH=$PATH:/usr/X11R6/bin' >>/etc/profile or: &prompt.root; echo 'set path = ($path /usr/X11R6/bin)' >>/etc/csh.login Alternatively, make sure everybody who uses X puts /usr/X11R6/bin in their shell's PATH variable. Next, invoke ldconfig to put the shared libraries in ld.so's cache: &prompt.root; ldconfig -m /usr/X11R6/lib You can omit invoking ldconfig if you plan to reboot before using X. You do not need to uncompress the font files, but if you do, you must run mkfontdir in the corresponding font directory, otherwise your server will abort with the message could not open default font `fixed'. Assigning a virtual terminal to X Next, make sure you have a spare virtual console which is running a getty. First check how many virtual consoles you have: &prompt.root; dmesg | grep virtual sc0: VGA color <16 virtual consoles, flags=0x0> Then check /etc/ttys to make sure there is at least one virtual terminal (ttyvxx device) which does not have a getty enabled. Look for the keyword off: &prompt.root; grep ttyv /etc/ttys ttyv0 "/usr/libexec/getty Pc" cons25 on secure ttyv1 "/usr/libexec/getty Pc" cons25 on secure ttyv2 "/usr/libexec/getty Pc" cons25 on secure ttyv3 "/usr/libexec/getty Pc" cons25 off secure In this case, /dev/ttyv3 is available, if your kernel has least 4 VTs. If not, either disable a getty in /etc/ttys by changing on to off, or build another kernel with more virtual terminals. Configuring X for Your Hardware After installing the X software, you will need to customize the file XF86Config, which tells the X server about your hardware and how you want to run it. In order to set up XF86Config, you will need the following hardware information: Your mouse type, the bit rate if it is a serial mouse, and the name of the device to which it is connected. This will typically be /dev/ttyd0 or /dev/ttyd1 for a serial mouse, /dev/psm0 for a PS/2 mouse, or /dev/mse0 for a bus mouse. The type of the video board and the amount of display memory. If it is a no-name board, establish what VGA chip set it uses. The parameters of your monitor; vertical and horizontal frequency. Identifying the hardware How do you decide what your hardware is? The manufacturer should tell you, but very often the information you get about your display board and monitor is pitiful; Super VGA board with 76 Hz refresh rate and 16,777,216 colors. This tells you the maximum pixel depth (24 bits – - the number of colors is 2(pixel depth)), but it doesn't tell you anything else about the display board. As we will see later, the real parameters you need to know are the maximum horizontal frequency, the dot clock range, the chipset and the amount of display memory. You could be unlucky trying to get some of this information, but you can get some with the SuperProbe program. It should always be able to tell you the chipset and the amount of memory on board. Occasionally SuperProbe can crash your system. Make sure you are not doing anything important when you run it. Running SuperProbe looks like this: &prompt.root; SuperProbe (warnings and acknowledgments omitted) First video: Super-VGA Chipset: Tseng ET4000 (Port Probed) Memory: 1024 Kbytes RAMDAC: Generic 8-bit pseudo-color DAC (with 6-bit wide lookup tables (or in 6-bit mode)) SuperProbe is very finicky about running at all, and you will often get messages like: SuperProbe: Cannot be run while an X server is running SuperProbe: If an X server is not running, unset $DISPLAY and try again SuperProbe: Cannot open video In other words, even if no X server is running, SuperProbe will not work if you have the environment variable DISPLAY set. How do you unset it? With Bourne-style shells, you enter: &prompt.root; unset DISPLAY In the C shell, you enter: &prompt.root; unsetenv DISPLAY Running <command>xf86config</command> The easy way to create your configuration file is with one of the utilities xf86config (note the lower case name) or XF86Setup. Both lead you through the configuration step by step. xf86config runs in character mode, while XF86Setup runs in a graphical mode. XF86Setup can have problems with unusual hardware, so I personally prefer xf86config. You can also use sysinstall, but this does not change much; sysinstall just starts xf86config for you, and it is easier to start it directly. In this section, we will use an example to illustrate configuration via xf86config. We are installing X for an ancient Diamond SpeedStar with 1 MB of display memory, a Logitech MouseMan mouse, and an ADI MicroScan 5AP monitor. The mouse is connected to the system via the first serial port, /dev/ttyd0. To run xf86config, type in the name. If /usr/X11R6/bin is included in your PATH environment variable, you just need to type xf86config. If it is not, you need to type out the full path to xf86config, like so: &prompt.root; /usr/X11R6/bin/xf86config This program will create a basic XF86Configfile, based on menu selections you make. The XF86Config file usually resides in /usr/X11R6/lib/X11 or /etc. A sample XF86Config file is supplied with XFree86; it is configured for a standard VGA card and monitor with 640x480 resolution. This program will ask for a pathname when it is ready to write the file. You can either take the sample XF86Config as a base and edit it for your configuration, or let this program produce a base XF86Config file for your configuration and fine-tune it. Refer to /usr/X11R6/lib/X11/doc/README.Config for a detailed overview of the configuration process. For accelerated servers (including accelerated drivers in the SVGA server), there are many chipset and card-specific options and settings. This program does not know about these. On some configurations some of these settings must be specified. Refer to the server man pages and chipset-specific READMEs. Before continuing with this program, make sure you know the chipset and amount of video memory on your video card. SuperProbe can help with this. It is also helpful if you know what server you want to run. Press enter to continue, or ctrl-c to abort. ENTER First specify a mouse protocol type. Choose one from the following list: 1. Microsoft compatible (2-button protocol) 2. Mouse Systems (3-button protocol) 3. Bus Mouse 4. PS/2 Mouse 5. Logitech Mouse (serial, old type, Logitech protocol) 6. Logitech MouseMan (Microsoft compatible) 7. MM Series 8. MM HitTablet 9. Microsoft IntelliMouse If you have a two-button mouse, it is most likely of type 1, and if you have a three-button mouse, it can probably support both protocol 1 and 2. There are two main varieties of the latter type; mice with a switch to select the protocol, and mice that default to 1 and require a button to be held at boot-time to select protocol 2. Some mice can be convinced to do 2 by sending a special sequence to the serial port (see the ClearDTR/ClearRTS options). Enter a protocol number: 6 Logitech MouseMan You have selected a Logitech MouseMan type mouse. You might want to enable ChordMiddle which could cause the third button to work. Please answer the following question with either 'y' or 'n'. Do you want to enable ChordMiddle? n You definitely want to enable the third button on your mouse, since many X clients use it. With a genuine Logitech mouse, however, you don't need to enable ChordMiddle in order to use the button. If you find that the third button does not work when you start X, you can enable ChordMiddle by editing the configuration file – it is much easier and less error-prone than re-running XF86Setup. Continuing through the setup: If your mouse has only two buttons, it is recommended that you enable Emulate3Buttons. Please answer the following question with either 'y' or 'n'. Do you want to enable Emulate3Buttons? n Now give the full device name that the mouse is connected to, for example /dev/tty00. Just pressing enter will use the default, /dev/mouse. Mouse device: /dev/ttyd1 Be very careful about this entry. You must specify the correct name for the device to which the mouse is connected. xf86config is not specific to FreeBSD, and the suggested example is just plain wrong for FreeBSD. Use the names /dev/ttyd0 through /dev/ttyd3 for serial mice, /dev/psm0 for PS/2 mice or /dev/mse0 for a bus mouse. Continuing, we see: Beginning with XFree86 3.1.2D, you can use the new X11R6.1 XKEYBOARD extension to manage the keyboard layout. If you answer 'n' to the following question, the server will use the old method, and you have to adjust your keyboard layout with xmodmap. Please answer the following question with either 'y' or 'n'. Do you want to use XKB? y The following dialogue will allow you to select from a list of already preconfigured keymaps. If you don't find a suitable keymap in the list, the program will try to combine a keymap from additional information you are asked then. Such a keymap is by default untested and may require manual tuning. Please report success or required changes for such a keymap to XFREE86@XFREE86.ORG for addition to the list of preconfigured keymaps in the future. Press enter to continue, or ctrl-c to abort. List of preconfigured keymaps: 1 Standard 101-key, US encoding 2 Microsoft Natural, US encoding 3 KeyTronic FlexPro, US encoding 4 Standard 101-key, US encoding with ISO9995-3 extensions 5 Standard 101-key, German encoding 6 Standard 101-key, French encoding 7 Standard 101-key, Thai encoding 8 Standard 101-key, Swiss/German encoding 9 Standard 101-key, Swiss/French encoding 10 None of the above Enter a number to choose the keymap. 1 Choose the standard US keyboard Now we want to set the specifications of the monitor. The two critical parameters are the vertical refresh rate, which is the rate at which the whole screen is refreshed, and most importantly the horizontal sync rate, which is the rate at which scanlines are displayed. The valid range for horizontal sync and vertical sync should be documented in the manual of your monitor. If in doubt, check the monitor database /usr/X11R6/lib/X11/doc/Monitors to see if your monitor is there. Press enter to continue, or ctrl-c to abort. ENTER You must indicate the horizontal sync range of your monitor. You can either select one of the predefined ranges below that correspond to industry- standard monitor types, or give a specific range. It is VERY IMPORTANT that you do not specify a monitor type with a horizontal sync range that is beyond the capabilities of your monitor. If in doubt, choose a conservative setting. hsync in kHz; monitor type with characteristic modes 1 31.5; Standard VGA, 640x480 @@ 60 Hz 2 31.5 - 35.1; Super VGA, 800x600 @@ 56 Hz 3 31.5, 35.5; 8514 Compatible, 1024x768 @@ 87 Hz interlaced (no 800x600) 4 31.5, 35.15, 35.5; Super VGA, 1024x768 @@ 87 Hz interlaced, 800x600 @@ 56 Hz 5 31.5 - 37.9; Extended Super VGA, 800x600 @@ 60 Hz, 640x480 @@ 72 Hz 6 31.5 - 48.5; Non-Interlaced SVGA, 1024x768 @@ 60 Hz, 800x600 @@ 72 Hz 7 31.5 - 57.0; High Frequency SVGA, 1024x768 @@ 70 Hz 8 31.5 - 64.3; Monitor that can do 1280x1024 @@ 60 Hz 9 31.5 - 79.0; Monitor that can do 1280x1024 @@ 74 Hz 10 31.5 - 82.0; Monitor that can do 1280x1024 @@ 76 Hz 11 Enter your own horizontal sync range Enter your choice (1-11): Unfortunately, our monitor is not mentioned in the file /usr/X11R6/lib/X11/doc/Monitors, but by chance the manual does specify the frequency range in the Technical Data section. The horizontal frequency range is from 30 to 64 kHz, and the vertical frequency range is from 50 to 100 Hz. The horizontal frequency range is almost exactly covered by choice 8, but that setting threatens to go 0.3 kHz higher in frequency than the technical data state. Do you want to risk it? Doing so will most likely not be a problem, since it is unlikely that the monitor will die at such a small deviation from the specs, and it is also unlikely that your XF86Config will actually generate a horizontal frequency between 64.0 and 64.3 kHz. However, there is no need to take even this slight risk. Just specify the real values: Enter your choice (1-11): 11 Please enter the horizontal sync range of your monitor, in the format used in the table of monitor types above. You can either specify one or more continuous ranges (e.g. 15-25, 30-50), or one or more fixed sync frequencies. Horizontal sync range: 30-64 Next, we select the vertical frequency range: You must indicate the vertical sync range of your monitor. You can either select one of the predefined ranges below that correspond to industry-standard monitor types, or give a specific range. For interlaced modes, the number that counts is the high one (e.g., 87 Hz rather than 43 Hz). 1 50-70 2 50-90 3 50-100 4 40-150 5 Enter your own vertical sync range Enter your choice: 3 exactly the range of the monitor The next step is to specify identification strings. You can think out names if you want, but unless you are juggling a lot of different hardware, you can let xf86config do it for you: You must now enter a few identification/description strings, namely an identifier, a vendor name, and a model name. Just pressing enter will fill in default names. The strings are free-form, spaces are allowed. Enter an identifier for your monitor definition: ENTER Enter the vendor name of your monitor: ENTER Enter the model name of your monitor: ENTER Next comes the choice of the video board. We have an elderly Diamond SpeedStar Plus with an ET4000 chip, and unknown Ramdac and Clock Chip. Let's see how we fare: Now we must configure video card specific settings. At this point you can choose to make a selection out of a database of video card definitions. Because there can be variation in Ramdacs and clock generators even between cards of the same model, it is not sensible to blindly copy the settings (e.g., a Device section). For this reason, after you make a selection, you will still be asked about the components of the card, with the settings from the chosen database entry presented as a strong hint. The database entries include information about the chipset, what server to run, the Ramdac and ClockChip, and comments that will be included in the Device section. However, a lot of definitions only hint about what server to run (based on the chipset the card uses) and are untested. If you can't find your card in the database, there's nothing to worry about. You should only choose a database entry that is exactly the same model as your card; choosing one that looks similar is just a bad idea (e.g. a GemStone Snail 64 may be as different from a GemStone Snail 64+ in terms of hardware as can be). Do you want to look at the card database? y 0 2 the Max MAXColor S3 Trio64V+ S3 Trio64V+ 1 928Movie S3 928 2 AGX (generic) AGX-014/15/16 3 ALG-5434(E) CL-GD5434 4 ASUS 3Dexplorer RIVA128 5 ASUS PCI-AV264CT ATI-Mach64 6 ASUS PCI-V264CT ATI-Mach64 7 ASUS Video Magic PCI V864 S3 864 8 ASUS Video Magic PCI VT64 S3 Trio64 9 AT25 Alliance AT3D 10 AT3D Alliance AT3D 11 ATI 3D Pro Turbo ATI-Mach64 12 ATI 3D Xpression ATI-Mach64 13 ATI 3D Xpression+ PC2TV ATI-Mach64 14 ATI 8514 Ultra (no VGA) ATI-Mach8 15 ATI All-in-Wonder ATI-Mach64 16 ATI Graphics Pro Turbo ATI-Mach64 17 ATI Graphics Pro Turbo 1600 ATI-Mach64 Enter a number to choose the corresponding card definition. Press enter for the next page, q to continue configuration. ENTER Dozens of board definitions come in alphabetic order. Finally we see: 108 DSV3325 S3 ViRGE 109 DSV3326 S3 Trio64V+ 110 DataExpert DSV3325 S3 ViRGE 111 DataExpert DSV3365 S3 Trio64V+ 112 Dell S3 805 S3 801/805 113 Dell onboard ET4000 ET4000 114 Diamond Edge 3D nv1 115 Diamond Multimedia Stealth 3D 2000 S3 ViRGE 116 Diamond Multimedia Stealth 3D 2000 PRO S3 ViRGE/DX 117 Diamond SpeedStar (Plus) ET4000 118 Diamond SpeedStar 24 ET4000 119 Diamond SpeedStar 24X (not fully supported) WD90C31 120 Diamond SpeedStar 64 CL-GD5434 121 Diamond SpeedStar HiColor ET4000 122 Diamond SpeedStar Pro (not SE) CL-GD5426/28 123 Diamond SpeedStar Pro 1100 CL-GD5420/2/4/6/8/9 124 Diamond SpeedStar Pro SE (CL-GD5430/5434) CL-GD5430/5434 125 Diamond SpeedStar64 Graphics 2000/2200 CL-GD5434 Enter a number to choose the corresponding card definition. Press enter for the next page, q to continue configuration. 117 Your selected card definition: Identifier: Diamond SpeedStar (Plus) Chipset: ET4000 Server: XF86_SVGA Press enter to continue, or ctrl-c to abort.ENTER Now you must determine which server to run. Refer to the man pages and other documentation. The following servers are available (they may not all be installed on your system): 1 The XF86_Mono server. This a monochrome server that should work on any VGA-compatible card, in 640x480 (more on some SVGA chipsets). 2 The XF86_VGA16 server. This is a 16-color VGA server that should work on any VGA-compatible card. 3 The XF86_SVGA server. This is a 256 color SVGA server that supports a number of SVGA chipsets. On some chipsets it is accelerated or supports higher color depths. 4 The accelerated servers. These include XF86_S3, XF86_Mach32, XF86_Mach8, XF86_8514, XF86_P9000, XF86_AGX, XF86_W32, XF86_Mach64, XF86_I128 and XF86_S3V. These four server types correspond to the four different "Screen" sections in XF86Config (vga2, vga16, svga, accel). 5 Choose the server from the card definition, XF86_SVGA. Which one of these screen types do you intend to run by default (1-5)? The system already chose XF86_SVGA for us. Do we want to change? We would need a good reason. In this case, we do not have a reason, so we will keep the server from the card definition: Which one of these screen types do you intend to run by default (1-5)? 5 The server to run is selected by changing the symbolic link 'X'. For example, the SVGA server. Please answer the following question with either 'y' or 'n'. Do you want me to set the symbolic link? y All the programs that start X (xinit, startx, and xdm) start a program /usr/X11R6/bin/X. This symbolic link makes /usr/X11R6/bin/X point to your X server. If you don't have a link, you will not be able to start X. Now you must give information about your video card. This will be used for the "Device" section of your video card in XF86Config. You must indicate how much video memory you have. It is probably a good idea to use the same approximate amount as that detected by the server you intend to use. If you encounter problems that are due to the used server not supporting the amount memory you have (e.g. ATI Mach64 is limited to 1024K with the SVGA server), specify the maximum amount supported by the server. How much video memory do you have on your video card: 1 256K 2 512K 3 1024K 4 2048K 5 4096K 6 Other Enter your choice: 3 You must now enter a few identification/description strings, namely an identifier, a vendor name, and a model name. Just pressing enter will fill in default names (possibly from a card definition). Your card definition is Diamond SpeedStar (Plus). The strings are free-form, spaces are allowed. Enter an identifier for your video card definition: ENTER You can simply press enter here if you have a generic card, or want to describe your card with one string. Enter the vendor name of your video card: ENTER Enter the model (board) name of your video card: ENTER Especially for accelerated servers, Ramdac, Dacspeed and ClockChip settings or special options may be required in the Device section. The RAMDAC setting only applies to the S3, AGX, W32 servers, and some drivers in the SVGA servers. Some RAMDAC's are auto-detected by the server. The detection of a RAMDAC is forced by using a Ramdac "identifier" line in the Device section. The identifiers are shown at the right of the following table of RAMDAC types: 1 AT&T 20C490 (S3 and AGX servers, ARK driver) att20c490 2 AT&T 20C498/21C498/22C498 (S3, autodetected) att20c498 3 AT&T 20C409/20C499 (S3, autodetected) att20c409 4 AT&T 20C505 (S3) att20c505 5 BrookTree BT481 (AGX) bt481 6 BrookTree BT482 (AGX) bt482 7 BrookTree BT485/9485 (S3) bt485 8 Sierra SC15025 (S3, AGX) sc15025 9 S3 GenDAC (86C708) (autodetected) s3gendac 10 S3 SDAC (86C716) (autodetected) s3_sdac 11 STG-1700 (S3, autodetected) stg1700 12 STG-1703 (S3, autodetected) stg1703 Enter a number to choose the corresponding RAMDAC. Press enter for the next page, q to quit without selection of a RAMDAC. q We don't need this A Clockchip line in the Device section forces the detection of a programmable clock device. With a clockchip enabled, any required clock can be programmed without requiring probing of clocks or a Clocks line. Most cards don't have a programmable clock chip. Choose from the following list: 1 Chrontel 8391 ch8391 2 ICD2061A and compatibles (ICS9161A, DCS2824) icd2061a 3 ICS2595 ics2595 4 ICS5342 (similar to SDAC, but not completely compatible) ics5342 5 ICS5341 ics5341 6 S3 GenDAC (86C708) and ICS5300 (autodetected) s3gendac 7 S3 SDAC (86C716) s3_sdac 8 STG 1703 (autodetected) stg1703 9 Sierra SC11412 sc11412 10 TI 3025 (autodetected) ti3025 11 TI 3026 (autodetected) ti3026 12 IBM RGB 51x/52x (autodetected) ibm_rgb5xx Just press enter if you don't want a Clockchip setting. What Clockchip setting do you want (1-12)? ENTER For most configurations, a Clocks line is useful since it prevents the slow and nasty sounding clock probing at server start-up. Probed clocks are displayed at server startup, along with other server and hardware configuration info. You can save this information in a file by running imprecise; some clocks may be slightly too high (varies per run). At this point I can run X -probeonly, and try to extract the clock information from the output. It is recommended that you do this yourself and add a clocks line (note that the list of clocks may be split over multiple Clocks lines) to your Device section afterwards. Be aware that a clocks line is not appropriate for drivers that have a fixed set of clocks and don't probe by default (e.g. Cirrus). Also, for the P9000 server you must simply specify clocks line that matches the modes you want to use. For the S3 server with a programmable clock chip you need a 'ClockChip' line and no Clocks line. You must be root to be able to run X -probeonly now. Do you want me to run 'X -probeonly' now? This last question is worth thinking about. You should run X -probeonly at some point, but it requires some extra work. We'll take the recommendation and try it later. Do you want me to run 'X -probeonly' now? n For each depth, a list of modes (resolutions) is defined. The default resolution that the server will start-up with will be the first listed mode that can be supported by the monitor and card. Currently it is set to: "640x480" "800x600" "1024x768" for 8bpp "640x480" "800x600" for 16bpp "640x480" for 24bpp "640x400" for 32bpp Note that 16, 24 and 32bpp are only supported on a few configurations. Modes that cannot be supported due to monitor or clock constraints will be automatically skipped by the server. 1 Change the modes for 8pp (256 colors) 2 Change the modes for 16bpp (32K/64K colors) 3 Change the modes for 24bpp (24-bit color, packed pixel) 4 Change the modes for 32bpp (24-bit color) 5 The modes are OK, continue. Enter your choice: 5 accept the defaults You can have a virtual screen (desktop), which is screen area that is larger than the physical screen and which is panned by moving the mouse to the edge of the screen. If you don't want virtual desktop at a certain resolution, you cannot have modes listed that are larger. Each color depth can have a differently-sized virtual screen Please answer the following question with either 'y' or 'n'. Do you want a virtual screen that is larger than the physical screen? n It is difficult to decide whether you want a virtual screen larger than the physical screen. I find it extremely disturbing, so I suggest you answer n. You might find it useful, especially if your highest resolution is small. Now the configuration is complete, and sysinstall just need to write the configuration file: I am going to write the XF86Config file now. Make sure you don't accidently overwrite a previously configured one. Shall I write it to /etc/XF86Config? y File has been written. Take a look at it before running 'startx'. Note that the XF86Config file must be in one of the directories searched by the server (e.g. /usr/X11R6/lib/X11) in order to be used. Within the server press ctrl, alt and '+' simultaneously to cycle video resolutions. Pressing ctrl, alt and backspace simultaneously immediately exits the server (use if the monitor doesn't sync for a particular mode). For further configuration, refer to /usr/X11R6/lib/X11/doc/README.Config. Once you have completed this configuration, you are ready to start X. Using Fonts in XFree86 TrueType Fonts The default fonts that ship with XFree86 are less than ideal for typical desktop publishing applications. Large presentation fonts show up jagged and unprofessional looking and small fonts in Netscape are almost completely unintelligible. Fortunately, XFree86 can be configured to use TrueType fonts with a minimum of effort. XFree86 4.0 has built in support for rendering TrueType fonts. There are two different modules that can enable this functionality. The "freetype" module is used in this example because it is more consistent with the other font rendering backends. To enable the freetype module just add the following line to the module section of your /etc/X11/XF86Config file. - - Load "freetype" - +Load "freetype" For XFree86 3.3.X you will need to run a separate TrueType font server. Xfstt is commonly used for this purpose. To install Xfstt on your FreeBSD system simply install the port from /usr/ports/x11-servers/Xfstt You should now make a directory for your TrueType fonts (e.g. /usr/X11R6/lib/X11/fonts/TrueType) and copy all of your TrueType fonts into this directory. Keep in mind that you can not take TrueType fonts directly from a Macintosh; they must be in Unix/DOS/Windows format for use by XFree86. Once you have copied the files into this directory you need to use ttmkfdir to create a fonts.dir file so that the X font renderer knows that you've installed these new files. There is a FreeBSD port for ttmkfdir in /usr/ports/x11-fonts/ttmkfdir. - - &prompt.root; cd /usr/X11R6/lib/X11/fonts/TrueType - &prompt.root; ttmkfdir > fonts.dir - +&prompt.root; cd /usr/X11R6/lib/X11/fonts/TrueType + &prompt.root; ttmkfdir > fonts.dir Now you need to add your TrueType directory to your fonts path. The easiest way to do this is to add the following entries into your ~/.xinitrc file. - - &prompt.user; xset fp+ /usr/X11R6/lib/X11/fonts/TrueType - &prompt.user; xset fp rehash - +&prompt.user; xset fp+ /usr/X11R6/lib/X11/fonts/TrueType + &prompt.user; xset fp rehash That's it. Now Netscape, Gimp, StarOffice, and all of your other X applications should now recognize your installed TrueType fonts. Extremely small fonts (as with text in a high resolution display on a web page) and extremely large fonts (within StarOffice) will look much better now. Starting with version 4.0.2, XFree86 supports antialiased fonts. Currently, most software has not been updated to take advantage of this new functionality. However, Qt (the toolkit for the KDE desktop) does; so if you are running XFree86 4.0.2 (or higher), Qt 2.3 (or higher) and KDE, all your KDE/Qt applications will use anti-aliased fonts. The X Display Manager Overview The X Display Manager (XDM) is an optional part of the X Window System that is used for login session management. This is useful for several types of situations, including minimal X Terminals (see section ), desktops, and large network display servers. Since the X Window System is network and protocol independent, there are a wide variety of possible configurations for running X clients and servers on different machines connected by a network. XDM provides a graphical interface for choosing which display server to connect to, and entering authorization information such as a login and password combination. You may think of XDM as providing the same functionality to the user as the &man.getty.8; utility (see for details). That is, it performs system logins to the display being connected to and then runs a session manager on behalf of the user (usually an X window manager). XDM then waits for this program to exit, signaling that the user is done and should be logged out of the display. At this point, XDM can display the login and display chooser screens for the next user to login. Using XDM The XDM daemon program is located in /usr/X11R6/bin/xdm. You can run this program at any time as root and it will start managing the X display on the local machine. If you want XDM to run in the background every time the machine boots up, a convenient way to do this is by adding an entry to /etc/ttys. For more information about the format and usage of this file, see . There is a line in the default /etc/ttys file for running the xdm daemon on a virtual terminal: - -ttyv8 "/usr/X11R6/bin/xdm -nodaemon" xterm off secure - +ttyv8 "/usr/X11R6/bin/xdm -nodaemon" xterm off secure By default this entry is disabled, and in order to enable it you will need to change field 5 from off to on and then restart &man.init.8; using the directions in . The first field, the name of the terminal this program will manage, is ttyv8. This means that XDM will start running on the 9th virtual terminal. Configuring XDM The XDM configuration directory is located in /usr/X11R6/lib/X11/xdm. In this directory you will see several files used to change the behavior and appearance of XDM. Typically you will find these files: File Description Xaccess Client authorization ruleset. Xresources Default X resource values. Xservers List of remote and local displays to manage. Xsession Default session script for logins. Xsetup_* xdm-config Global configuration for all displays running on this machine. xdm-errors Errors generated by the server program. xdm-pid The process ID of the currently running XDM. Also in this directory are a few scripts and programs used to setup the desktop when XDM is running. In the next few sections I will briefly describe the purpose of each of these files. The exact syntax and usage of all of these files is described in &man.xdm.1; The default configuration is a simple rectangular login window with the hostname of the machine displayed at the top in a large font and Login: and Password: prompts below. This is a good starting point if you are planning to design your own look and feel for the XDM screens. Xaccess The protocol for connecting to XDM controlled displays is called the X Display Manager Connection Protocol (XDMCP). This file is basically just a ruleset for controlling XDMCP connections from remote machines. By default, it allows any client to connect, but you will see this will not matter because the default xdm-config file does not listen for remote connections. Xresources This is an application-defaults file for the display chooser and the login screens. This is where you can customize the appearance of the login program. The format is identical to the app-defaults file described in the XFree86 documentation. Xservers This is a list of the remote displays the chooser should provide as choices. Xsession This is the default session script for XDM to run after a user has logged in. Normally each user will have a customized session script in ~/.xsessionrc that overrides this script. Xsetup_* These files contain scripts that will be run automatically before displaying the chooser or login interfaces. There is a script for each display being used, named Xsetup_followed by the local display number (for instance Xsetup_0). Typically these scripts will run one or two programs in the background such as xconsole. xdm-config This file contains settings in the form of app-defaults that are applicable to every display that this installation manages. xdm-errors This file contains the output of the X servers that XDM is trying to run. If a display that XDM is trying to start hangs for some reason, this is a good place to look for error messages. These messages are also written to the user's ~/.xsession-errors file on a per-session basis Running A Network Display Server In order for other clients to connect to your display server, you will need to edit the access control rules, and enable the connection listener. By default these are set to conservative values, which is a good decision security-wise. To get XDM to listen for connections first comment out a line in the xdm-config file: - -! SECURITY: do not listen for XDMCP or Chooser requests +! SECURITY: do not listen for XDMCP or Chooser requests ! Comment out this line if you want to manage X terminals with xdm -DisplayManager.requestPort: 0 - +DisplayManager.requestPort: 0 and then restart XDM. Remember that comments in app-defaults files begin with a ! character, not a #. After this, you may need to put more strict access controls in place. Look at the example entries in Xaccess file, and refer to the &man.xdm.1; manual page. Desktop Environments Written by &a.logo;, June 2001 This section describes the different desktop environments available for X-Windows on FreeBSD. For our purposes a "desktop environment" will mean anything ranging from a simple window manager, to a complete suite of desktop applications such as KDE or GNOME. GNOME About GNOME GNOME is a user-friendly desktop environment that enables users to easily use and configure their computers. GNOME includes a panel (for starting applications and displaying status), a desktop (where data and applications can be placed), a set of standard desktop tools and applications, and a set of conventions that make it easy for applications to cooperate and be consistent with each other. Users of other operating systems or environments should feel right at home using the powerful graphics-driven environment that GNOME provides. Installing GNOME To install GNOME from the network, simply type: &prompt.root; pkg_add -r gnome If you would rather build GNOME from source, then use the ports tree: &prompt.root; cd /usr/ports/x11/gnome &prompt.root; make install clean Once GNOME is installed, we must have the X server start GNOME instead of a default window mananger. If you have already customized your .xinitrc file then you should simply replace the line that starts your current window manager with one that starts /usr/X11R6/bin/gnome-wm instead. If you haven't added anything special to your configuration file, then it is enough to simply type: &prompt.root; echo "/usr/X11R6/bin/gnome-wm" > ~/.xinirc That's it. Type 'startx' and you will be in the GNOME desktop environment. Note: if you're running a display manager like xdm, this will not work. Instead, you should create an executable .xsession file with the same command in it. To do this, edit your file (if you already have one) and replace the existing wm command with /usr/X11R6/bin/gnome-wm; or else, &prompt.root; echo "#!/bin/sh" > ~/.xsession &prompt.root; echo "/usr/X11R6/bin/gnome-wm" >> ~/.xsession &prompt.root; chmod +x ~/.xsession Another option is to configure your display manager to allow choosing the window manager at runtime; the section on KDE2 details explains how to do this for kdm, the display manager of KDE. KDE2 About KDE2 KDE is an easy to use contemporary desktop environment. Some of the things that KDE brings to the user are: A beautiful contemporary desktop A desktop exhibiting complete network transparency An integrated help system allowing for convenient, consistent access to help on the use of the KDE desktop and its applications Consistent look and feel of all KDE applications Standardized menu and toolbars, keybindings, color-schemes, etc. Internationalization: KDE is available in more than 40 languages Centralized consisted dialog driven desktop configuration A great number of useful KDE applications KDE has an office application suite based on KDE's KParts technology consisting of a spread-sheet, a presentation application, an organizer, a news client and more. KDE is also comes with a web browser called Konqeuror, which represents already a solid competitor to other existing web browsers on Unix systems. More information on KDE can be found on the KDE website Installing KDE2 At the time of writing, a package for kde2 doesn't exist yet. No problem! The ports tree hides all the complexity of building a package from source. To install KDE2, do this : &prompt.root; cd /usr/ports/x11/kde2 &prompt.root; make install clean This command will fetch all the necessary files from the Internet, configure and compile KDE2, install the applications, and then clean up after itself. Now you're going to have to tell the X server to launch KDE2 instead of a default window manager. Do this by typing this: &prompt.root; echo "/usr/X11R6/bin/startkde" > ~/.xinitrc Now, whenever you go into X-Windows, KDE2 will be your desktop. (Note: this will not work if you're logging in via a display manager like xdm. In that case you have two options: create an .xsession file as described in the section on GNOME, but with the /usr/X11R6/bin/startkde command instead of the gnome-wm command; or, configure your display manager to allow choosing a desktop at login time. Below it is explained how to do this for kdm, KDE's display manager. More details on KDE2 Now that KDE2 is installed on your system, you'll find that you can learn a lot from its help pages, or just by pointing and clicking at various menus. Windows or Mac users will feel quite at home. The best reference for KDE is the on-line documentation. KDE comes with its own web browser, Konqueror, dozens of useful applications, and extensive documentation. This section only discusses somewhat technical things which are difficult to learn just by random exploration. The KDE desktop manager If you're an administrator on a multi-user system, you may like to have a graphical login screen to welcome users. You can use xdm, as described earlier. However, KDE includes an alternative, kdm, which is designed to look more attractive and include more login-time options. In particular, users can easily choose (via a menu) which desktop environment (KDE2, GNOME, or something else) to run at runtime. If you're slightly adventurous and you want this added flexibility and visual appeal, read on. To begin with, run the KDE2 control panel, kcontrol, as root. Note: it is generally considered unsafe to run your entire X environment as root. Instead, run your window manager as a normal user, open a terminal window (such as xterm or KDE's konsole, become root with su (you need to be in the "wheel" group in /etc/group for this), and then type kcontrol. Click on the icon on the left marked "System", then on "Login manager". On the right you'll see various configurable options, which the KDE manual will explain in greater detail. Click on "sessions" on the right. Depending on what window managers or desktop environments you have currently installed, you can type their names in "New type" and add them. (These are just labels so far, not commands, so you can write "KDE" and "GNOME" rather than "startkde" or "gnome-wm".) Include a label "failsafe". Play with the other menus as you like (those are mainly cosmetic and self-explanatory). When you're done, click on "Apply" at the bottom, and quit the control center. To make sure kdm understands what your above labels (KDE, GNOME etc) mean, you need to edit some more files: the same ones used by xdm. In your terminal window, as root, edit the file /usr/X11R6/lib/X11/xdm/Xsession. You will come across a section in the middle looking like this (by default): - -case $# in +case $# in 1) case $1 in failsafe) exec xterm -geometry 80x24-0-0 ;; esac -esac - +esac You will need to add a few lines to this section. Assuming the labels you gave earlier were KDE2 and GNOME, the following will do: - -case $# in +case $# in 1) case $1 in KDE2) exec /usr/X11R6/bin/startkde ;; GNOME) exec /usr/X11R6/bin/gnome-wm ;; failsafe) exec xterm -geometry 80x24-0-0 ;; esac -esac - +esac To make sure your KDE choice of a login-time desktop background is also honored, you will need to add the following line to /usr/X11R6/lib/X11/xdm/Xsetup_0: - - /usr/X11R6/bin/kdmdesktop - + /usr/X11R6/bin/kdmdesktop Now, you need only to make sure kdm is started at the next bootup. To learn how to do this, read the section on xdm, and do the same thing replacing references to the xdm program by kdm. That's it. Your next login screen should have a pretty face and lots of menus. Anti-aliased fonts Tired of blocky staircase edges to your fonts under X11? Tired of unreadable text in web browsers? Well, no more. Starting with version 4.0.2, XFree86 supports anti-aliasing via its "RENDER" extension, and starting with version 2.3, Qt (the toolkit used by KDE) supports this extension. So if you're running up-to-date software, anti-aliasing is possible on your KDE2 desktop. Just go to your KDE2 menu, go to Preferences -> Look and Feel -> Style, and click on the checkbox "Use Anti-Aliasing for Fonts and Icons". That's all. A caveat: anti-aliasing works by replacing sharp black/white borders of fonts by shades of grey, in effect finely blurring them. This makes very small fonts more readable, and improves the appearance of very large fonts, but normal-sized fonts when anti-aliased can strain the eyes over long periods of time. It is possible to configure the X font server to anti-alias only certain fonts and only for certain (ranges of) font sizes; this involves editing the file /usr/X11R6/lib/X11/Xftconfig. For details, consult, for instance, the tutorial by Keith Packard (who introduced the "RENDER" extension). Anti-aliasing is still new to the FreeBSD/XFree86 world, but configuring it should become much easier with time. It is also available with the GNOME desktop using patches to the gtk+ toolkit; since these patches break internationalisation support, they are not officially included at present.