diff --git a/en_US.ISO8859-1/books/handbook/config/chapter.sgml b/en_US.ISO8859-1/books/handbook/config/chapter.sgml index d1ae9d3bbc..6da99b10e7 100644 --- a/en_US.ISO8859-1/books/handbook/config/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/config/chapter.sgml @@ -1,1229 +1,1229 @@ Chern Lee Written by Mike Smith Based on a tutorial written by Matt Dillon Also based on tuning(7) written by Configuration and Tuning Synopsis system configuration/optimization Configuring a system correctly can substantially reduce the amount of work involved in maintaining and upgrading it in the future. This chapter describes some of the aspects of administrative configuration of FreeBSD systems. This chapter will also describe some of the parameters that can be set to tune a FreeBSD system for optimum performance. After reading this chapter, you will know: Why and how to efficiently size, layout, and place filesystems and swap partitions on your hard drive. The basics of the rc.conf configuration and /usr/local/etc/rc.d startup systems. How to configure virtual hosts on your network devices. How to use the various configuration files in /etc. How to tune FreeBSD using sysctl variables. How to tune disk performance and modify kernel limitations. Before reading this chapter, you should: Understand the basics of Unix and FreeBSD (). Be familiar with keeping FreeBSD sources up to date (), and the basics of kernel configuration/compilation (). Initial Configuration Partition Layout Partition layout /etc /var /usr Base Partitions When laying out your filesystem with &man.disklabel.8; or &man.sysinstall.8;, it is important to remember that hard drives can transfer data at a faster rate from the outer tracks than the inner. Knowing this, you should place your smaller, heavily-accessed filesystems, such as root and swap, closer to the outside of the drive, while placing larger partitions, such as /usr, towards the inner. To do so, it is a good idea to create partitions in a similar order: root, swap, /var, /usr. The size of your /var partition reflects the intended use of your machine. /var is primarily used to hold mailboxes, log files, and printer spools. Mailboxes and log files, in particular, can grow to unexpected sizes based upon how many users are on your system and how long your log files are kept. If you intend to run a mail server, a /var partition of over a gigabyte can be suitable. Additionally, /var/tmp must be large enough to contain any packages you may wish to add. The /usr partition holds the bulk of the files required to support the system and a subdirectory within it called /usr/local holds the bulk of the files installed from the &man.ports.7; hierarchy. If you do not use ports all that much and do not intend to keep system source (/usr/src) on the machine, you can get away with a 1 gigabyte /usr partition. However, if you install a lot of ports (especially window managers and Linux binaries), we recommend at least a two gigabyte /usr and if you also intend to keep system source on the machine, we recommend a three gigabyte /usr. Do not underestimate the amount of space you will need in this partition, it can creep up and surprise you! When sizing your partitions, keep in mind the space requirements for your system to grow. Running out of space in one partition while having plenty in another can lead to much frustration. Some users who have used &man.sysinstall.8;'s Auto-defaults partition sizer have found either their root or /var partitions too small later on. Partition wisely and generously. Swap Partition swap sizing swap partition As a rule of thumb, your swap space should typically be double the amount of main memory. For example, if the machine has 128 megabytes of memory, the swap file should be 256 megabytes. Systems with lesser memory may perform better with a lot more swap. It is not recommended that you configure any less than 256 megabytes of swap on a system and you should keep in mind future memory expansion when sizing the swap partition. The kernel's VM paging algorithms are tuned to perform best when the swap partition is at least two times the size of main memory. Configuring too little swap can lead to inefficiencies in the VM page scanning code as well as create issues later on if you add more memory to your machine. Finally, on larger systems with multiple SCSI disks (or multiple IDE disks operating on different controllers), it is strongly recommend that you configure swap on each drive (up to four drives). The swap partitions on the drives should be approximately the same size. The kernel can handle arbitrary sizes but internal data structures scale to 4 times the largest swap partition. Keeping the swap partitions near the same size will allow the kernel to optimally stripe swap space across the disks. Do not worry about overdoing it a little, swap space is the saving grace of Unix. Even if you do not normally use much swap, it can give you more time to recover from a runaway program before being forced to reboot. Why Partition? Why partition at all? Why not create one big root partition and be done with it? Then I do not have to worry about undersizing things! There are several reasons this is not a good idea. First, each partition has different operational characteristics and separating them allows the filesystem to tune itself to those characteristics. For example, the root and /usr partitions are read-mostly, with very little writing, while a lot of reading and writing could occur in /var and /var/tmp. By properly partitioning your system, fragmentation introduced in the smaller more heavily write-loaded partitions will not bleed over into the mostly-read partitions. Additionally, keeping the write-loaded partitions closer to the edge of the disk, for example before the really big partition instead of after in the partition table, will increase I/O performance in the partitions where you need it the most. Now it is true that you might also need I/O performance in the larger partitions, but they are so large that shifting them more towards the edge of the disk will not lead to a significant performance improvement whereas moving /var to the edge can have a huge impact. Finally, there are safety concerns. Having a small, neat root partition that is essentially read-only gives it a greater chance of surviving a bad crash intact. Core Configuration rc files rc.conf The principal location for system configuration information is within /etc/rc.conf. This file contains a wide range of configuration information, principally used at system startup to configure the system. Its name directly implies this; it is configuration information for the rc* files. An administrator should make entries in the rc.conf file to override the default settings from /etc/defaults/rc.conf. The defaults file should not be copied verbatim to /etc - it contains default values, not examples. All system-specific changes should be made in the rc.conf file itself. A number of strategies may be applied in clustered applications to separate site-wide configuration from system-specific configuration in order to keep administration overhead down. The recommended approach is to place site-wide configuration into another file, such as /etc/rc.conf.site, and then include this file into /etc/rc.conf, which will contain only system-specific information. As rc.conf is read by &man.sh.1; it is trivial to achieve this. For example: rc.conf: . rc.conf.site hostname="node15.example.com" network_interfaces="fxp0 lo0" ifconfig_fxp0="inet 10.1.1.1" rc.conf.site: defaultrouter="10.1.1.254" saver="daemon" blanktime="100" The rc.conf.site file can then be distributed to every system using rsync or a similar program, while the rc.conf file remains unique. Upgrading the system using &man.sysinstall.8; or make world will not overwrite the rc.conf file, so system configuration information will not be lost. Application Configuration Typically, installed applications have their own configuration files, with their own syntax, etc. It is important that these files be kept separate from the base system, so that they may be easily located and managed by the package management tools. /usr/local/etc Typically, these files are installed in /usr/local/etc. In the case where an application has a large number of configuration files, a subdirectory will be created to hold them. Normally, when a port or package is installed, sample configuration files are also installed. These are usually identified with a .default suffix. If there are no existing configuration files for the application, they will be created by copying the .default files. For example, consider the contents of the directory /usr/local/etc/apache: -rw-r--r-- 1 root wheel 2184 May 20 1998 access.conf -rw-r--r-- 1 root wheel 2184 May 20 1998 access.conf.default -rw-r--r-- 1 root wheel 9555 May 20 1998 httpd.conf -rw-r--r-- 1 root wheel 9555 May 20 1998 httpd.conf.default -rw-r--r-- 1 root wheel 12205 May 20 1998 magic -rw-r--r-- 1 root wheel 12205 May 20 1998 magic.default -rw-r--r-- 1 root wheel 2700 May 20 1998 mime.types -rw-r--r-- 1 root wheel 2700 May 20 1998 mime.types.default -rw-r--r-- 1 root wheel 7980 May 20 1998 srm.conf -rw-r--r-- 1 root wheel 7933 May 20 1998 srm.conf.default The filesize difference shows that only the srm.conf file has been changed. A later update of the apache port would not overwrite this changed file. Starting Services services It is common for a system to host a number of services. These may be started in several different fashions, each having different advantages. /usr/local/etc/rc.d Software installed from a port or the packages collection will often place a script in /usr/local/etc/rc.d which is invoked at system startup with a argument, and at system shutdown with a argument. This is the recommended way for starting system-wide services that are to be run as root, or that expect to be started as root. These scripts are registered as part of the installation of the package, and will be removed when the package is removed. A generic startup script in /usr/local/etc/rc.d looks like: #!/bin/sh echo -n ' FooBar' case "$1" in start) /usr/local/bin/foobar ;; stop) kill -9 `cat /var/run/foobar.pid` ;; *) echo "Usage: `basename $0` {start|stop}" >&2 exit 64 ;; esac exit 0 The startup scripts of FreeBSD will look in /usr/local/etc/rc.d for scripts that have an .sh extension and are executable by root. Those scripts that are found are called with an option at startup, and at shutdown to allow them to carry out their purpose. So if you wanted the above sample script to be picked up and run at the proper time during system startup, you should save it to a file called FooBar.sh in /usr/local/etc/rc.d and make sure it's executable. You can make a shell script executable with &man.chmod.1; as shown below: &prompt.root; chmod 755 FooBar.sh Some services expect to be invoked by &man.inetd.8; when a connection is received on a suitable port. This is common for mail reader servers (POP and IMAP, etc.). These services are enabled by editing the file /etc/inetd.conf. See &man.inetd.8; for details on editing this file. Some additional system services may not be covered by the toggles in /etc/rc.conf. These are traditionally enabled by placing the command(s) to invoke them in /etc/rc.local. As of FreeBSD 3.1 there is no default /etc/rc.local; if it is created by the administrator it will however be honored in the normal fashion. Note that rc.local is generally regarded as the location of last resort; if there is a better place to start a service, do it there. Do not place any commands in /etc/rc.conf. To start daemons, or run any commands at boot time, place a script in /usr/local/etc/rc.d instead. It is also possible to use the &man.cron.8; daemon to start system services. This approach has a number of advantages, not least being that because &man.cron.8; runs these processes as the owner of the crontab, services may be started and maintained by non-root users. This takes advantage of a feature of &man.cron.8;: the time specification may be replaced by @reboot, which will cause the job to be run when &man.cron.8; is started shortly after system boot. Virtual Hosts virtual hosts ip aliases A very common use of FreeBSD is virtual site hosting, where one server appears to the network as many servers. This is achieved by assigning multiple network addresses to a single interface. A given network interface has one real address, and may have any number of alias addresses. These aliases are normally added by placing alias entries in /etc/rc.conf. An alias entry for the interface fxp0 looks like: ifconfig_fxp0_alias0="inet xxx.xxx.xxx.xxx netmask xxx.xxx.xxx.xxx" Note that alias entries must start with alias0 and proceed upwards in order, (for example, _alias1, _alias2, and so on). The configuration process will stop at the first missing number. The calculation of alias netmasks is important, but fortunately quite simple. For a given interface, there must be one address which correctly represents the network's netmask. Any other addresses which fall within this network must have a netmask of all 1's. For example, consider the case where the fxp0 interface is connected to two networks, the 10.1.1.0 network with a netmask of 255.255.255.0 and the 202.0.75.16 network with a netmask of 255.255.255.240. We want the system to appear at 10.1.1.1 through 10.1.1.5 and at 202.0.75.17 through 202.0.75.20. The following entries configure the adapter correctly for this arrangement: ifconfig_fxp0="inet 10.1.1.1 netmask 255.255.255.0" ifconfig_fxp0_alias0="inet 10.1.1.2 netmask 255.255.255.255" ifconfig_fxp0_alias1="inet 10.1.1.3 netmask 255.255.255.255" ifconfig_fxp0_alias2="inet 10.1.1.4 netmask 255.255.255.255" ifconfig_fxp0_alias3="inet 10.1.1.5 netmask 255.255.255.255" ifconfig_fxp0_alias4="inet 202.0.75.17 netmask 255.255.255.240" ifconfig_fxp0_alias5="inet 202.0.75.18 netmask 255.255.255.255" ifconfig_fxp0_alias6="inet 202.0.75.19 netmask 255.255.255.255" ifconfig_fxp0_alias7="inet 202.0.75.20 netmask 255.255.255.255" Configuration Files <filename>/etc</filename> Layout There are a number of directories in which configuration information is kept. These include: /etc Generic system configuration information; data here is system-specific. /etc/defaults Default versions of system configuration files. /etc/mail Extra &man.sendmail.8; configuration, other MTA configuration files. /etc/ppp Configuration for both user- and kernel-ppp programs. /etc/namedb Default location for &man.named.8; data. Normally named.conf and zone files are stored here. /usr/local/etc Configuration files for installed applications. May contain per-application subdirectories. /usr/local/etc/rc.d Start/stop scripts for installed applications. /var/db Automatically generated system-specific database files, such as the package database, the locate database, and so on Hostnames hostname DNS <filename>/etc/resolv.conf</filename> resolv.conf /etc/resolv.conf dictates how FreeBSD's resolver accesses the Internet Domain Name System (DNS). The most common entries to resolv.conf are: nameserver The IP address of a name server the resolver should query. The servers are queried in the order listed with a maximum of three. search Search list for hostname lookup. This is normally determined by the domain of the local hostname. domain The local domain name. A typical resolv.conf: search example.com nameserver 147.11.1.11 nameserver 147.11.100.30 Only one of the search and domain options should be used. If you are using DHCP, &man.dhclient.8; usually rewrites resolv.conf with information received from the DHCP server. <filename>/etc/hosts</filename> hosts /etc/hosts is a simple text database reminiscent of the old Internet. It works in conjunction with DNS and NIS providing name to IP address mappings. Local computers connected via a LAN can be placed in here for simplistic naming purposes instead of setting up a &man.named.8; server. Additionally, /etc/hosts can be used to provide a local record of Internet names, reducing the need to query externally for commonly accessed names. # $FreeBSD$ # # Host Database # This file should contain the addresses and aliases # for local hosts that share this file. # In the presence of the domain name service or NIS, this file may # not be consulted at all; see /etc/nsswitch.conf for the resolution order. # # ::1 localhost localhost.my.domain myname.my.domain 127.0.0.1 localhost localhost.my.domain myname.my.domain # # Imaginary network. #10.0.0.2 myname.my.domain myname #10.0.0.3 myfriend.my.domain myfriend # # According to RFC 1918, you can use the following IP networks for # private nets which will never be connected to the Internet: # # 10.0.0.0 - 10.255.255.255 # 172.16.0.0 - 172.31.255.255 # 192.168.0.0 - 192.168.255.255 # # In case you want to be able to connect to the Internet, you need # real official assigned numbers. PLEASE PLEASE PLEASE do not try # to invent your own network numbers but instead get one from your # network provider (if any) or from the Internet Registry (ftp to # rs.internic.net, directory `/templates'). # /etc/hosts takes on the simple format of: [Internet address] [official hostname] [alias1] [alias2] ... For example: 10.0.0.1 myRealHostname.example.com myRealHostname foobar1 foobar2 Consult &man.hosts.5; for more information. Log File Configuration log files <filename>syslog.conf</filename> syslog.conf syslog.conf is the configuration file for the &man.syslogd.8; program. It indicates which types of syslog messages are logged to particular log files. # $FreeBSD$ # # Spaces ARE valid field separators in this file. However, # other *nix-like systems still insist on using tabs as field # separators. If you are sharing this file between systems, you # may want to use only tabs as field separators here. # Consult the syslog.conf(5) manual page. *.err;kern.debug;auth.notice;mail.crit /dev/console *.notice;kern.debug;lpr.info;mail.crit;news.err /var/log/messages security.* /var/log/security mail.info /var/log/maillog lpr.info /var/log/lpd-errs cron.* /var/log/cron *.err root *.notice;news.err root *.alert root *.emerg * # uncomment this to log all writes to /dev/console to /var/log/console.log #console.info /var/log/console.log # uncomment this to enable logging of all log messages to /var/log/all.log #*.* /var/log/all.log # uncomment this to enable logging to a remote log host named loghost #*.* @loghost # uncomment these if you're running inn # news.crit /var/log/news/news.crit # news.err /var/log/news/news.err # news.notice /var/log/news/news.notice !startslip *.* /var/log/slip.log !ppp *.* /var/log/ppp.log Consult the &man.syslog.conf.5; manual page for more information. <filename>newsyslog.conf</filename> newsyslog.conf newsyslog.conf is the configuration file for &man.newsyslog.8;, a program that is normally scheduled to run by &man.cron.8;. &man.newsyslog.8; determines when log files require archiving or rearranging. logfile is moved to logfile.0, logfile.0 is moved to logfile.1, and so on. Alternatively, the log files may be archived in &man.gzip.1; format causing them to be named: logfile.0.gz, logfile.1.gz, and so on. newsyslog.conf indicates which log files are to be managed, how many are to be kept, and when they are to be touched. Log files can be rearranged and/or archived when they have either reached a certain size, or at a certain periodic time/date. # configuration file for newsyslog # $FreeBSD$ # # filename [owner:group] mode count size when [ZB] [/pid_file] [sig_num] /var/log/cron 600 3 100 * Z /var/log/amd.log 644 7 100 * Z /var/log/kerberos.log 644 7 100 * Z /var/log/lpd-errs 644 7 100 * Z /var/log/maillog 644 7 * @T00 Z /var/log/sendmail.st 644 10 * 168 B /var/log/messages 644 5 100 * Z /var/log/all.log 600 7 * @T00 Z /var/log/slip.log 600 3 100 * Z /var/log/ppp.log 600 3 100 * Z /var/log/security 600 10 100 * Z /var/log/wtmp 644 3 * @01T05 B /var/log/daily.log 640 7 * @T00 Z /var/log/weekly.log 640 5 1 $W6D0 Z /var/log/monthly.log 640 12 * $M1D0 Z /var/log/console.log 640 5 100 * Z Consult the &man.newsyslog.8; manual page for more information. <filename>sysctl.conf</filename> sysctl.conf sysctl sysctl.conf looks much like rc.conf. Values are set in a variable=value form. The specified values are set after the system goes into multi-user mode. Not all variables are settable in this mode. A sample sysctl.conf turning off logging of fatal signal exits and letting Linux programs know they are really running under FreeBSD. kern.logsigexit=0 # Do not log fatal signal exits (e.g. sig 11) compat.linux.osname=FreeBSD compat.linux.osrelease=4.3-STABLE Tuning with sysctl sysctl Tuning with sysctl &man.sysctl.8; is an interface that allows you to make changes to a running FreeBSD system. This includes many advanced options of the TCP/IP stack and virtual memory system that can dramatically improve performance for an experienced system administrator. Over five hundred system variables can be read and set using &man.sysctl.8;. At its core, &man.sysctl.8; serves two functions: to read and to modify system settings. To view all readable variables: &prompt.user; sysctl -a To read a particular variable, for example, kern.maxproc: &prompt.user; sysctl kern.maxproc kern.maxproc: 1044 To set a particular variable, use the intuitive variable=value syntax: &prompt.root; sysctl kern.maxfiles=5000 kern.maxfiles: 2088 -> 5000 Settings of sysctl variables are usually either strings, numbers, or booleans (a boolean being 1 for yes or a 0 for no). Tuning Disks Sysctl Variables <varname>vfs.vmiodirenable</varname> vfs.vmiodirenable The vfs.vmiodirenable sysctl variable may be set to either 0 (off) or 1 (on); it is 1 by default. This variable controls how directories are cached by the system. Most directories are small, using just a single fragment (typically 1K) in the filesystem and less (typically 512 bytes) in the buffer cache. However, when operating in the default mode the buffer cache will only cache a fixed number of directories even if you have a huge amount of memory. Turning on this sysctl allows the buffer cache to use the VM Page Cache to cache the directories, making all the memory available for caching directories. However, the minimum in-core memory used to cache a directory is the physical page size (typically 4K) rather than 512 bytes. We recommend turning this option on if you are running any services which manipulate large numbers of files. Such services can include web caches, large mail systems, and news systems. Turning on this option will generally not reduce performance even with the wasted memory but you should experiment to find out. <varname>hw.ata.wc</varname> hw.ata.wc FreeBSD 4.3 flirted with turning off IDE write caching. This reduced write bandwidth to IDE disks but was considered necessary due to serious data consistency issues introduced by hard drive vendors. The problem is that IDE drives lie about when a write completes. With IDE write caching turned on, IDE hard drives not only write data to disk out of order, but will sometimes delay writing some blocks indefinitely when under heavy disk loads. A crash or power failure may cause serious filesystem corruption. FreeBSD's default was changed to be safe. Unfortunately, the result was such a huge performance loss that we changed write caching back to on by default after the release. You should check the default on your system by observing the hw.ata.wc sysctl variable. If IDE write caching is turned off, you can turn it back on by setting the kernel variable back to 1. This must be done from the boot loader at boot time. Attempting to do it after the kernel boots will have no effect. For more information, please see &man.ata.4;. - + Soft Updates Soft Updates tunefs The &man.tunefs.8; program can be used to fine-tune a filesystem. This program has many different options, but for now we are only concerned with toggling Soft Updates on and off, which is done by: &prompt.root; tunefs -n enable /filesystem &prompt.root; tunefs -n disable /filesystem A filesystem cannot be modified with &man.tunefs.8; while it is mounted. A good time to enable Soft Updates is before any partitions have been mounted, in single-user mode. As of FreeBSD 4.5, it is possible to enable Soft Updates at filesystem creation time, through use of the -U option to &man.newfs.8;. Soft Updates drastically improves meta-data performance, mainly file creation and deletion, through the use of a memory cache. We recommend turning Soft Updates on on all of your filesystems. There are two downsides to Soft Updates that you should be aware of: First, Soft Updates guarantees filesystem consistency in the case of a crash but could very easily be several seconds (even a minute!) behind updating the physical disk. If your system crashes you may lose more work than otherwise. Secondly, Soft Updates delays the freeing of filesystem blocks. If you have a filesystem (such as the root filesystem) which is almost full, performing a major update, such as make installworld, can cause the filesystem to run out of space and the update to fail. More details about Soft Updates Soft Updates (Details) There are two traditional approaches to writing a filesystem's meta-data back to disk. (Meta-data updates are updates to non-content data like inodes or directories.) Historically, the default behavior was to write out meta-data updates synchronously. If a directory had been changed, the system waited until the change was actually written to disk. The file data buffers (file contents) were passed through the buffer cache and backed up to disk later on asynchronously. The advantage of this implementation is that it operates safely. If there is a failure during an update, the meta-data are always in a consistent state. A file is either created completely or not at all. If the data blocks of a file did not find their way out of the buffer cache onto the disk by the time of the crash, &man.fsck.8; is able to recognize this and repair the filesystem by setting the file length to 0. Additionally, the implementation is clear and simple. The disadvantage is that meta-data changes are slow. An rm -r, for instance, touches all the files in a directory sequentially, but each directory change (deletion of a file) will be written synchronously to the disk. This includes updates to the directory itself, to the inode table, and possibly to indirect blocks allocated by the file. Similar considerations apply for unrolling large hierarchies (tar -x). The second case is asynchronous meta-data updates. This is the default for Linux/ext2fs and mount -o async for *BSD ufs. All meta-data updates are simply being passed through the buffer cache too, that is, they will be intermixed with the updates of the file content data. The advantage of this implementation is there is no need to wait until each meta-data update has been written to disk, so all operations which cause huge amounts of meta-data updates work much faster than in the synchronous case. Also, the implementation is still clear and simple, so there is a low risk for bugs creeping into the code. The disadvantage is that there is no guarantee at all for a consistent state of the filesystem. If there is a failure during an operation that updated large amounts of meta-data (like a power failure, or someone pressing the reset button), the filesystem will be left in an unpredictable state. There is no opportunity to examine the state of the filesystem when the system comes up again; the data blocks of a file could already have been written to the disk while the updates of the inode table or the associated directory were not. It is actually impossible to implement a fsck which is able to clean up the resulting chaos (because the necessary information is not available on the disk). If the filesystem has been damaged beyond repair, the only choice is to newfs it and restore it from backup. The usual solution for this problem was to implement dirty region logging, which is also referred to as journaling, although that term is not used consistently and is occasionally applied to other forms of transaction logging as well. Meta-data updates are still written synchronously, but only into a small region of the disk. Later on they will be moved to their proper location. Because the logging area is a small, contiguous region on the disk, there are no long distances for the disk heads to move, even during heavy operations, so these operations are quicker than synchronous updates. Additionally the complexity of the implementation is fairly limited, so the risk of bugs being present is low. A disadvantage is that all meta-data are written twice (once into the logging region and once to the proper location) so for normal work, a performance pessimization might result. On the other hand, in case of a crash, all pending meta-data operations can be quickly either rolled-back or completed from the logging area after the system comes up again, resulting in a fast filesystem startup. Kirk McKusick, the developer of Berkeley FFS, solved this problem with Soft Updates: all pending meta-data updates are kept in memory and written out to disk in a sorted sequence (ordered meta-data updates). This has the effect that, in case of heavy meta-data operations, later updates to an item catch the earlier ones if the earlier ones are still in memory and have not already been written to disk. So all operations on, say, a directory are generally performed in memory before the update is written to disk (the data blocks are sorted according to their position so that they will not be on the disk ahead of their meta-data). If the system crashes, this causes an implicit log rewind: all operations which did not find their way to the disk appear as if they had never happened. A consistent filesystem state is maintained that appears to be the one of 30 to 60 seconds earlier. The algorithm used guarantees that all resources in use are marked as such in their appropriate bitmaps: blocks and inodes. After a crash, the only resource allocation error that occurs is that resources are marked as used which are actually free. &man.fsck.8; recognizes this situation, and frees the resources that are no longer used. It is safe to ignore the dirty state of the filesystem after a crash by forcibly mounting it with mount -f. In order to free resources that may be unused, &man.fsck.8; needs to be run at a later time. This is the idea behind the background fsck: at system startup time, only a snapshot of the filesystem is recorded. The fsck can be run later on. All filesystems can then be mounted dirty, so the system startup proceeds in multiuser mode. Then, background fscks will be scheduled for all filesystems where this is required, to free resources that may be unused. (Filesystems that do not use Soft Updates still need the usual foreground fsck though.) The advantage is that meta-data operations are nearly as fast as asynchronous updates (i.e. faster than with logging, which has to write the meta-data twice). The disadvantages are the complexity of the code (implying a higher risk for bugs in an area that is highly sensitive regarding loss of user data), and a higher memory consumption. Additionally there are some idiosyncrasies one has to get used to. After a crash, the state of the filesystem appears to be somewhat older. In situations where the standard synchronous approach would have caused some zero-length files to remain after the fsck, these files do not exist at all with a Soft Updates filesystem because neither the meta-data nor the file contents have ever been written to disk. Disk space is not released until the updates have been written to disk, which may take place some time after running rm. This may cause problems when installing large amounts of data on a filesystem that does not have enough free space to hold all the files twice. Tuning Kernel Limits Tuning kernel limits File/Process Limits <varname>kern.maxfiles</varname> kern.maxfiles kern.maxfiles can be raised or lowered based upon your system requirements. This variable indicates the maximum number of file descriptors on your system. When the file descriptor table is full, file: table is full will show up repeatedly in the system message buffer, which can be viewed with the dmesg command. Each open file, socket, or fifo uses one file descriptor. A large-scale production server may easily require many thousands of file descriptors, depending on the kind and number of services running concurrently. kern.maxfile's default value is dictated by the option in your kernel configuration file. kern.maxfiles grows proportionally to the value of . When compiling a custom kernel, it is a good idea to set this kernel configuration option according to the uses of your system. From this number, the kernel is given most of its pre-defined limits. Even though a production machine may not actually have 256 users connected as once, the resources needed may be similar to a high-scale web server. As of FreeBSD 4.5, setting to 0 in your kernel configuration file will choose a reasonable default value based on the amount of RAM present in your system. Network Limits The kernel configuration option dictates the amount of network mbufs available to the system. A heavily-trafficked server with a low number of MBUFs will hinder FreeBSD's ability. Each cluster represents approximately 2K of memory, so a value of 1024 represents 2 megabytes of kernel memory reserved for network buffers. A simple calculation can be done to figure out how many are needed. If you have a web server which maxes out at 1000 simultaneous connections, and each connection eats a 16K receive and 16K send buffer, you need approximately 32MB worth of network buffers to cover the web server. A good rule of thumb is to multiply by 2, so 2x32 MB / 2 KB = 64 MB / 2 kB = 32768. Adding Swap Space No matter how well you plan, sometimes a system doesn't run as you expect. If you find you need more swap space, it's simple enough to add. You have three ways to increase swap space: adding a new hard drive, enabling swap over NFS, and creating a swap file on an existing partition. Swap on a New Hard Drive The best way to add swap, of course, is to use this as an excuse to add another hard drive. You can always use another hard drive, after all. If you can do this, go reread the discussion of swap space from the Initial Configuration section of the Handbook for some suggestions on how to best arrange your swap. Swapping over NFS Swapping over NFS is only recommended if you do not have a local hard disk to swap to. Swapping over NFS is slow and inefficient in versions of FreeBSD prior to 4.X. It is reasonably fast and efficient in 4.0-RELEASE and newer. Even with newer versions of FreeBSD, NFS swapping will be limited by the available network bandwidth and puts an additional burden on the NFS server. Swapfiles You can create a file of a specified size to use as a swap file. In our example here we will use a 64Mb file called /usr/swap0. You can use any name you want, of course. Creating a Swapfile Be certain that your kernel configuration includes the vnode driver. It is not in recent versions of GENERIC. pseudo-device vn 1 #Vnode driver (turns a file into a device) create a vn-device: &prompt.root; cd /dev &prompt.root; sh MAKEDEV vn0 create a swapfile (/usr/swap0): &prompt.root; dd if=/dev/zero of=/usr/swap0 bs=1024k count=64 set proper permissions on (/usr/swap0): &prompt.root; chmod 0600 /usr/swap0 enable the swap file in /etc/rc.conf: swapfile="/usr/swap0" # Set to name of swapfile if aux swapfile desired. Reboot the machine or to enable the swap file immediately, type: &prompt.root; vnconfig -e /dev/vn0b /usr/swap0 swap diff --git a/en_US.ISO8859-1/books/handbook/disks/chapter.sgml b/en_US.ISO8859-1/books/handbook/disks/chapter.sgml index 71f7ec9884..856d6cf56c 100644 --- a/en_US.ISO8859-1/books/handbook/disks/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/disks/chapter.sgml @@ -1,2413 +1,2419 @@ Storage Synopsis This chapter covers the use of disks in FreeBSD. This includes memory-backed disks, network-attached disks, and standard SCSI/IDE storage devices. After reading this chapter, you will know: The terminology FreeBSD uses to describe the organization of data on a physical disk (partitions and slices). How to mount and unmount filesystems. How to add additional hard disks to your system. How to setup virtual filesystems, such as memory disks. How to use quotas to limit disk space usage. How to create and burn CDs and DVDs on FreeBSD. The various storage media options for backups. How to use backup programs available under FreeBSD. How to backup to floppy disks. Device Names The following is a list of physical storage devices supported in FreeBSD, and the device names associated with them. Physical Disk Naming Conventions Drive type Drive device name IDE hard drives ad IDE CDROM drives acd SCSI hard drives and USB Mass storage devices da SCSI CDROM drives cd Assorted non-standard CDROM drives mcd for Mitsumi CD-ROM, scd for Sony CD-ROM, matcd for Matsushita/Panasonic CD-ROM Floppy drives fd SCSI tape drives sa IDE tape drives ast Flash drives fla for DiskOnChip Flash device RAID drives myxd for Mylex, and amrd for AMI MegaRAID, idad for Compaq Smart RAID.
David O'Brien Originally contributed by Adding Disks disks adding Lets say we want to add a new SCSI disk to a machine that currently only has a single drive. First turn off the computer and install the drive in the computer following the instructions of the computer, controller, and drive manufacturer. Due to the wide variations of procedures to do this, the details are beyond the scope of this document. Login as user root. After you have installed the drive, inspect /var/run/dmesg.boot to ensure the new disk was found. Continuing with our example, the newly added drive will be da1 and we want to mount it on /1 (if you are adding an IDE drive, the device name will be wd1 in pre-4.0 systems, or ad1 in most 4.X systems). partitions slices fdisk Because FreeBSD runs on IBM-PC compatible computers, it must take into account the PC BIOS partitions. These are different from the traditional BSD partitions. A PC disk has up to four BIOS partition entries. If the disk is going to be truly dedicated to FreeBSD, you can use the dedicated mode. Otherwise, FreeBSD will have to live within one of the PC BIOS partitions. FreeBSD calls the PC BIOS partitions slices so as not to confuse them with traditional BSD partitions. You may also use slices on a disk that is dedicated to FreeBSD, but used in a computer that also has another operating system installed. This is to not confuse the fdisk utility of the other operating system. In the slice case the drive will be added as /dev/da1s1e. This is read as: SCSI disk, unit number 1 (second SCSI disk), slice 1 (PC BIOS partition 1), and e BSD partition. In the dedicated case, the drive will be added simply as /dev/da1e. Using &man.sysinstall.8; sysinstall adding disks su Navigating <application>Sysinstall</application> You may use /stand/sysinstall to partition and label a new disk using its easy to use menus. Either login as user root or use the su command. Run /stand/sysinstall and enter the Configure menu. Within the FreeBSD Configuration Menu, scroll down and select the Fdisk option. <application>fdisk</application> Partition Editor Once inside fdisk, we can type A to use the entire disk for FreeBSD. When asked if you want to remain cooperative with any future possible operating systems, answer YES. Write the changes to the disk using W. Now exit the FDISK editor by typing q. Next you will be asked about the Master Boot Record. Since you are adding a disk to an already running system, choose None. Disk Label Editor BSD partitions Next, you need to exit sysinstall and start it again. Follow the directions above, although this time choose the Label option. This will enter the Disk Label Editor. This is where you will create the traditional BSD partitions. A disk can have up to eight partitions, labeled a-h. A few of the partition labels have special uses. The a partition is used for the root partition (/). Thus only your system disk (e.g, the disk you boot from) should have an a partition. The b partition is used for swap partitions, and you may have many disks with swap partitions. The c partition addresses the entire disk in dedicated mode, or the entire FreeBSD slice in slice mode. The other partitions are for general use. sysinstall's Label editor favors the e partition for non-root, non-swap partitions. Within the Label editor, create a single filesystem by typing C. When prompted if this will be a FS (filesystem) or swap, choose FS and type in a mount point (e.g, /mnt). When adding a disk in post-install mode, sysinstall will not create entries in /etc/fstab for you, so the mount point you specify is not important. You are now ready to write the new label to the disk and create a filesystem on it. Do this by typing W. Ignore any errors from sysinstall that it could not mount the new partition. Exit the Label Editor and sysinstall completely. Finish The last step is to edit /etc/fstab to add an entry for your new disk. Using Command Line Utilities Using Slices This setup will allow your disk to work correctly with other operating systems that might be installed on your computer and will not confuse other operating systems' fdisk utilities. It is recommended to use this method for new disk installs. Only use dedicated mode if you have a good reason to do so! &prompt.root; dd if=/dev/zero of=/dev/rda1 bs=1k count=1 &prompt.root; fdisk -BI da1 #Initialize your new disk &prompt.root; disklabel -B -w -r da1s1 auto #Label it. &prompt.root; disklabel -e da1s1 # Edit the disklabel just created and add any partitions. &prompt.root; mkdir -p /1 &prompt.root; newfs /dev/da1s1e # Repeat this for every partition you created. &prompt.root; mount -t ufs /dev/da1s1e /1 # Mount the partition(s) &prompt.root; vi /etc/fstab # Add the appropriate entry/entries to your /etc/fstab. If you have an IDE disk, substitute ad for da. On pre-4.X systems use wd. Dedicated OS/2 If you will not be sharing the new drive with another operating system, you may use the dedicated mode. Remember this mode can confuse Microsoft operating systems; however, no damage will be done by them. IBM's OS/2 however, will appropriate any partition it finds which it does not understand. &prompt.root; dd if=/dev/zero of=/dev/rda1 bs=1k count=1 &prompt.root; disklabel -Brw da1 auto &prompt.root; disklabel -e da1 # create the `e' partition &prompt.root; newfs -d0 /dev/rda1e &prompt.root; mkdir -p /1 &prompt.root; vi /etc/fstab # add an entry for /dev/da1e &prompt.root; mount /1 An alternate method is: &prompt.root; dd if=/dev/zero of=/dev/rda1 count=2 &prompt.root; disklabel /dev/rda1 | disklabel -BrR da1 /dev/stdin &prompt.root; newfs /dev/rda1e &prompt.root; mkdir -p /1 &prompt.root; vi /etc/fstab # add an entry for /dev/da1e &prompt.root; mount /1 Network, Memory, and File-Based Filesystems virtual disks disks virtual Aside from the disks you physically insert into your computer: floppies, CDs, hard drives, and so forth; other forms of disks are understood by FreeBSD - the virtual disks. NFS Coda disks memory These include network filesystems such as the Network Filesystem and Coda, memory-based filesystems such as md and file-backed filesystems created by vnconfig or mdconfig. vnconfig: File-Backed Filesystem disks file-backed &man.vnconfig.8; configures and enables vnode pseudo-disk devices. A vnode is a representation of a file, and is the focus of file activity. This means that &man.vnconfig.8; uses files to create and operate a filesystem. One possible use is the mounting of floppy or CD images kept in files. To mount an existing filesystem image: Using vnconfig to mount an Existing Filesystem Image &prompt.root; vnconfig vn0 diskimage &prompt.root; mount /dev/vn0c /mnt To create a new filesystem image with vnconfig: Creating a New File-Backed Disk with vnconfig &prompt.root; dd if=/dev/zero of=newimage bs=1k count=5k 5120+0 records in 5120+0 records out &prompt.root; vnconfig -s labels -c vn0 newimage &prompt.root; disklabel -r -w vn0 auto &prompt.root; newfs vn0c Warning: 2048 sector(s) in last cylinder unallocated /dev/rvn0c: 10240 sectors in 3 cylinders of 1 tracks, 4096 sectors 5.0MB in 1 cyl groups (16 c/g, 32.00MB/g, 1280 i/g) super-block backups (for fsck -b #) at: 32 &prompt.root; mount /dev/vn0c /mnt &prompt.root; df /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/vn0c 4927 1 4532 0% /mnt md: Memory Filesystem disks memory filesystem md is a simple, efficient means to create memory filesystems. Simply take a filesystem you have prepared with, for example, &man.vnconfig.8;, and: md Memory Disk &prompt.root; dd if=newimage of=/dev/md0 5120+0 records in 5120+0 records out &prompt.root; mount /dev/md0c /mnt &prompt.root; df /mnt Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/md0c 4927 1 4532 0% /mnt Tom Rhodes Contributed by File System Snapshots File System Snapshots Snapshots - FreeBSD 5.0 offers a new feature in conjunction with the - option. File system snapshots. + FreeBSD 5.0 offers a new feature in conjunction with + Soft Updates: File system snapshots. Snapshots allow a user to create an image of specified file - systems and treat this image as a file. The -u (or update) flag - in &man.mount.8; is required with this option. + systems and treat this image as a file. Snapshot files must be created in the file system that the action is performed on, and a user may create no more than 20 snapshots per file system. Active snapshots are recorded in the superblock so they are persistent across unmount and remount operations along with system reboots. When a snapshot is no longer required, it can be removed with the standard &man.rm.1; command, like regular files. Snapshots may be removed in any order, however all the used space may not be acquired as another snapshot will possibly claim some of the blocks that were released. - During initial creation, the schg flag (see &man.chflags.1; manual page) - is set on to ensure that not even root can write to the snapshot. + During initial creation, the flag (see &man.chflags.1; manual page) + is set on to ensure that not even root can write to the snapshot. The &man.unlink.1; command makes an exception for snapshot files, however, in which it allows them to be removed even - though they have the schg flag set, so it is not necessary to - clear the schg flag before removing a snapshot file. + though they have the flag set, so it is not necessary to + clear the flag before removing a snapshot file. + + Snapshots are created with the &man.mount.8; command. To place + a snapshot of /var in the file + /var/snapshot/snap use the following + command: + +&prompt.root; mount -u -o snapshot /var/snapshot/snap /var Once a snapshot has been created, there are several interesting things that an administrator can do with them: Some administrators will use a snapshot file for backup purposes, where the snapshot can be transfered to a CD or tape. File integrity, &man.fsck.8; may be ran on the snapshot file. Assuming that the file system was clean when it was mounted, you should always get a clean (and unchanging) result from running &man.fsck.8; on the snapshot. This is essentially what the background &man.fsck.8; process does. Run the &man.dump.8; utility on the snapshot. A dump will be returned that is as consistent with the file system as the timestamp of the snapshot. As of this writing &man.dump.8; has not yet - been changed to set the dumpdates file correctly, so + been changed to set the dumpdates file correctly, so do not use this feature in production until that fix is made. &man.mount.8; the snapshot as a frozen image of the file system. To &man.mount.8; the snapshot /var/snapshot/snap: &prompt.root; mdconfig -a -t vnode -f /var/snapshot/snap -u 4 &prompt.root; mount -r /dev/md4 /mnt You can now walk the hierarchy of your frozen /var file system mounted at /mnt. Everything will be in the same state it was during the snapshot creation time. The only exception being that any earlier snapshots will appear as zero length files. When the use of a snapshot has delimited, it can be unmounted with: &prompt.root; umount /mnt &prompt.root; mdconfig -d -u 4 For more information about and file system snapshots, including technical papers, you can visit Marshall Kirk McKusick's website at http://www.mckusick.com File System Quotas accounting disk space disk quotas Quotas are an optional feature of the operating system that allow you to limit the amount of disk space and/or the number of files a user or members of a group may allocate on a per-file system basis. This is used most often on timesharing systems where it is desirable to limit the amount of resources any one user or group of users may allocate. This will prevent one user or group of users from consuming all of the available disk space. Configuring Your System to Enable Disk Quotas Before attempting to use disk quotas, it is necessary to make sure that quotas are configured in your kernel. This is done by adding the following line to your kernel configuration file: options QUOTA The stock GENERIC kernel does not have this enabled by default, so you will have to configure, build and install a custom kernel in order to use disk quotas. Please refer to for more information on kernel configuration. Next you will need to enable disk quotas in /etc/rc.conf. This is done by adding the line: enable_quotas=YES disk quotas checking For finer control over your quota startup, there is an additional configuration variable available. Normally on bootup, the quota integrity of each filesystem is checked by the quotacheck program. The quotacheck facility insures that the data in the quota database properly reflects the data on the filesystem. This is a very time consuming process that will significantly affect the time your system takes to boot. If you would like to skip this step, a variable in /etc/rc.conf is made available for the purpose: check_quotas=NO If you are running FreeBSD prior to 3.2-RELEASE, the configuration is simpler, and consists of only one variable. Set the following in your /etc/rc.conf: check_quotas=YES Finally you will need to edit /etc/fstab to enable disk quotas on a per-filesystem basis. This is where you can either enable user or group quotas or both for all of your filesystems. To enable per-user quotas on a filesystem, add the userquota option to the options field in the /etc/fstab entry for the filesystem you want to enable quotas on. For example: /dev/da1s2g /home ufs rw,userquota 1 2 Similarly, to enable group quotas, use the groupquota option instead of userquota. To enable both user and group quotas, change the entry as follows: /dev/da1s2g /home ufs rw,userquota,groupquota 1 2 By default, the quota files are stored in the root directory of the filesystem with the names quota.user and quota.group for user and group quotas respectively. See &man.fstab.5; for more information. Even though the &man.fstab.5; manual page says that you can specify an alternate location for the quota files, this is not recommended because the various quota utilities do not seem to handle this properly. At this point you should reboot your system with your new kernel. /etc/rc will automatically run the appropriate commands to create the initial quota files for all of the quotas you enabled in /etc/fstab, so there is no need to manually create any zero length quota files. In the normal course of operations you should not be required to run the quotacheck, quotaon, or quotaoff commands manually. However, you may want to read their manual pages just to be familiar with their operation. Setting Quota Limits disk quotas limits Once you have configured your system to enable quotas, verify that they really are enabled. An easy way to do this is to run: &prompt.root; quota -v You should see a one line summary of disk usage and current quota limits for each filesystem that quotas are enabled on. You are now ready to start assigning quota limits with the edquota command. You have several options on how to enforce limits on the amount of disk space a user or group may allocate, and how many files they may create. You may limit allocations based on disk space (block quotas) or number of files (inode quotas) or a combination of both. Each of these limits are further broken down into two categories: hard and soft limits. hard limit A hard limit may not be exceeded. Once a user reaches his hard limit he may not make any further allocations on the file system in question. For example, if the user has a hard limit of 500 blocks on a filesystem and is currently using 490 blocks, the user can only allocate an additional 10 blocks. Attempting to allocate an additional 11 blocks will fail. soft limit Soft limits, on the other hand, can be exceeded for a limited amount of time. This period of time is known as the grace period, which is one week by default. If a user stays over his or her soft limit longer than the grace period, the soft limit will turn into a hard limit and no further allocations will be allowed. When the user drops back below the soft limit, the grace period will be reset. The following is an example of what you might see when you run the edquota command. When the edquota command is invoked, you are placed into the editor specified by the EDITOR environment variable, or in the vi editor if the EDITOR variable is not set, to allow you to edit the quota limits. &prompt.root; edquota -u test Quotas for user test: /usr: blocks in use: 65, limits (soft = 50, hard = 75) inodes in use: 7, limits (soft = 50, hard = 60) /usr/var: blocks in use: 0, limits (soft = 50, hard = 75) inodes in use: 0, limits (soft = 50, hard = 60) You will normally see two lines for each filesystem that has quotas enabled. One line for the block limits, and one line for inode limits. Simply change the value you want updated to modify the quota limit. For example, to raise this user's block limit from a soft limit of 50 and a hard limit of 75 to a soft limit of 500 and a hard limit of 600, change: /usr: blocks in use: 65, limits (soft = 50, hard = 75) to: /usr: blocks in use: 65, limits (soft = 500, hard = 600) The new quota limits will be in place when you exit the editor. Sometimes it is desirable to set quota limits on a range of uids. This can be done by use of the option on the edquota command. First, assign the desired quota limit to a user, and then run edquota -p protouser startuid-enduid. For example, if user test has the desired quota limits, the following command can be used to duplicate those quota limits for uids 10,000 through 19,999: &prompt.root; edquota -p test 10000-19999 For more information see &man.edquota.8;. Checking Quota Limits and Disk Usage disk quotas checking You can use either the quota or the repquota commands to check quota limits and disk usage. The quota command can be used to check individual user or group quotas and disk usage. A user may only examine his own quota, and the quota of a group he is a member of. Only the super-user may view all user and group quotas. The repquota command can be used to get a summary of all quotas and disk usage for filesystems with quotas enabled. The following is some sample output from the quota -v command for a user that has quota limits on two filesystems. Disk quotas for user test (uid 1002): Filesystem blocks quota limit grace files quota limit grace /usr 65* 50 75 5days 7 50 60 /usr/var 0 50 75 0 50 60 grace period On the /usr filesystem in the above example, this user is currently 15 blocks over the soft limit of 50 blocks and has 5 days of the grace period left. Note the asterisk * which indicates that the user is currently over his quota limit. Normally filesystems that the user is not using any disk space on will not show up in the output from the quota command, even if he has a quota limit assigned for that filesystem. The option will display those filesystems, such as the /usr/var filesystem in the above example. Quotas over NFS NFS Quotas are enforced by the quota subsystem on the NFS server. The &man.rpc.rquotad.8; daemon makes quota information available to the &man.quota.1; command on NFS clients, allowing users on those machines to see their quota statistics. Enable rpc.rquotad in /etc/inetd.conf like so: rquotad/1 dgram rpc/udp wait root /usr/libexec/rpc.rquotad rpc.rquotad Now restart inetd: &prompt.root; kill -HUP `cat /var/run/inetd.pid` Mike Meyer Contributed by Creating and Using Optical Media (CDs & DVDs) CDROMs creating Introduction CDs have a number of features that differentiate them from conventional disks. Initially, they were not writable by the user. They are designed so that they can be read continuously without delays to move the head between tracks. They are also much easier to transport between systems than similarly sized media were at the time. CDs do have tracks, but this refers to a section of data to be read continuously and not a physical property of the disk. To produce a CD on FreeBSD, you prepare the data files that are going to make up the tracks on the CD, then write the tracks to the CD. ISO 9660 filesystems ISO-9660 The ISO 9660 filesystem was designed to deal with these differences. It unfortunately codifies filesystem limits that were common then. Fortunately, it provides an extension mechanism that allows properly written CDs to exceed those limits while still working with systems that do not support those extensions. sysutils/mkisofs The sysutils/mkisofs program is used to produce a data file containing an ISO 9660 file system. It has options that support various extensions, and is described below. You can install it with the sysutils/mkisofs ports. CD burner ATAPI Which tool to use to burn the CD depends on whether your CD burner is ATAPI or something else. ATAPI CD burners use the burncd program that is part of the base system. SCSI and USB CD burners should use cdrecord from the sysutils/cdrtools port. burncd has a limited number of supported drives. To find out if a drive is supported, see CD-R/RW supported drives. mkisofs sysutils/mkisofs produces an ISO 9660 filesystem that is an image of a directory tree in the Unix filesystem name space. The simplest usage is: &prompt.root; mkisofs imagefile.iso /path/to/tree filesystems ISO-9660 This command will create an imagefile containing an ISO 9660 filesystem that is a copy of the tree at /path/to/tree. In the process, it will map the file names to names that fit the limitations of the standard ISO 9660 filesystem, and will exclude files that have names uncharacteristic of ISO filesystems. filesystems HFS filesystems Joliet A number of options are available to overcome those restrictions. In particular, enables the Rock Ridge extensions common to Unix systems, enables Joliet extensions used by Microsoft systems, and can be used to create HFS filesystems used by MacOS. For CDs that are going to be used only on FreeBSD systems, can be used to disable all filename restrictions. When used with , it produces a filesystem image that is identical to the FreeBSD tree you started from, though it may violate the ISO 9660 standard in a number of ways. CDROMs creating bootable The last option of general use is . This is used to specify the location of the boot image for use in producing an El Torito bootable CD. This option takes an argument which is the path to a boot image from the top of the tree being written to the CD. So, given that /tmp/myboot holds a bootable FreeBSD system with the boot image in /tmp/myboot/boot/cdboot, you could produce the image of an ISO 9660 filesystem in /tmp/bootable.iso like so: &prompt.root; mkisofs -U -R -b boot/cdboot -o /tmp/bootable.iso /tmp/myboot Having done that, if you have vn configured in your kernel, you can mount the filesystem with: &prompt.root; vnconfig -e vn0c /tmp/bootable.iso &prompt.root; mount -t cd9660 /dev/vn0c /mnt At which point you can verify that /mnt and /tmp/myboot are identical. There are many other options you can use with sysutils/mkisofs to fine-tune its behavior. In particular: modifications to an ISO 9660 layout and the creation of Joilet and HFS discs. See the &man.mkisofs.8; manual page for details. burncd CDROMs burning If you have an ATAPI CD burner, you can use the burncd command to burn an ISO image onto a CD. burncd is part of the base system, installed as /usr/sbin/burncd. Usage is very simple, as it has few options: &prompt.root; burncd cddevice data imagefile.iso fixate Will burn a copy of imagefile.iso on cddevice. The default device is /dev/acd0c. See &man.burncd.8; for options to set the write speed, eject the CD after burning, and write audio data. cdrecord If you do not have an ATAPI CD burner, you will have to use cdrecord to burn your CDs. cdrecord is not part of the base system; you must install it from either the port at sysutils/cdrtools or the appropriate package. Changes to the base system can cause binary versions of this program to fail, possibly resulting in a coaster. You should therefore either upgrade the port when you upgrade your system, or if you are tracking -STABLE, upgrade the port when a new version becomes available. While cdrecord has many options, basic usage is even simpler than burncd. Burning an ISO 9660 image is done with: &prompt.root; cdrecord device imagefile.iso The tricky part of using cdrecord is finding the to use. To find the proper setting, use the flag of cdrecord, which might produce results like this: CDROMs burning &prompt.root; cdrecord Cdrecord 1.9 (i386-unknown-freebsd4.2) Copyright (C) 1995-2000 Jörg Schilling Using libscg version 'schily-0.1' scsibus0: 0,0,0 0) 'SEAGATE ' 'ST39236LW ' '0004' Disk 0,1,0 1) 'SEAGATE ' 'ST39173W ' '5958' Disk 0,2,0 2) * 0,3,0 3) 'iomega ' 'jaz 1GB ' 'J.86' Removable Disk 0,4,0 4) 'NEC ' 'CD-ROM DRIVE:466' '1.26' Removable CD-ROM 0,5,0 5) * 0,6,0 6) * 0,7,0 7) * scsibus1: 1,0,0 100) * 1,1,0 101) * 1,2,0 102) * 1,3,0 103) * 1,4,0 104) * 1,5,0 105) 'YAMAHA ' 'CRW4260 ' '1.0q' Removable CD-ROM 1,6,0 106) 'ARTEC ' 'AM12S ' '1.06' Scanner 1,7,0 107) * This lists the appropriate value for the devices on the list. Locate your CD burner, and use the three numbers separated by commas as the value for . In this case, the CRW device is 1,5,0, so the appropriate input would be =1,5,0. There are easier ways to specify this value; see &man.cdrecord.1; for details. That is also the place to look for information on writing audio tracks, controlling the speed, and other things. Duplicating Audio CDs You can duplicate an audio CD by extracting the audio data from the CD to a series of files, and then writing these files to a blank CD. The process is slightly different for ATAPI and SCSI drives. SCSI drives Use cdda2wav to extract the audio. &prompt.user; cdda2wav -v255 -D2,0 -B -Owav Use cdrecord to write the .wav files. &prompt.user; cdrecord -v dev=2,0 -dao -useinfo *.wav Make sure that 2.0 is set appropriately, as described in . ATAPI drives The ATAPI CD driver makes each track available as /dev/acddtn, where d is the drive number, and n is the track number. So the first track on the first disk is /dev/acd0t1. Make sure the appropriate files exist in /dev. &prompt.root; cd /dev &prompt.root; sh MAKEDEV acd0t99 Extract each track using &man.dd.1;. You must also use a specific blocksize when extracting the files. &prompt.root; dd if=/dev/acd0t1 of=track1.cdr bs=2352 &prompt.root; dd if=/dev/acd0t2 of=track2.cdr bs=2352 ... Burn the extracted files to disk using burncd. You must specify that these are audio files, and that burncd should fixate the disk when finished. &prompt.root; burncd -f /dev/acd0c audio track1.cdr track2.cdr ... fixate Duplicating Data CDs You can copy a data CD to a image file that is functionally equivalent to the image file created with sysutils/mkisofs, and you can use it to duplicate any data CD. The example given here assumes that your CDROM device is acd0. Substitute your correct CDROM device. A c must be appended to the end of the device name to indicate the entire partition or, in the case of CDROMs, the entire disc. &prompt.root; dd if=/dev/acd0c of=file.iso bs=2048 Now that you have an image, you can burn it to CD as described above. Using Data CDs Now that you have created a standard data CDROM, you probably want to mount it and read the data on it. By default, &man.mount.8; assumes that a filesystem is of type ufs. If you try something like: &prompt.root; mount /dev/cd0c /mnt you will get a complaint about Incorrect super block, and no mount. The CDROM is not a UFS filesystem, so attempts to mount it as such will fail. You just need to tell &man.mount.8; that the filesystem is of type ISO9660, and everything will work. You do this by specifying the option &man.mount.8;. For example, if you want to mount the CDROM device, /dev/cd0c, under /mnt, you would execute: &prompt.root; mount -t cd9660 /dev/cd0c /mnt Note that your device name (/dev/cd0c in this example) could be different, depending on the interface your CDROM uses. Also, the option just executes &man.mount.cd9660.8;. The above example could be shortened to: &prompt.root; mount_cd9660 /dev/cd0c /mnt You can generally use data CDROMs from any vendor in this way. Disks with certain ISO 9660 extensions might behave oddly, however. For example, Joliet disks store all filenames in two-byte Unicode characters. The FreeBSD kernel does not speak Unicode (yet!), so non-English characters show up as question marks. (If you are running FreeBSD 4.3 or later, the CD9660 driver includes hooks to load an appropriate Unicode conversion table on the fly. Modules for some of the common encodings are available via the sysutils/cd9660_unicode port.) Occasionally, you might get Device not configured when trying to mount a CDROM. This usually means that the CDROM drive thinks that there is no disk in the tray, or that the drive is not visible on the bus. It can take a couple of seconds for a CDROM drive to realize that it has been fed, so be patient. Sometimes, a SCSI CDROM may be missed because it didn't have enough time to answer the bus reset. If you have a SCSI CDROM please add the following option to your kernel configuration and rebuild your kernel. options SCSI_DELAY=15000 This tells your SCSI bus to pause 15 seconds during boot, to give your CDROM drive every possible chance to answer the bus reset. Burning Raw Data CDs You can choose to burn a file directly to CD, without creating an ISO 9660 filesystem. Some people do this for backup purposes. This runs more quickly than burning a standard CD: &prompt.root; burncd -f /dev/acd1c -s 12 data archive.tar.gz fixate In order to retrieve the data burned to such a CD, you must read data from the raw device node: &prompt.root; tar xzvf /dev/acd1c You cannot mount this disk as you would a normal CDROM. Such a CDROM cannot be read under any operating system except FreeBSD. If you want to be able to mount the CD, or share data with another operating system, you must use sysutils/mkisofs as described above. RAID Software RAID Christopher Shumway Written by Valentino Vaschetto Marked up by ccd (Concatenated Disk Configuration) It seems like today everyone has a collection of multimedia files. Everything from mp3's to video clips. I have converted most of my audio CDROM collection to mp3's so I can have all of my music in one centralized location, and not have to hunt down the audio CD with that one song I got stuck in my head. The problem I was faced with is where to store all these files? When choosing a mass storage solution, the most important factors to consider are speed, reliability, and cost. It is very rare to have all three in favor, normally a fast, reliable mass storage device is expensive, and to cut back on cost either speed or reliability must be sacrificed. In designing my system, I ranked the requirements by most favorable to least favorable. In this situation, cost was the biggest factor. I needed a lot of storage for a reasonable price. The next factor, speed, is not quite as important, since most of the usage would be over a one hundred megabit switched Ethernet, and that would most likely be the bottleneck. The ability to spread the file input/output operations out over several disks would be more than enough speed for this network. Finally, the consideration of reliability was an easy one to answer. All of the data being put on this mass storage device was already backed up on CD-R's. This drive was primarily here for online live storage for easy access, so if a drive went bad, I could just replace it, rebuild the filesystem, and copy back the data from CD-R's. To sum it up, I need something that will give me the most amount of storage space for my money. The cost of large IDE disks are cheap these days. I found a place that was selling Western Digital 30.7gb 5400 RPM IDE disks for about one-hundred and thirty US dollars. I bought three of them, giving me approximately ninety gigabytes of online storage. Installing the Hardware I installed the hard drives in a system that already had one IDE disk in as the system disk. The ideal solution would be for each IDE disk to have its own IDE controller and cable, but without fronting more costs to acquire a dual IDE controller this would not be a possibility. So, I jumpered two disks as slaves, and one as master. One went on the first IDE controller as a slave to the system disk, and the other two where slave/master on the secondary IDE controller. Upon reboot, the system BIOS was configured to automatically detect the disks attached. More importantly, FreeBSD detected them on reboot: ad0: 19574MB <WDC WD205BA> [39770/16/63] at ata0-master UDMA33 ad1: 29333MB <WDC WD307AA> [59598/16/63] at ata0-slave UDMA33 ad2: 29333MB <WDC WD307AA> [59598/16/63] at ata1-master UDMA33 ad3: 29333MB <WDC WD307AA> [59598/16/63] at ata1-slave UDMA33 At this point, if FreeBSD does not detect the disks, be sure that you have jumpered them correctly. I have heard numerous reports with problems using cable select instead of true slave/master configuration. The next consideration was how to attach them as part of the filesystem. I did a little research on &man.vinum.8; and FreeBSD's &man.ccd.4;. In this particular configuration, &man.ccd.4; appeared to be a better choice mainly because it has fewer parts. Less parts tends to indicate less chance of breakage. Vinum appears to be a bit of an overkill for my needs. Setting up the CCD CCD allows me to take several identical disks and concatenate them into one logical filesystem. In order to use ccd, I need a kernel with ccd support built into it. I added this line to my kernel configuration file and rebuilt the kernel: pseudo-device ccd 4 ccd support can also be loaded as a kernel loadable module in FreeBSD 4.0 or later. To set up ccd, first I need to disklabel the disks. Here is how I disklabeled them: disklabel -r -w ad1 auto disklabel -r -w ad2 auto disklabel -r -w ad3 auto This created a disklabel ad1c, ad2c and ad3c that spans the entire disk. The next step is to change the disklabel type. To do that I had to edit the disklabel: disklabel -e ad1 disklabel -e ad2 disklabel -e ad3 This opened up the current disklabel on each disk respectively in whatever editor the EDITOR environment variable was set to, in my case, &man.vi.1;. Inside the editor I had a section like this: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 60074784 0 unused 0 0 0 # (Cyl. 0 - 59597) I needed to add a new "e" partition for &man.ccd.4; to use. This usually can be copied of the "c" partition, but the must be 4.2BSD. Once I was done, my disklabel should look like this: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 60074784 0 unused 0 0 0 # (Cyl. 0 - 59597) e: 60074784 0 4.2BSD 0 0 0 # (Cyl. 0 - 59597) Building the Filesystem Now that I have all of the disks labeled, I needed to build the ccd. To do that, I used a utility called &man.ccdconfig.8;. ccdconfig takes several arguments, the first argument being the device to configure, in this case, /dev/ccd0c. The device node for ccd0c may not exist yet, so to create it, perform the following commands: cd /dev sh MAKEDEV ccd0 The next argument ccdconfig expects is the interleave for the filesystem. The interleave defines the size of a stripe in disk blocks, normally five hundred and twelve bytes. So, an interleave of thirty-two would be sixteen thousand three hundred and eighty-four bytes. After the interleave comes the flags for ccdconfig. If you want to enable drive mirroring, you can specify a flag here. In this configuration, I am not mirroring the ccd, so I left it as zero. The final arguments to ccdconfig are the devices to place into the array. Putting it all together I get this command: ccdconfig ccd0 32 0 /dev/ad1e /dev/ad2e /dev/ad3e This configures the ccd. I can now &man.newfs.8; the filesystem. newfs /dev/ccd0c Making it all Automatic Finally, if I want to be able to mount the ccd, I need to configure it first. I write out my current configuration to /etc/ccd.conf using the following command: ccdconfig -g > /etc/ccd.conf When I reboot, the script /etc/rc runs ccdconfig -C if /etc/ccd.conf exists. This automatically configures the ccd so it can be mounted. If you are booting into single user mode, before you can mount the ccd, you need to issue the following command to configure the array: ccdconfig -C Then, we need an entry for the ccd in /etc/fstab so it will be mounted at boot time. /dev/ccd0c /media ufs rw 2 2 vinum (Logical Volume Manager) XXX Hardware RAID RAID Hardware FreeBSD supports a wide variety of hardware RAID controllers from many popular manufacturers such as Adaptec, 3Ware, Mylex, DPT, AMI, Dell, HP, IBM, and more. The list of supported adapters is growing all the time, so make sure to check the release notes for complete information. Tape Backup Media tape media The major tape media are the 4mm, 8mm, QIC, mini-cartridge and DLT. 4mm (DDS: Digital Data Storage) tape media DDS (4mm) tapes tape media QIC tapes 4mm tapes are replacing QIC as the workstation backup media of choice. This trend accelerated greatly when Conner purchased Archive, a leading manufacturer of QIC drives, and then stopped production of QIC drives. 4mm drives are small and quiet but do not have the reputation for reliability that is enjoyed by 8mm drives. The cartridges are less expensive and smaller (3 x 2 x 0.5 inches, 76 x 51 x 12 mm) than 8mm cartridges. 4mm, like 8mm, has comparatively short head life for the same reason, both use helical scan. Data throughput on these drives starts ~150kB/s, peaking at ~500kB/s. Data capacity starts at 1.3 GB and ends at 2.0 GB. Hardware compression, available with most of these drives, approximately doubles the capacity. Multi-drive tape library units can have 6 drives in a single cabinet with automatic tape changing. Library capacities reach 240 GB. The DDS-3 standard now supports tape capacities up to 12 GB (or 24 GB compressed). 4mm drives, like 8mm drives, use helical-scan. All the benefits and drawbacks of helical-scan apply to both 4mm and 8mm drives. Tapes should be retired from use after 2,000 passes or 100 full backups. 8mm (Exabyte) tape media Exabyte (8mm) tapes 8mm tapes are the most common SCSI tape drives; they are the best choice of exchanging tapes. Nearly every site has an Exabyte 2 GB 8mm tape drive. 8mm drives are reliable, convenient and quiet. Cartridges are inexpensive and small (4.8 x 3.3 x 0.6 inches; 122 x 84 x 15 mm). One downside of 8mm tape is relatively short head and tape life due to the high rate of relative motion of the tape across the heads. Data throughput ranges from ~250kB/s to ~500kB/s. Data sizes start at 300 MB and go up to 7 GB. Hardware compression, available with most of these drives, approximately doubles the capacity. These drives are available as single units or multi-drive tape libraries with 6 drives and 120 tapes in a single cabinet. Tapes are changed automatically by the unit. Library capacities reach 840+ GB. The Exabyte Mammoth model supports 12 GB on one tape (24 GB with compression) and costs approximately twice as much as conventional tape drives. Data is recorded onto the tape using helical-scan, the heads are positioned at an angle to the media (approximately 6 degrees). The tape wraps around 270 degrees of the spool that holds the heads. The spool spins while the tape slides over the spool. The result is a high density of data and closely packed tracks that angle across the tape from one edge to the other. QIC tape media QIC-150 QIC-150 tapes and drives are, perhaps, the most common tape drive and media around. QIC tape drives are the least expensive "serious" backup drives. The downside is the cost of media. QIC tapes are expensive compared to 8mm or 4mm tapes, up to 5 times the price per GB data storage. But, if your needs can be satisfied with a half-dozen tapes, QIC may be the correct choice. QIC is the most common tape drive. Every site has a QIC drive of some density or another. Therein lies the rub, QIC has a large number of densities on physically similar (sometimes identical) tapes. QIC drives are not quiet. These drives audibly seek before they begin to record data and are clearly audible whenever reading, writing or seeking. QIC tapes measure (6 x 4 x 0.7 inches; 15.2 x 10.2 x 1.7 mm). Mini-cartridges, which also use 1/4" wide tape are discussed separately. Tape libraries and changers are not available. Data throughput ranges from ~150kB/s to ~500kB/s. Data capacity ranges from 40 MB to 15 GB. Hardware compression is available on many of the newer QIC drives. QIC drives are less frequently installed; they are being supplanted by DAT drives. Data is recorded onto the tape in tracks. The tracks run along the long axis of the tape media from one end to the other. The number of tracks, and therefore the width of a track, varies with the tape's capacity. Most if not all newer drives provide backward-compatibility at least for reading (but often also for writing). QIC has a good reputation regarding the safety of the data (the mechanics are simpler and more robust than for helical scan drives). Tapes should be retired from use after 5,000 backups. XXX* Mini-Cartridge DLT tape media DLT DLT has the fastest data transfer rate of all the drive types listed here. The 1/2" (12.5mm) tape is contained in a single spool cartridge (4 x 4 x 1 inches; 100 x 100 x 25 mm). The cartridge has a swinging gate along one entire side of the cartridge. The drive mechanism opens this gate to extract the tape leader. The tape leader has an oval hole in it which the drive uses to "hook" the tape. The take-up spool is located inside the tape drive. All the other tape cartridges listed here (9 track tapes are the only exception) have both the supply and take-up spools located inside the tape cartridge itself. Data throughput is approximately 1.5MB/s, three times the throughput of 4mm, 8mm, or QIC tape drives. Data capacities range from 10 GB to 20 GB for a single drive. Drives are available in both multi-tape changers and multi-tape, multi-drive tape libraries containing from 5 to 900 tapes over 1 to 20 drives, providing from 50 GB to 9 TB of storage. With compression, DLT Type IV format supports up to 70 GB capacity. Data is recorded onto the tape in tracks parallel to the direction of travel (just like QIC tapes). Two tracks are written at once. Read/write head lifetimes are relatively long; once the tape stops moving, there is no relative motion between the heads and the tape. AIT tape media AIT AIT is a new format from Sony, and can hold up to 50 GB (with compression) per tape. The tapes contain memory chips which retain an index of the tape's contents. This index can be rapidly read by the tape drive to determine the position of files on the tape, instead of the several minutes that would be required for other tapes. Software such as SAMS:Alexandria can operate forty or more AIT tape libraries, communicating directly with the tape's memory chip to display the contents on screen, determine what files were backed up to which tape, locate the correct tape, load it, and restore the data from the tape. Libraries like this cost in the region of $20,000, pricing them a little out of the hobbyist market. Using a New Tape for the First Time The first time that you try to read or write a new, completely blank tape, the operation will fail. The console messages should be similar to: sa0(ncr1:4:0): NOT READY asc:4,1 sa0(ncr1:4:0): Logical unit is in process of becoming ready The tape does not contain an Identifier Block (block number 0). All QIC tape drives since the adoption of QIC-525 standard write an Identifier Block to the tape. There are two solutions: mt fsf 1 causes the tape drive to write an Identifier Block to the tape. Use the front panel button to eject the tape. Re-insert the tape and dump data to the tape. dump will report DUMP: End of tape detected and the console will show: HARDWARE FAILURE info:280 asc:80,96. rewind the tape using: mt rewind. Subsequent tape operations are successful. Backup Programs backup software The three major programs are &man.dump.8;, &man.tar.1;, and &man.cpio.1;. Dump and Restore backup software dump / restore dump restore The traditional Unix backup programs are dump and restore. They operate on the drive as a collection of disk blocks, below the abstractions of files, links and directories that are created by the filesystems. dump backs up an entire filesystem on a device. It is unable to backup only part of a filesystem or a directory tree that spans more than one filesystem. dump does not write files and directories to tape, but rather writes the raw data blocks that comprise files and directories. If you use dump on your root directory, you would not back up /home, /usr or many other directories since these are typically mount points for other filesystems or symbolic links into those filesystems. dumphas quirks that remain from its early days in Version 6 of AT&T Unix (circa 1975). The default parameters are suitable for 9-track tapes (6250 bpi), not the high-density media available today (up to 62,182 ftpi). These defaults must be overridden on the command line to utilize the capacity of current tape drives. rhosts It is also possible to backup data across the network to a tape drive attached to another computer with rdump and rrestore. Both programs rely upon rcmd and ruserok to access the remote tape drive. Therefore, the user performing the backup must have rhosts access to the remote computer. The arguments to rdump and rrestore must be suitable to use on the remote computer. (e.g. When rdumping from a FreeBSD computer to an Exabyte tape drive connected to a Sun called komodo, use: /sbin/rdump 0dsbfu 54000 13000 126 komodo:/dev/nrsa8 /dev/rda0a 2>&1) Beware: there are security implications to allowing rhosts commands. Evaluate your situation carefully. It is also possible to use rdump and rrestore in a more secure fashion over ssh. Using <command>rdump</command> over <application>ssh</application> &prompt.root; /sbin/dump -0uan -f - /usr | gzip -2 | ssh1 -c blowfish \ targetuser@targetmachine.example.com dd of=/mybigfiles/dump-usr-l0.gz <command>tar</command> backup software tar &man.tar.1; also dates back to Version 6 of AT&T Unix (circa 1975). tar operates in cooperation with the filesystem; tar writes files and directories to tape. tar does not support the full range of options that are available from &man.cpio.1;, but tar does not require the unusual command pipeline that cpio uses. tar Most versions of tar do not support backups across the network. The GNU version of tar, which FreeBSD utilizes, supports remote devices using the same syntax as rdump. To tar to an Exabyte tape drive connected to a Sun called komodo, use: /usr/bin/tar cf komodo:/dev/nrsa8 . 2>&1. For versions without remote device support, you can use a pipeline and rsh to send the data to a remote tape drive. &prompt.root; tar cf - . | rsh hostname dd of=tape-device obs=20b If you are worried about the security of backing up over a network you should use the ssh command instead of rsh. <command>cpio</command> backup software cpio &man.cpio.1; is the original Unix file interchange tape program for magnetic media. cpio has options (among many others) to perform byte-swapping, write a number of different archive formats, and pipe the data to other programs. This last feature makes cpio and excellent choice for installation media. cpio does not know how to walk the directory tree and a list of files must be provided through stdin. cpio cpio does not support backups across the network. You can use a pipeline and rsh to send the data to a remote tape drive. &prompt.root; for f in directory_list; do find $f >> backup.list done &prompt.root; cpio -v -o --format=newc < backup.list | ssh user@host "cat > backup_device" Where directory_list is the list of directories you want to back up, user@host is the user/hostname combination that will be performing the backups, and backup_device is where the backups should be written to (e.g., /dev/nrsa0). <command>pax</command> backup software pax pax POSIX IEEE &man.pax.1; is IEEE/POSIX's answer to tar and cpio. Over the years the various versions of tar and cpio have gotten slightly incompatible. So rather than fight it out to fully standardize them, POSIX created a new archive utility. pax attempts to read and write many of the various cpio and tar formats, plus new formats of its own. Its command set more resembles cpio than tar. <application>Amanda</application> backup software Amanda Amanda Amanda (Advanced Maryland Network Disk Archiver) is a client/server backup system, rather than a single program. An Amanda server will backup to a single tape drive any number of computers that have Amanda clients and a network connection to the Amanda server. A common problem at sites with a number of large disks is that the length of time required to backup to data directly to tape exceeds the amount of time available for the task. Amanda solves this problem. Amanda can use a "holding disk" to backup several filesystems at the same time. Amanda creates "archive sets": a group of tapes used over a period of time to create full backups of all the filesystems listed in Amanda's configuration file. The "archive set" also contains nightly incremental (or differential) backups of all the filesystems. Restoring a damaged filesystem requires the most recent full backup and the incremental backups. The configuration file provides fine control of backups and the network traffic that Amanda generates. Amanda will use any of the above backup programs to write the data to tape. Amanda is available as either a port or a package, it is not installed by default. Do Nothing Do nothing is not a computer program, but it is the most widely used backup strategy. There are no initial costs. There is no backup schedule to follow. Just say no. If something happens to your data, grin and bear it! If your time and your data is worth little to nothing, then Do nothing is the most suitable backup program for your computer. But beware, Unix is a useful tool, you may find that within six months you have a collection of files that are valuable to you. Do nothing is the correct backup method for /usr/obj and other directory trees that can be exactly recreated by your computer. An example is the files that comprise the HTML or Postscript version of this Handbook. These document formats have been created from SGML input files. Creating backups of the HTML or PostScript files is not necessary. The SGML files are backed up regularly. Which Backup Program Is Best? LISA &man.dump.8; Period. Elizabeth D. Zwicky torture tested all the backup programs discussed here. The clear choice for preserving all your data and all the peculiarities of Unix filesystems is dump. Elizabeth created filesystems containing a large variety of unusual conditions (and some not so unusual ones) and tested each program by doing a backup and restore of those filesystems. The peculiarities included: files with holes, files with holes and a block of nulls, files with funny characters in their names, unreadable and unwritable files, devices, files that change size during the backup, files that are created/deleted during the backup and more. She presented the results at LISA V in Oct. 1991. See torture-testing Backup and Archive Programs. Emergency Restore Procedure Before the Disaster There are only four steps that you need to perform in preparation for any disaster that may occur. disklabel First, print the disklabel from each of your disks (e.g. disklabel da0 | lpr), your filesystem table (/etc/fstab) and all boot messages, two copies of each. fix-it floppies Second, determine that the boot and fix-it floppies (boot.flp and fixit.flp) have all your devices. The easiest way to check is to reboot your machine with the boot floppy in the floppy drive and check the boot messages. If all your devices are listed and functional, skip on to step three. Otherwise, you have to create two custom bootable floppies which have a kernel that can mount all of your disks and access your tape drive. These floppies must contain: fdisk, disklabel, newfs, mount, and whichever backup program you use. These programs must be statically linked. If you use dump, the floppy must contain restore. Third, create backup tapes regularly. Any changes that you make after your last backup may be irretrievably lost. Write-protect the backup tapes. Fourth, test the floppies (either boot.flp and fixit.flp or the two custom bootable floppies you made in step two.) and backup tapes. Make notes of the procedure. Store these notes with the bootable floppy, the printouts and the backup tapes. You will be so distraught when restoring that the notes may prevent you from destroying your backup tapes (How? In place of tar xvf /dev/rsa0, you might accidentally type tar cvf /dev/rsa0 and over-write your backup tape). For an added measure of security, make bootable floppies and two backup tapes each time. Store one of each at a remote location. A remote location is NOT the basement of the same office building. A number of firms in the World Trade Center learned this lesson the hard way. A remote location should be physically separated from your computers and disk drives by a significant distance. A Script for Creating a Bootable Floppy /mnt/sbin/init gzip -c -best /sbin/fsck > /mnt/sbin/fsck gzip -c -best /sbin/mount > /mnt/sbin/mount gzip -c -best /sbin/halt > /mnt/sbin/halt gzip -c -best /sbin/restore > /mnt/sbin/restore gzip -c -best /bin/sh > /mnt/bin/sh gzip -c -best /bin/sync > /mnt/bin/sync cp /root/.profile /mnt/root cp -f /dev/MAKEDEV /mnt/dev chmod 755 /mnt/dev/MAKEDEV chmod 500 /mnt/sbin/init chmod 555 /mnt/sbin/fsck /mnt/sbin/mount /mnt/sbin/halt chmod 555 /mnt/bin/sh /mnt/bin/sync chmod 6555 /mnt/sbin/restore # # create the devices nodes # cd /mnt/dev ./MAKEDEV std ./MAKEDEV da0 ./MAKEDEV da1 ./MAKEDEV da2 ./MAKEDEV sa0 ./MAKEDEV pty0 cd / # # create minimum filesystem table # cat > /mnt/etc/fstab < /mnt/etc/passwd < /mnt/etc/master.passwd < After the Disaster The key question is: did your hardware survive? You have been doing regular backups so there is no need to worry about the software. If the hardware has been damaged. First, replace those parts that have been damaged. If your hardware is okay, check your floppies. If you are using a custom boot floppy, boot single-user (type -s at the boot: prompt). Skip the following paragraph. If you are using the boot.flp and fixit.flp floppies, keep reading. Insert the boot.flp floppy in the first floppy drive and boot the computer. The original install menu will be displayed on the screen. Select the Fixit--Repair mode with CDROM or floppy. option. Insert the fixit.flp when prompted. restore and the other programs that you need are located in /mnt2/stand. Recover each filesystem separately. mount root partition disklabel newfs Try to mount (e.g. mount /dev/da0a /mnt) the root partition of your first disk. If the disklabel was damaged, use disklabel to re-partition and label the disk to match the label that you printed and saved. Use newfs to re-create the filesystems. Re-mount the root partition of the floppy read-write (mount -u -o rw /mnt). Use your backup program and backup tapes to recover the data for this filesystem (e.g. restore vrf /dev/sa0). Unmount the filesystem (e.g. umount /mnt) Repeat for each filesystem that was damaged. Once your system is running, backup your data onto new tapes. Whatever caused the crash or data loss may strike again. Another hour spent now may save you from further distress later. * I did not prepare for the Disaster, What Now? ]]> Julio Merino Contributed by Floppy disks Floppy disks are, nowadays, an old-fashioned medium to store/share data. Although, there are still some times when you need to use them, because you do not have any other removable storage media or you need to use what you have saved in them on any other computer. This section will explain how to use floppy disks in FreeBSD, that is, formating and copying/restoring data from them. But... I really have written this to help you about how to create forced-size floppies. The device Floppy disks are accessed through entries in /dev (like any other device). To access the raw floppy disk you can use /dev/rfdX, where X stands for the drive number, usually 0. When the disk is formatted you can use /dev/fdX, or whichever of the other devices named /dev/fdXY, where Y stands for a letter. These are all the same. Other important devices are /dev/fdX.size, where size is a floppy disk size in kilobytes. These entries are used at low-level format time to determine the disk size. Sometimes you will have to (re)create these entries under /dev. To do it, you can issue: &prompt.root; cd /dev && ./MAKEDEV "fd*" Formatting A floppy disk needs to be low-level formated before it can be used. This is usually done by the vendor but you may want to do it to check media integrity or to force the disk capacity to be bigger. To format the floppy at a low-level fashion you need to use fdformat. This utility expects the device name as an argument. We will use those /dev/fdX.size devices, which will allow us to format the floppy to its real size, or force them. So you insert a new 3.5inch floppy disk in your drive and issue: &prompt.root; /usr/sbin/fdformat /dev/rfd0.1440 This will take a while... You should notice any disk error here (this can help you determining which disks are good or bad). To force the floppy disk size, we will use other entries in /dev. Get the same floppy and issue: &prompt.root; /usr/sbin/fdformat /dev/rfd0.1720 It will take some more time than before (forced disks are slower). When it finishes, you will have a 1720kb floppy disk, but for the moment you will not notice any difference. You may use other disk sizes that you can find in /dev, but the most stable/compatible is the 1720kb for 3.5inch disks. The disklabel After low-level formatting the disk, you will need to place a disklabel on it. This disklabel will be destroyed later, but it is needed by the system to determine the size of the disk and its geometry later. The new disklabel will take over the whole disk, and will contain all the proper information about the geometry of the normal or forced floppy. Take a look to /etc/disktab now; you will see geometry values of all kinds of floppy disks. You can run now disklabel like: &prompt.root; /sbin/disklabel -B -r -w /dev/rfd0 fdsize Replace fdsize with fd1440, fd1720 or whichever size you want. The last field instructs disklabel which entry to take from /etc/disktab to use. The filesystem Now your floppy is ready to be high-level formated. This will place a new filesystem on it, which will let FreeBSD read and write to the disk. After creating the new filesystem, the disklabel is destroyed, so if you want to reformat the disk, you will have to recreate the disklabel another time. You can choose now which filesystem to use on your floppy. You can use UFS or FAT, though UFS is not a good idea for floppies. Choose FAT which is nice for floppies. To put a new filesystem on the floppy do this: &prompt.root; /sbin/newfs_msdos /dev/fd0 As we created a disklabel before, newfs will be able to fetch disk data and construct the new filesystem. And now, your disk is ready for use... Using the floppy You have two choices to use the floppy. You can either mount the disk with mount_msdos, or you can use mtools. Mtools are great, but you will need to install them from the ports system. Try it; issue a mdir. If you forced the disk, you will notice its extra size! A last note about forced disks: they are compatible with practically all other operating systems without any external utility to read/write them. Microsoft systems will recognize them without problems. But note that there may be times when the floppy drive itself is not able to read them (this may happen with very old drives). Backups to Floppies Can I Use floppies for Backing Up My Data? backup floppies floppy disks Floppy disks are not really a suitable media for making backups as: The media is unreliable, especially over long periods of time Backing up and restoring is very slow They have a very limited capacity (the days of backing up an entire hard disk onto a dozen or so floppies has long since passed). However, if you have no other method of backing up your data then floppy disks are better than no backup at all. If you do have to use floppy disks then ensure that you use good quality ones. Floppies that have been lying around the office for a couple of years are a bad choice. Ideally use new ones from a reputable manufacturer. So How Do I Backup My Data to Floppies? The best way to backup to floppy disk is to use tar with the (multi volume) option, which allows backups to span multiple floppies. To backup all the files in the current directory and sub-directory use this (as root): &prompt.root; tar Mcvf /dev/fd0 * When the first floppy is full tar will prompt you to insert the next volume (because tar is media independent it refers to volumes. In this context it means floppy disk) Prepare volume #2 for /dev/fd0 and hit return: This is repeated (with the volume number incrementing) until all the specified files have been archived. Can I Compress My Backups? tar gzip compression Unfortunately, tar will not allow the option to be used for multi-volume archives. You could, of course, gzip all the files, tar them to the floppies, then gunzip the files again! How Do I Restore My Backups? To restore the entire archive use: &prompt.root; tar Mxvf /dev/fd0 There are two ways that you can use to restore only specific files. First, you can start with the first floppy and use: &prompt.root; tar Mxvf /dev/fd0 filename tar will prompt you to insert subsequent floppies until it finds the required file. Alternatively, if you know which floppy the file is on then you can simply insert that floppy and use the same command as above. Note that if the first file on the floppy is a continuation from the previous one then tar will warn you that it cannot restore it, even if you have not asked it to!