diff --git a/documentation/content/en/books/design-44bsd/_index.adoc b/documentation/content/en/books/design-44bsd/_index.adoc index 289efebf32..c149640619 100644 --- a/documentation/content/en/books/design-44bsd/_index.adoc +++ b/documentation/content/en/books/design-44bsd/_index.adoc @@ -1,852 +1,853 @@ --- title: The Design and Implementation of the 4.4BSD Operating System authors: - author: Marshall Kirk McKusick - author: Keith Bostic - author: Michael J. Karels - author: John S. Quarterman copyright: 1996 Addison-Wesley Longman, Inc description: Donated by Addison-Wesley, provides a design overview of 4.4BSD, from which FreeBSD was originally derived trademarks: ["design-44bsd"] bookOrder: 60 tags: ["4.4BSD", "design", "operating system", "BSD", "UNIX"] +layout: single --- = The Design and Implementation of the 4.4BSD Operating System :doctype: book :toc: macro :toclevels: 2 :icons: font :sectnums: :sectnumlevels: 6 :partnums: :sectnumoffset: 2 :source-highlighter: rouge :experimental: :images-path: books/design-44bsd/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../images/{images-path} include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] include::../../../../../shared/asciidoctor.adoc[] endif::[] ''' toc::[] [[overview]] == Design Overview of 4.4BSD [[overview-facilities]] === 4.4BSD Facilities and the Kernel The 4.4BSD kernel provides four basic facilities: processes, a filesystem, communications, and system startup. This section outlines where each of these four basic services is described in this book. . Processes constitute a thread of control in an address space. Mechanisms for creating, terminating, and otherwise controlling processes are described in Chapter 4. The system multiplexes separate virtual-address spaces for each process; this memory management is discussed in Chapter 5. . The user interface to the filesystem and devices is similar; common aspects are discussed in Chapter 6. The filesystem is a set of named files, organized in a tree-structured hierarchy of directories, and of operations to manipulate them, as presented in Chapter 7. Files reside on physical media such as disks. 4.4BSD supports several organizations of data on the disk, as set forth in Chapter 8. Access to files on remote machines is the subject of Chapter 9. Terminals are used to access the system; their operation is the subject of Chapter 10. . Communication mechanisms provided by traditional UNIX systems include simplex reliable byte streams between related processes (see pipes, Section 11.1), and notification of exceptional events (see signals, Section 4.7). 4.4BSD also has a general interprocess-communication facility. This facility, described in Chapter 11, uses access mechanisms distinct from those of the filesystem, but, once a connection is set up, a process can access it as though it were a pipe. There is a general networking framework, discussed in Chapter 12, that is normally used as a layer underlying the IPC facility. Chapter 13 describes a particular networking implementation in detail. . Any real operating system has operational issues, such as how to start it running. Startup and operational issues are described in Chapter 14. Sections 2.3 through 2.14 present introductory material related to Chapters 3 through 14. We shall define terms, mention basic system calls, and explore historical developments. Finally, we shall give the reasons for many major design decisions. ==== The Kernel The _kernel_ is the part of the system that runs in protected mode and mediates access by all user programs to the underlying hardware (e.g., CPU, disks, terminals, network links) and software constructs (e.g., filesystem, network protocols). The kernel provides the basic system facilities; it creates and manages processes, and provides functions to access the filesystem and communication facilities. These functions, called _system calls_ appear to user processes as library subroutines. These system calls are the only interface that processes have to these facilities. Details of the system-call mechanism are given in Chapter 3, as are descriptions of several kernel mechanisms that do not execute as the direct result of a process doing a system call. A _kernel_ in traditional operating-system terminology, is a small nucleus of software that provides only the minimal facilities necessary for implementing additional operating-system services. In contemporary research operating systems -- such as Chorus <>, Mach <>, Tunis <>, and the V Kernel <> -- this division of functionality is more than just a logical one. Services such as filesystems and networking protocols are implemented as client application processes of the nucleus or kernel. The 4.4BSD kernel is not partitioned into multiple processes. This basic design decision was made in the earliest versions of UNIX. The first two implementations by Ken Thompson had no memory mapping, and thus made no hardware-enforced distinction between user and kernel space <>. A message-passing system could have been implemented as readily as the actually implemented model of kernel and user processes. The monolithic kernel was chosen for simplicity and performance. And the early kernels were small; the inclusion of facilities such as networking into the kernel has increased its size. The current trend in operating-systems research is to reduce the kernel size by placing such services in user space. Users ordinarily interact with the system through a command-language interpreter, called a _shell_, and perhaps through additional user application programs. Such programs and the shell are implemented with processes. Details of such programs are beyond the scope of this book, which instead concentrates almost exclusively on the kernel. Sections 2.3 and 2.4 describe the services provided by the 4.4BSD kernel, and give an overview of the latter's design. Later chapters describe the detailed design and implementation of these services as they appear in 4.4BSD. [[overview-kernel-organization]] === Kernel Organization In this section, we view the organization of the 4.4BSD kernel in two ways: [arabic] . As a static body of software, categorized by the functionality offered by the modules that make up the kernel . By its dynamic operation, categorized according to the services provided to users The largest part of the kernel implements the system services that applications access through system calls. In 4.4BSD, this software has been organized according to the following: * Basic kernel facilities: timer and system-clock handling, descriptor management, and process management * Memory-management support: paging and swapping * Generic system interfaces: the I/O, control, and multiplexing operations performed on descriptors * The filesystem: files, directories, pathname translation, file locking, and I/O buffer management * Terminal-handling support: the terminal-interface driver and terminal line disciplines * Interprocess-communication facilities: sockets * Support for network communication: communication protocols and generic network facilities, such as routing .Machine-independent software in the 4.4BSD kernel [[table-mach-indep]] [cols=",,",options="header",] |=== |Category |Lines of code |Percentage of kernel |headers |9,393 |4.6 |initialization |1,107 |0.6 |kernel facilities |8,793 |4.4 |generic interfaces |4,782 |2.4 |interprocess communication |4,540 |2.2 |terminal handling |3,911 |1.9 |virtual memory |11,813 |5.8 |vnode management |7,954 |3.9 |filesystem naming |6,550 |3.2 |fast filestore |4,365 |2.2 |log-structure filestore |4,337 |2.1 |memory-based filestore |645 |0.3 |cd9660 filesystem |4,177 |2.1 |miscellaneous filesystems (10) |12,695 |6.3 |network filesystem |17,199 |8.5 |network communication |8,630 |4.3 |internet protocols |11,984 |5.9 |ISO protocols |23,924 |11.8 |X.25 protocols |10,626 |5.3 |XNS protocols |5,192 |2.6 |=== Most of the software in these categories is machine independent and is portable across different hardware architectures. The machine-dependent aspects of the kernel are isolated from the mainstream code. In particular, none of the machine-independent code contains conditional code for specific architecture. When an architecture-dependent action is needed, the machine-independent code calls an architecture-dependent function that is located in the machine-dependent code. The software that is machine dependent includes * Low-level system-startup actions * Trap and fault handling * Low-level manipulation of the run-time context of a process * Configuration and initialization of hardware devices * Run-time support for I/O devices .Machine-dependent software for the HP300 in the 4.4BSD kernel [[table-mach-dep]] [cols=",,",options="header",] |=== |Category |Lines of code |Percentage of kernel |machine dependent headers |1,562 |0.8 |device driver headers |3,495 |1.7 |device driver source |17,506 |8.7 |virtual memory |3,087 |1.5 |other machine dependent |6,287 |3.1 |routines in assembly language |3,014 |1.5 |HP/UX compatibility |4,683 |2.3 |=== <> summarizes the machine-independent software that constitutes the 4.4BSD kernel for the HP300. The numbers in column 2 are for lines of C source code, header files, and assembly language. Virtually all the software in the kernel is written in the C programming language; less than 2 percent is written in assembly language. As the statistics in <> show, the machine-dependent software, excluding HP/UX and device support, accounts for a minuscule 6.9 percent of the kernel. Only a small part of the kernel is devoted to initializing the system. This code is used when the system is _bootstrapped_ into operation and is responsible for setting up the kernel hardware and software environment (see Chapter 14). Some operating systems (especially those with limited physical memory) discard or _overlay_ the software that performs these functions after that software has been executed. The 4.4BSD kernel does not reclaim the memory used by the startup code because that memory space is barely 0.5 percent of the kernel resources used on a typical machine. Also, the startup code does not appear in one place in the kernel -- it is scattered throughout, and it usually appears in places logically associated with what is being initialized. [[overview-kernel-service]] === Kernel Services The boundary between the kernel- and user-level code is enforced by hardware-protection facilities provided by the underlying hardware. The kernel operates in a separate address space that is inaccessible to user processes. Privileged operations -- such as starting I/O and halting the central processing unit (CPU) -- are available to only the kernel. Applications request services from the kernel with _system calls_. System calls are used to cause the kernel to execute complicated operations, such as writing data to secondary storage, and simple operations, such as returning the current time of day. All system calls appear _synchronous_ to applications: The application does not run while the kernel does the actions associated with a system call. The kernel may finish some operations associated with a system call after it has returned. For example, a _write_ system call will copy the data to be written from the user process to a kernel buffer while the process waits, but will usually return from the system call before the kernel buffer is written to the disk. A system call usually is implemented as a hardware trap that changes the CPU's execution mode and the current address-space mapping. Parameters supplied by users in system calls are validated by the kernel before being used. Such checking ensures the integrity of the system. All parameters passed into the kernel are copied into the kernel's address space, to ensure that validated parameters are not changed as a side effect of the system call. System-call results are returned by the kernel, either in hardware registers or by their values being copied to user-specified memory addresses. Like parameters passed into the kernel, addresses used for the return of results must be validated to ensure that they are part of an application's address space. If the kernel encounters an error while processing a system call, it returns an error code to the user. For the C programming language, this error code is stored in the global variable _errno_, and the function that executed the system call returns the value -1. User applications and the kernel operate independently of each other. 4.4BSD does not store I/O control blocks or other operating-system-related data structures in the application's address space. Each user-level application is provided an independent address space in which it executes. The kernel makes most state changes, such as suspending a process while another is running, invisible to the processes involved. [[overview-process-management]] === Process Management 4.4BSD supports a multitasking environment. Each task or thread of execution is termed a _process_. The _context_ of a 4.4BSD process consists of user-level state, including the contents of its address space and the run-time environment, and kernel-level state, which includes scheduling parameters, resource controls, and identification information. The context includes everything used by the kernel in providing services for the process. Users can create processes, control the processes' execution, and receive notification when the processes' execution status changes. Every process is assigned a unique value, termed a _process identifier_ (PID). This value is used by the kernel to identify a process when reporting status changes to a user, and by a user when referencing a process in a system call. The kernel creates a process by duplicating the context of another process. The new process is termed a _child process_ of the original _parent process_ The context duplicated in process creation includes both the user-level execution state of the process and the process's system state managed by the kernel. Important components of the kernel state are described in Chapter 4. [[fig-process-lifecycle]] .Process lifecycle image:fig1.png[Process lifecycle] The process lifecycle is depicted in <>. A process may create a new process that is a copy of the original by using the _fork_ system call. The _fork_ call returns twice: once in the parent process, where the return value is the process identifier of the child, and once in the child process, where the return value is 0. The parent-child relationship induces a hierarchical structure on the set of processes in the system. The new process shares all its parent's resources, such as file descriptors, signal-handling status, and memory layout. Although there are occasions when the new process is intended to be a copy of the parent, the loading and execution of a different program is a more useful and typical action. A process can overlay itself with the memory image of another program, passing to the newly created image a set of parameters, using the system call _execve_. One parameter is the name of a file whose contents are in a format recognized by the system -- either a binary-executable file or a file that causes the execution of a specified interpreter program to process its contents. A process may terminate by executing an _exit_ system call, sending 8 bits of exit status to its parent. If a process wants to communicate more than a single byte of information with its parent, it must either set up an interprocess-communication channel using pipes or sockets, or use an intermediate file. Interprocess communication is discussed extensively in Chapter 11. A process can suspend execution until any of its child processes terminate using the _wait_ system call, which returns the PID and exit status of the terminated child process. A parent process can arrange to be notified by a signal when a child process exits or terminates abnormally. Using the _wait4_ system call, the parent can retrieve information about the event that caused termination of the child process and about resources consumed by the process during its lifetime. If a process is orphaned because its parent exits before it is finished, then the kernel arranges for the child's exit status to be passed back to a special system process _init_: see Sections 3.1 and 14.6). The details of how the kernel creates and destroys processes are given in Chapter 5. Processes are scheduled for execution according to a _process-priority_ parameter. This priority is managed by a kernel-based scheduling algorithm. Users can influence the scheduling of a process by specifying a parameter (_nice_) that weights the overall scheduling priority, but are still obligated to share the underlying CPU resources according to the kernel's scheduling policy. ==== Signals The system defines a set of _signals_ that may be delivered to a process. Signals in 4.4BSD are modeled after hardware interrupts. A process may specify a user-level subroutine to be a _handler_ to which a signal should be delivered. When a signal is generated, it is blocked from further occurrence while it is being _caught_ by the handler. Catching a signal involves saving the current process context and building a new one in which to run the handler. The signal is then delivered to the handler, which can either abort the process or return to the executing process (perhaps after setting a global variable). If the handler returns, the signal is unblocked and can be generated (and caught) again. Alternatively, a process may specify that a signal is to be _ignored_, or that a default action, as determined by the kernel, is to be taken. The default action of certain signals is to terminate the process. This termination may be accompanied by creation of a _core file_ that contains the current memory image of the process for use in postmortem debugging. Some signals cannot be caught or ignored. These signals include _SIGKILL_, which kills runaway processes, and the job-control signal _SIGSTOP_. A process may choose to have signals delivered on a special stack so that sophisticated software stack manipulations are possible. For example, a language supporting coroutines needs to provide a stack for each coroutine. The language run-time system can allocate these stacks by dividing up the single stack provided by 4.4BSD. If the kernel does not support a separate signal stack, the space allocated for each coroutine must be expanded by the amount of space required to catch a signal. All signals have the same _priority_. If multiple signals are pending simultaneously, the order in which signals are delivered to a process is implementation specific. Signal handlers execute with the signal that caused their invocation to be blocked, but other signals may yet occur. Mechanisms are provided so that processes can protect critical sections of code against the occurrence of specified signals. The detailed design and implementation of signals is described in Section 4.7. ==== Process Groups and Sessions Processes are organized into _process groups_. Process groups are used to control access to terminals and to provide a means of distributing signals to collections of related processes. A process inherits its process group from its parent process. Mechanisms are provided by the kernel to allow a process to alter its process group or the process group of its descendents. Creating a new process group is easy; the value of a new process group is ordinarily the process identifier of the creating process. The group of processes in a process group is sometimes referred to as a _job_ and is manipulated by high-level system software, such as the shell. A common kind of job created by a shell is a _pipeline_ of several processes connected by pipes, such that the output of the first process is the input of the second, the output of the second is the input of the third, and so forth. The shell creates such a job by forking a process for each stage of the pipeline, then putting all those processes into a separate process group. A user process can send a signal to each process in a process group, as well as to a single process. A process in a specific process group may receive software interrupts affecting the group, causing the group to suspend or resume execution, or to be interrupted or terminated. A terminal has a process-group identifier assigned to it. This identifier is normally set to the identifier of a process group associated with the terminal. A job-control shell may create a number of process groups associated with the same terminal; the terminal is the _controlling terminal_ for each process in these groups. A process may read from a descriptor for its controlling terminal only if the terminal's process-group identifier matches that of the process. If the identifiers do not match, the process will be blocked if it attempts to read from the terminal. By changing the process-group identifier of the terminal, a shell can arbitrate a terminal among several different jobs. This arbitration is called _job control_ and is described, with process groups, in Section 4.8. Just as a set of related processes can be collected into a process group, a set of process groups can be collected into a _session_. The main uses for sessions are to create an isolated environment for a daemon process and its children, and to collect together a user's login shell and the jobs that shell spawns. [[overview-memory-management]] === Memory Management Each process has its own private address space. The address space is initially divided into three logical segments: _text_, _data_, and _stack_. The text segment is read-only and contains the machine instructions of a program. The data and stack segments are both readable and writable. The data segment contains the initialized and uninitialized data portions of a program, whereas the stack segment holds the application's run-time stack. On most machines, the stack segment is extended automatically by the kernel as the process executes. A process can expand or contract its data segment by making a system call, whereas a process can change the size of its text segment only when the segment's contents are overlaid with data from the filesystem, or when debugging takes place. The initial contents of the segments of a child process are duplicates of the segments of a parent process. The entire contents of a process address space do not need to be resident for a process to execute. If a process references a part of its address space that is not resident in main memory, the system _pages_ the necessary information into memory. When system resources are scarce, the system uses a two-level approach to maintain available resources. If a modest amount of memory is available, the system will take memory resources away from processes if these resources have not been used recently. Should there be a severe resource shortage, the system will resort to _swapping_ the entire context of a process to secondary storage. The _demand paging_ and _swapping_ done by the system are effectively transparent to processes. A process may, however, advise the system about expected future memory utilization as a performance aid. ==== BSD Memory-Management Design Decisions The support of large sparse address spaces, mapped files, and shared memory was a requirement for 4.2BSD. An interface was specified, called _mmap_, that allowed unrelated processes to request a shared mapping of a file into their address spaces. If multiple processes mapped the same file into their address spaces, changes to the file's portion of an address space by one process would be reflected in the area mapped by the other processes, as well as in the file itself. Ultimately, 4.2BSD was shipped without the _mmap_ interface, because of pressure to make other features, such as networking, available. Further development of the _mmap_ interface continued during the work on 4.3BSD. Over 40 companies and research groups participated in the discussions leading to the revised architecture that was described in the Berkeley Software Architecture Manual <>. Several of the companies have implemented the revised interface <>. Once again, time pressure prevented 4.3BSD from providing an implementation of the interface. Although the latter could have been built into the existing 4.3BSD virtual-memory system, the developers decided not to put it in because that implementation was nearly 10 years old. Furthermore, the original virtual-memory design was based on the assumption that computer memories were small and expensive, whereas disks were locally connected, fast, large, and inexpensive. Thus, the virtual-memory system was designed to be frugal with its use of memory at the expense of generating extra disk traffic. In addition, the 4.3BSD implementation was riddled with VAX memory-management hardware dependencies that impeded its portability to other computer architectures. Finally, the virtual-memory system was not designed to support the tightly coupled multiprocessors that are becoming increasingly common and important today. Attempts to improve the old implementation incrementally seemed doomed to failure. A completely new design, on the other hand, could take advantage of large memories, conserve disk transfers, and have the potential to run on multiprocessors. Consequently, the virtual-memory system was completely replaced in 4.4BSD. The 4.4BSD virtual-memory system is based on the Mach 2.0 VM system <>. with updates from Mach 2.5 and Mach 3.0. It features efficient support for sharing, a clean separation of machine-independent and machine-dependent features, as well as (currently unused) multiprocessor support. Processes can map files anywhere in their address space. They can share parts of their address space by doing a shared mapping of the same file. Changes made by one process are visible in the address space of the other process, and also are written back to the file itself. Processes can also request private mappings of a file, which prevents any changes that they make from being visible to other processes mapping the file or being written back to the file itself. Another issue with the virtual-memory system is the way that information is passed into the kernel when a system call is made. 4.4BSD always copies data from the process address space into a buffer in the kernel. For read or write operations that are transferring large quantities of data, doing the copy can be time consuming. An alternative to doing the copying is to remap the process memory into the kernel. The 4.4BSD kernel always copies the data for several reasons: * Often, the user data are not page aligned and are not a multiple of the hardware page length. * If the page is taken away from the process, it will no longer be able to reference that page. Some programs depend on the data remaining in the buffer even after those data have been written. * If the process is allowed to keep a copy of the page (as it is in current 4.4BSD semantics), the page must be made _copy-on-write_. A copy-on-write page is one that is protected against being written by being made read-only. If the process attempts to modify the page, the kernel gets a write fault. The kernel then makes a copy of the page that the process can modify. Unfortunately, the typical process will immediately try to write new data to its output buffer, forcing the data to be copied anyway. * When pages are remapped to new virtual-memory addresses, most memory-management hardware requires that the hardware address-translation cache be purged selectively. The cache purges are often slow. The net effect is that remapping is slower than copying for blocks of data less than 4 to 8 Kbyte. The biggest incentives for memory mapping are the needs for accessing big files and for passing large quantities of data between processes. The _mmap_ interface provides a way for both of these tasks to be done without copying. ==== Memory Management Inside the Kernel The kernel often does allocations of memory that are needed for only the duration of a single system call. In a user process, such short-term memory would be allocated on the run-time stack. Because the kernel has a limited run-time stack, it is not feasible to allocate even moderate-sized blocks of memory on it. Consequently, such memory must be allocated through a more dynamic mechanism. For example, when the system must translate a pathname, it must allocate a 1-Kbyte buffer to hold the name. Other blocks of memory must be more persistent than a single system call, and thus could not be allocated on the stack even if there was space. An example is protocol-control blocks that remain throughout the duration of a network connection. Demands for dynamic memory allocation in the kernel have increased as more services have been added. A generalized memory allocator reduces the complexity of writing code inside the kernel. Thus, the 4.4BSD kernel has a single memory allocator that can be used by any part of the system. It has an interface similar to the C library routines _malloc_ and _free_ that provide memory allocation to application programs <>. Like the C library interface, the allocation routine takes a parameter specifying the size of memory that is needed. The range of sizes for memory requests is not constrained; however, physical memory is allocated and is not paged. The free routine takes a pointer to the storage being freed, but does not require the size of the piece of memory being freed. [[overview-io-system]] === I/O System The basic model of the UNIX I/O system is a sequence of bytes that can be accessed either randomly or sequentially. There are no _access methods_ and no _control blocks_ in a typical UNIX user process. Different programs expect various levels of structure, but the kernel does not impose structure on I/O. For instance, the convention for text files is lines of ASCII characters separated by a single newline character (the ASCII line-feed character), but the kernel knows nothing about this convention. For the purposes of most programs, the model is further simplified to being a stream of data bytes, or an _I/O stream_. It is this single common data form that makes the characteristic UNIX tool-based approach work <>. An I/O stream from one program can be fed as input to almost any other program. (This kind of traditional UNIX I/O stream should not be confused with the Eighth Edition stream I/O system or with the System V, Release 3 STREAMS, both of which can be accessed as traditional I/O streams.) ==== Descriptors and I/O UNIX processes use _descriptors_ to reference I/O streams. Descriptors are small unsigned integers obtained from the _open_ and _socket_ system calls. The _open_ system call takes as arguments the name of a file and a permission mode to specify whether the file should be open for reading or for writing, or for both. This system call also can be used to create a new, empty file. A _read_ or _write_ system call can be applied to a descriptor to transfer data. The _close_ system call can be used to deallocate any descriptor. Descriptors represent underlying objects supported by the kernel, and are created by system calls specific to the type of object. In 4.4BSD, three kinds of objects can be represented by descriptors: files, pipes, and sockets. * A _file_ is a linear array of bytes with at least one name. A file exists until all its names are deleted explicitly and no process holds a descriptor for it. A process acquires a descriptor for a file by opening that file's name with the _open_ system call. I/O devices are accessed as files. * A _pipe_ is a linear array of bytes, as is a file, but it is used solely as an I/O stream, and it is unidirectional. It also has no name, and thus cannot be opened with _open_. Instead, it is created by the _pipe_ system call, which returns two descriptors, one of which accepts input that is sent to the other descriptor reliably, without duplication, and in order. The system also supports a named pipe or FIFO. A FIFO has properties identical to a pipe, except that it appears in the filesystem; thus, it can be opened using the _open_ system call. Two processes that wish to communicate each open the FIFO: One opens it for reading, the other for writing. * A _socket_ is a transient object that is used for interprocess communication; it exists only as long as some process holds a descriptor referring to it. A socket is created by the _socket_ system call, which returns a descriptor for it. There are different kinds of sockets that support various communication semantics, such as reliable delivery of data, preservation of message ordering, and preservation of message boundaries. In systems before 4.2BSD, pipes were implemented using the filesystem; when sockets were introduced in 4.2BSD, pipes were reimplemented as sockets. The kernel keeps for each process a _descriptor table_, which is a table that the kernel uses to translate the external representation of a descriptor into an internal representation. (The descriptor is merely an index into this table.) The descriptor table of a process is inherited from that process's parent, and thus access to the objects to which the descriptors refer also is inherited. The main ways that a process can obtain a descriptor are by opening or creation of an object, and by inheritance from the parent process. In addition, socket IPC allows passing of descriptors in messages between unrelated processes on the same machine. Every valid descriptor has an associated _file offset_ in bytes from the beginning of the object. Read and write operations start at this offset, which is updated after each data transfer. For objects that permit random access, the file offset also may be set with the _lseek_ system call. Ordinary files permit random access, and some devices do, as well. Pipes and sockets do not. When a process terminates, the kernel reclaims all the descriptors that were in use by that process. If the process was holding the final reference to an object, the object's manager is notified so that it can do any necessary cleanup actions, such as final deletion of a file or deallocation of a socket. ==== Descriptor Management Most processes expect three descriptors to be open already when they start running. These descriptors are 0, 1, 2, more commonly known as _standard input_, _standard output_, and _standard error_, respectively. Usually, all three are associated with the user's terminal by the login process (see Section 14.6) and are inherited through _fork_ and _exec_ by processes run by the user. Thus, a program can read what the user types by reading standard input, and the program can send output to the user's screen by writing to standard output. The standard error descriptor also is open for writing and is used for error output, whereas standard output is used for ordinary output. These (and other) descriptors can be mapped to objects other than the terminal; such mapping is called _I/O redirection_, and all the standard shells permit users to do it. The shell can direct the output of a program to a file by closing descriptor 1 (standard output) and opening the desired output file to produce a new descriptor 1. It can similarly redirect standard input to come from a file by closing descriptor 0 and opening the file. Pipes allow the output of one program to be input to another program without rewriting or even relinking of either program. Instead of descriptor 1 (standard output) of the source program being set up to write to the terminal, it is set up to be the input descriptor of a pipe. Similarly, descriptor 0 (standard input) of the sink program is set up to reference the output of the pipe, instead of the terminal keyboard. The resulting set of two processes and the connecting pipe is known as a _pipeline_. Pipelines can be arbitrarily long series of processes connected by pipes. The _open_, _pipe_, and _socket_ system calls produce new descriptors with the lowest unused number usable for a descriptor. For pipelines to work, some mechanism must be provided to map such descriptors into 0 and 1. The _dup_ system call creates a copy of a descriptor that points to the same file-table entry. The new descriptor is also the lowest unused one, but if the desired descriptor is closed first, _dup_ can be used to do the desired mapping. Care is required, however: If descriptor 1 is desired, and descriptor 0 happens also to have been closed, descriptor 0 will be the result. To avoid this problem, the system provides the _dup2_ system call; it is like _dup_, but it takes an additional argument specifying the number of the desired descriptor (if the desired descriptor was already open, _dup2_ closes it before reusing it). ==== Devices Hardware devices have filenames, and may be accessed by the user via the same system calls used for regular files. The kernel can distinguish a _device special file_ or _special file_, and can determine to what device it refers, but most processes do not need to make this determination. Terminals, printers, and tape drives are all accessed as though they were streams of bytes, like 4.4BSD disk files. Thus, device dependencies and peculiarities are kept in the kernel as much as possible, and even in the kernel most of them are segregated in the device drivers. Hardware devices can be categorized as either _structured_ or _unstructured_; they are known as _block_ or _character_ devices, respectively. Processes typically access devices through _special files_ in the filesystem. I/O operations to these files are handled by kernel-resident software modules termed _device drivers_. Most network-communication hardware devices are accessible through only the interprocess-communication facilities, and do not have special files in the filesystem name space, because the _raw-socket_ interface provides a more natural interface than does a special file. Structured or block devices are typified by disks and magnetic tapes, and include most random-access devices. The kernel supports read-modify-write-type buffering actions on block-oriented structured devices to allow the latter to be read and written in a totally random byte-addressed fashion, like regular files. Filesystems are created on block devices. Unstructured devices are those devices that do not support a block structure. Familiar unstructured devices are communication lines, raster plotters, and unbuffered magnetic tapes and disks. Unstructured devices typically support large block I/O transfers. Unstructured files are called _character devices_ because the first of these to be implemented were terminal device drivers. The kernel interface to the driver for these devices proved convenient for other devices that were not block structured. Device special files are created by the _mknod_ system call. There is an additional system call, _ioctl_, for manipulating the underlying device parameters of special files. The operations that can be done differ for each device. This system call allows the special characteristics of devices to be accessed, rather than overloading the semantics of other system calls. For example, there is an _ioctl_ on a tape drive to write an end-of-tape mark, instead of there being a special or modified version of _write_. ==== Socket IPC The 4.2BSD kernel introduced an IPC mechanism more flexible than pipes, based on _sockets_. A socket is an endpoint of communication referred to by a descriptor, just like a file or a pipe. Two processes can each create a socket, and then connect those two endpoints to produce a reliable byte stream. Once connected, the descriptors for the sockets can be read or written by processes, just as the latter would do with a pipe. The transparency of sockets allows the kernel to redirect the output of one process to the input of another process residing on another machine. A major difference between pipes and sockets is that pipes require a common parent process to set up the communications channel. A connection between sockets can be set up by two unrelated processes, possibly residing on different machines. System V provides local interprocess communication through FIFOs (also known as _named pipes_). FIFOs appear as an object in the filesystem that unrelated processes can open and send data through in the same way as they would communicate through a pipe. Thus, FIFOs do not require a common parent to set them up; they can be connected after a pair of processes are up and running. Unlike sockets, FIFOs can be used on only a local machine; they cannot be used to communicate between processes on different machines. FIFOs are implemented in 4.4BSD only because they are required by the POSIX.1 standard. Their functionality is a subset of the socket interface. The socket mechanism requires extensions to the traditional UNIX I/O system calls to provide the associated naming and connection semantics. Rather than overloading the existing interface, the developers used the existing interfaces to the extent that the latter worked without being changed, and designed new interfaces to handle the added semantics. The _read_ and _write_ system calls were used for byte-stream type connections, but six new system calls were added to allow sending and receiving addressed messages such as network datagrams. The system calls for writing messages include _send_, _sendto_, and _sendmsg_. The system calls for reading messages include _recv_, _recvfrom_, and _recvmsg_. In retrospect, the first two in each class are special cases of the others; _recvfrom_ and _sendto_ probably should have been added as library interfaces to _recvmsg_ and _sendmsg_, respectively. ==== Scatter/Gather I/O In addition to the traditional _read_ and _write_ system calls, 4.2BSD introduced the ability to do scatter/gather I/O. Scatter input uses the _readv_ system call to allow a single read to be placed in several different buffers. Conversely, the _writev_ system call allows several different buffers to be written in a single atomic write. Instead of passing a single buffer and length parameter, as is done with _read_ and _write_, the process passes in a pointer to an array of buffers and lengths, along with a count describing the size of the array. This facility allows buffers in different parts of a process address space to be written atomically, without the need to copy them to a single contiguous buffer. Atomic writes are necessary in the case where the underlying abstraction is record based, such as tape drives that output a tape block on each write request. It is also convenient to be able to read a single request into several different buffers (such as a record header into one place and the data into another). Although an application can simulate the ability to scatter data by reading the data into a large buffer and then copying the pieces to their intended destinations, the cost of memory-to-memory copying in such cases often would more than double the running time of the affected application. Just as _send_ and _recv_ could have been implemented as library interfaces to _sendto_ and _recvfrom_, it also would have been possible to simulate _read_ with _readv_ and _write_ with _writev_. However, _read_ and _write_ are used so much more frequently that the added cost of simulating them would not have been worthwhile. ==== Multiple Filesystem Support With the expansion of network computing, it became desirable to support both local and remote filesystems. To simplify the support of multiple filesystems, the developers added a new virtual node or _vnode_ interface to the kernel. The set of operations exported from the vnode interface appear much like the filesystem operations previously supported by the local filesystem. However, they may be supported by a wide range of filesystem types: * Local disk-based filesystems * Files imported using a variety of remote filesystem protocols * Read-only CD-ROM filesystems * Filesystems providing special-purpose interfaces -- for example, the `/proc` filesystem A few variants of 4.4BSD, such as FreeBSD, allow filesystems to be loaded dynamically when the filesystems are first referenced by the _mount_ system call. The vnode interface is described in Section 6.5; its ancillary support routines are described in Section 6.6; several of the special-purpose filesystems are described in Section 6.7. [[overview-filesystem]] === Filesystems A regular file is a linear array of bytes, and can be read and written starting at any byte in the file. The kernel distinguishes no record boundaries in regular files, although many programs recognize line-feed characters as distinguishing the ends of lines, and other programs may impose other structure. No system-related information about a file is kept in the file itself, but the filesystem stores a small amount of ownership, protection, and usage information with each file. A _filename_ component is a string of up to 255 characters. These filenames are stored in a type of file called a _directory_. The information in a directory about a file is called a _directory entry_ and includes, in addition to the filename, a pointer to the file itself. Directory entries may refer to other directories, as well as to plain files. A hierarchy of directories and files is thus formed, and is called a _filesystem_; .A small filesystem [[fig-small-fs]] image:fig2.png[A small filesystem] a small one is shown in <>. Directories may contain subdirectories, and there is no inherent limitation to the depth with which directory nesting may occur. To protect the consistency of the filesystem, the kernel does not permit processes to write directly into directories. A filesystem may include not only plain files and directories, but also references to other objects, such as devices and sockets. The filesystem forms a tree, the beginning of which is the _root directory_, sometimes referred to by the name _slash_, spelled with a single solidus character (/). The root directory contains files; in our example in Fig 2.2, it contains `vmunix`, a copy of the kernel-executable object file. It also contains directories; in this example, it contains the `usr` directory. Within the `usr` directory is the `bin` directory, which mostly contains executable object code of programs, such as the files `ls` and `vi`. A process identifies a file by specifying that file's _pathname_, which is a string composed of zero or more filenames separated by slash (/) characters. The kernel associates two directories with each process for use in interpreting pathnames. A process's _root directory_ is the topmost point in the filesystem that the process can access; it is ordinarily set to the root directory of the entire filesystem. A pathname beginning with a slash is called an _absolute pathname_, and is interpreted by the kernel starting with the process's root directory. A pathname that does not begin with a slash is called a _relative pathname_, and is interpreted relative to the _current working directory_ of the process. (This directory also is known by the shorter names _current directory_ or _working directory_.) The current directory itself may be referred to directly by the name _dot_, spelled with a single period (`.`) The filename _dot-dot_ (`..`) refers to a directory's parent directory. The root directory is its own parent. A process may set its root directory with the _chroot_ system call, and its current directory with the _chdir_ system call. Any process may do _chdir_ at any time, but _chroot_ is permitted only a process with superuser privileges. _Chroot_ is normally used to set up restricted access to the system. Using the filesystem shown in Fig. 2.2, if a process has the root of the filesystem as its root directory, and has `/usr` as its current directory, it can refer to the file `vi` either from the root with the absolute pathname `/usr/bin/vi`, or from its current directory with the relative pathname `bin/vi`. System utilities and databases are kept in certain well-known directories. Part of the well-defined hierarchy includes a directory that contains the _home directory_ for each user -- for example, `/usr/staff/mckusick` and `/usr/staff/karels` in Fig. 2.2. When users log in, the current working directory of their shell is set to the home directory. Within their home directories, users can create directories as easily as they can regular files. Thus, a user can build arbitrarily complex subhierarchies. The user usually knows of only one filesystem, but the system may know that this one virtual filesystem is really composed of several physical filesystems, each on a different device. A physical filesystem may not span multiple hardware devices. Since most physical disk devices are divided into several logical devices, there may be more than one filesystem per physical device, but there will be no more than one per logical device. One filesystem -- the filesystem that anchors all absolute pathnames -- is called the _root filesystem_, and is always available. Others may be mounted; that is, they may be integrated into the directory hierarchy of the root filesystem. References to a directory that has a filesystem mounted on it are converted transparently by the kernel into references to the root directory of the mounted filesystem. The _link_ system call takes the name of an existing file and another name to create for that file. After a successful _link_, the file can be accessed by either filename. A filename can be removed with the _unlink_ system call. When the final name for a file is removed (and the final process that has the file open closes it), the file is deleted. Files are organized hierarchically in _directories_. A directory is a type of file, but, in contrast to regular files, a directory has a structure imposed on it by the system. A process can read a directory as it would an ordinary file, but only the kernel is permitted to modify a directory. Directories are created by the _mkdir_ system call and are removed by the _rmdir_ system call. Before 4.2BSD, the _mkdir_ and _rmdir_ system calls were implemented by a series of _link_ and _unlink_ system calls being done. There were three reasons for adding systems calls explicitly to create and delete directories: [arabic] . The operation could be made atomic. If the system crashed, the directory would not be left half-constructed, as could happen when a series of link operations were used. . When a networked filesystem is being run, the creation and deletion of files and directories need to be specified atomically so that they can be serialized. . When supporting non-UNIX filesystems, such as an MS-DOS filesystem, on another partition of the disk, the other filesystem may not support link operations. Although other filesystems might support the concept of directories, they probably would not create and delete the directories with links, as the UNIX filesystem does. Consequently, they could create and delete directories only if explicit directory create and delete requests were presented. The _chown_ system call sets the owner and group of a file, and _chmod_ changes protection attributes. _Stat_ applied to a filename can be used to read back such properties of a file. The _fchown_, _fchmod_, and _fstat_ system calls are applied to a descriptor, instead of to a filename, to do the same set of operations. The _rename_ system call can be used to give a file a new name in the filesystem, replacing one of the file's old names. Like the directory-creation and directory-deletion operations, the _rename_ system call was added to 4.2BSD to provide atomicity to name changes in the local filesystem. Later, it proved useful explicitly to export renaming operations to foreign filesystems and over the network. The _truncate_ system call was added to 4.2BSD to allow files to be shortened to an arbitrary offset. The call was added primarily in support of the Fortran run-time library, which has the semantics such that the end of a random-access file is set to be wherever the program most recently accessed that file. Without the _truncate_ system call, the only way to shorten a file was to copy the part that was desired to a new file, to delete the old file, then to rename the copy to the original name. As well as this algorithm being slow, the library could potentially fail on a full filesystem. Once the filesystem had the ability to shorten files, the kernel took advantage of that ability to shorten large empty directories. The advantage of shortening empty directories is that it reduces the time spent in the kernel searching them when names are being created or deleted. Newly created files are assigned the user identifier of the process that created them and the group identifier of the directory in which they were created. A three-level access-control mechanism is provided for the protection of files. These three levels specify the accessibility of a file to [arabic] . The user who owns the file . The group that owns the file . Everyone else Each level of access has separate indicators for read permission, write permission, and execute permission. Files are created with zero length, and may grow when they are written. While a file is open, the system maintains a pointer into the file indicating the current location in the file associated with the descriptor. This pointer can be moved about in the file in a random-access fashion. Processes sharing a file descriptor through a _fork_ or _dup_ system call share the current location pointer. Descriptors created by separate _open_ system calls have separate current location pointers. Files may have _holes_ in them. Holes are void areas in the linear extent of the file where data have never been written. A process can create these holes by positioning the pointer past the current end-of-file and writing. When read, holes are treated by the system as zero-valued bytes. Earlier UNIX systems had a limit of 14 characters per filename component. This limitation was often a problem. For example, in addition to the natural desire of users to give files long descriptive names, a common way of forming filenames is as `basename.extension`, where the extension (indicating the kind of file, such as `.c` for C source or `.o` for intermediate binary object) is one to three characters, leaving 10 to 12 characters for the basename. Source-code-control systems and editors usually take up another two characters, either as a prefix or a suffix, for their purposes, leaving eight to 10 characters. It is easy to use 10 or 12 characters in a single English word as a basename (e.g., `multiplexer`). It is possible to keep within these limits, but it is inconvenient or even dangerous, because other UNIX systems accept strings longer than the limit when creating files, but then _truncate_ to the limit. A C language source file named `multiplexer.c` (already 13 characters) might have a source-code-control file with `s.` prepended, producing a filename `s.multiplexer` that is indistinguishable from the source-code-control file for `multiplexer.ms`, a file containing `troff` source for documentation for the C program. The contents of the two original files could easily get confused with no warning from the source-code-control system. Careful coding can detect this problem, but the long filenames first introduced in 4.2BSD practically eliminate it. [[overview-filestore]] === Filestores The operations defined for local filesystems are divided into two parts. Common to all local filesystems are hierarchical naming, locking, quotas, attribute management, and protection. These features are independent of how the data will be stored. 4.4BSD has a single implementation to provide these semantics. The other part of the local filesystem is the organization and management of the data on the storage media. Laying out the contents of files on the storage media is the responsibility of the filestore. 4.4BSD supports three different filestore layouts: * The traditional Berkeley Fast Filesystem * The log-structured filesystem, based on the Sprite operating-system design <> * A memory-based filesystem Although the organizations of these filestores are completely different, these differences are indistinguishable to the processes using the filestores. The Fast Filesystem organizes data into cylinder groups. Files that are likely to be accessed together, based on their locations in the filesystem hierarchy, are stored in the same cylinder group. Files that are not expected to accessed together are moved into different cylinder groups. Thus, files written at the same time may be placed far apart on the disk. The log-structured filesystem organizes data as a log. All data being written at any point in time are gathered together, and are written at the same disk location. Data are never overwritten; instead, a new copy of the file is written that replaces the old one. The old files are reclaimed by a garbage-collection process that runs when the filesystem becomes full and additional free space is needed. The memory-based filesystem is designed to store data in virtual memory. It is used for filesystems that need to support fast but temporary data, such as `/tmp`. The goal of the memory-based filesystem is to keep the storage packed as compactly as possible to minimize the usage of virtual-memory resources. [[overview-nfs]] === Network Filesystem Initially, networking was used to transfer data from one machine to another. Later, it evolved to allowing users to log in remotely to another machine. The next logical step was to bring the data to the user, instead of having the user go to the data -- and network filesystems were born. Users working locally do not experience the network delays on each keystroke, so they have a more responsive environment. Bringing the filesystem to a local machine was among the first of the major client-server applications. The _server_ is the remote machine that exports one or more of its filesystems. The _client_ is the local machine that imports those filesystems. From the local client's point of view, a remotely mounted filesystem appears in the file-tree name space just like any other locally mounted filesystem. Local clients can change into directories on the remote filesystem, and can read, write, and execute binaries within that remote filesystem identically to the way that they can do these operations on a local filesystem. When the local client does an operation on a remote filesystem, the request is packaged and is sent to the server. The server does the requested operation and returns either the requested information or an error indicating why the request was denied. To get reasonable performance, the client must cache frequently accessed data. The complexity of remote filesystems lies in maintaining cache consistency between the server and its many clients. Although many remote-filesystem protocols have been developed over the years, the most pervasive one in use among UNIX systems is the Network Filesystem (NFS), whose protocol and most widely used implementation were done by Sun Microsystems. The 4.4BSD kernel supports the NFS protocol, although the implementation was done independently from the protocol specification <>. The NFS protocol is described in Chapter 9. [[overview-terminal]] === Terminals Terminals support the standard system I/O operations, as well as a collection of terminal-specific operations to control input-character editing and output delays. At the lowest level are the terminal device drivers that control the hardware terminal ports. Terminal input is handled according to the underlying communication characteristics, such as baud rate, and according to a set of software-controllable parameters, such as parity checking. Layered above the terminal device drivers are line disciplines that provide various degrees of character processing. The default line discipline is selected when a port is being used for an interactive login. The line discipline is run in _canonical mode_; input is processed to provide standard line-oriented editing functions, and input is presented to a process on a line-by-line basis. Screen editors and programs that communicate with other computers generally run in _noncanonical mode_ (also commonly referred to as _raw mode_ or _character-at-a-time mode_). In this mode, input is passed through to the reading process immediately and without interpretation. All special-character input processing is disabled, no erase or other line editing processing is done, and all characters are passed to the program that is reading from the terminal. It is possible to configure the terminal in thousands of combinations between these two extremes. For example, a screen editor that wanted to receive user interrupts asynchronously might enable the special characters that generate signals and enable output flow control, but otherwise run in noncanonical mode; all other characters would be passed through to the process uninterpreted. On output, the terminal handler provides simple formatting services, including * Converting the line-feed character to the two-character carriage-return-line-feed sequence * Inserting delays after certain standard control characters * Expanding tabs * Displaying echoed nongraphic ASCII characters as a two-character sequence of the form `^C` (i.e., the ASCII caret character followed by the ASCII character that is the character's value offset from the ASCII `@` character). Each of these formatting services can be disabled individually by a process through control requests. [[overview-ipc]] === Interprocess Communication Interprocess communication in 4.4BSD is organized in _communication domains_. Domains currently supported include the _local domain_, for communication between processes executing on the same machine; the _internet domain_, for communication between processes using the TCP/IP protocol suite (perhaps within the Internet); the ISO/OSI protocol family for communication between sites required to run them; and the _XNS domain_, for communication between processes using the XEROX Network Systems (XNS) protocols. Within a domain, communication takes place between communication endpoints known as _sockets_. As mentioned in Section 2.6, the _socket_ system call creates a socket and returns a descriptor; other IPC system calls are described in Chapter 11. Each socket has a type that defines its communications semantics; these semantics include properties such as reliability, ordering, and prevention of duplication of messages. Each socket has associated with it a _communication protocol_. This protocol provides the semantics required by the socket according to the latter's type. Applications may request a specific protocol when creating a socket, or may allow the system to select a protocol that is appropriate for the type of socket being created. Sockets may have addresses bound to them. The form and meaning of socket addresses are dependent on the communication domain in which the socket is created. Binding a name to a socket in the local domain causes a file to be created in the filesystem. Normal data transmitted and received through sockets are untyped. Data-representation issues are the responsibility of libraries built on top of the interprocess-communication facilities. In addition to transporting normal data, communication domains may support the transmission and reception of specially typed data, termed _access rights_. The local domain, for example, uses this facility to pass descriptors between processes. Networking implementations on UNIX before 4.2BSD usually worked by overloading the character-device interfaces. One goal of the socket interface was for naive programs to be able to work without change on stream-style connections. Such programs can work only if the _read_ and _write_ systems calls are unchanged. Consequently, the original interfaces were left intact, and were made to work on stream-type sockets. A new interface was added for more complicated sockets, such as those used to send datagrams, with which a destination address must be presented with each _send_ call. Another benefit is that the new interface is highly portable. Shortly after a test release was available from Berkeley, the socket interface had been ported to System III by a UNIX vendor (although AT&T did not support the socket interface until the release of System V Release 4, deciding instead to use the Eighth Edition stream mechanism). The socket interface was also ported to run in many Ethernet boards by vendors, such as Excelan and Interlan, that were selling into the PC market, where the machines were too small to run networking in the main processor. More recently, the socket interface was used as the basis for Microsoft's Winsock networking interface for Windows. [[overview-network-communication]] === Network Communication Some of the communication domains supported by the _socket_ IPC mechanism provide access to network protocols. These protocols are implemented as a separate software layer logically below the socket software in the kernel. The kernel provides many ancillary services, such as buffer management, message routing, standardized interfaces to the protocols, and interfaces to the network interface drivers for the use of the various network protocols. At the time that 4.2BSD was being implemented, there were many networking protocols in use or under development, each with its own strengths and weaknesses. There was no clearly superior protocol or protocol suite. By supporting multiple protocols, 4.2BSD could provide interoperability and resource sharing among the diverse set of machines that was available in the Berkeley environment. Multiple-protocol support also provides for future changes. Today's protocols designed for 10- to 100-Mbit-per-second Ethernets are likely to be inadequate for tomorrow's 1- to 10-Gbit-per-second fiber-optic networks. Consequently, the network-communication layer is designed to support multiple protocols. New protocols are added to the kernel without the support for older protocols being affected. Older applications can continue to operate using the old protocol over the same physical network as is used by newer applications running with a newer network protocol. [[overview-network-implementation]] === Network Implementation The first protocol suite implemented in 4.2BSD was DARPA's Transmission Control Protocol/Internet Protocol (TCP/IP). The CSRG chose TCP/IP as the first network to incorporate into the socket IPC framework, because a 4.1BSD-based implementation was publicly available from a DARPA-sponsored project at Bolt, Beranek, and Newman (BBN). That was an influential choice: The 4.2BSD implementation is the main reason for the extremely widespread use of this protocol suite. Later performance and capability improvements to the TCP/IP implementation have also been widely adopted. The TCP/IP implementation is described in detail in Chapter 13. The release of 4.3BSD added the Xerox Network Systems (XNS) protocol suite, partly building on work done at the University of Maryland and at Cornell University. This suite was needed to connect isolated machines that could not communicate using TCP/IP. The release of 4.4BSD added the ISO protocol suite because of the latter's increasing visibility both within and outside the United States. Because of the somewhat different semantics defined for the ISO protocols, some minor changes were required in the socket interface to accommodate these semantics. The changes were made such that they were invisible to clients of other existing protocols. The ISO protocols also required extensive addition to the two-level routing tables provided by the kernel in 4.3BSD. The greatly expanded routing capabilities of 4.4BSD include arbitrary levels of routing with variable-length addresses and network masks. [[overview-operation]] === System Operation Bootstrapping mechanisms are used to start the system running. First, the 4.4BSD kernel must be loaded into the main memory of the processor. Once loaded, it must go through an initialization phase to set the hardware into a known state. Next, the kernel must do autoconfiguration, a process that finds and configures the peripherals that are attached to the processor. The system begins running in single-user mode while a start-up script does disk checks and starts the accounting and quota checking. Finally, the start-up script starts the general system services and brings up the system to full multiuser operation. During multiuser operation, processes wait for login requests on the terminal lines and network ports that have been configured for user access. When a login request is detected, a login process is spawned and user validation is done. When the login validation is successful, a login shell is created from which the user can run additional processes. :sectnums!: [bibliography] [[references]] == References [[biblio-accetta]] Accetta et al, 1986 Mach: A New Kernel Foundation for UNIX Development" M.Accetta R.Baron W.Bolosky D.Golub R.Rashid A.Tevanian M.Young 93-113 USENIX Association Conference Proceedings USENIX Association June 1986 [[biblio-cheriton]] Cheriton, 1988 The V Distributed System D. R.Cheriton 314-333 Comm ACM, 31, 3 March 1988 [[biblio-ewens]] Ewens et al, 1985 Tunis: A Distributed Multiprocessor Operating System P.Ewens D. R.Blythe M.Funkenhauser R. C.Holt 247-254 USENIX Assocation Conference Proceedings USENIX Association June 1985 [[biblio-gingell]] Gingell et al, 1987 Virtual Memory Architecture in SunOS R.Gingell J.Moran W.Shannon 81-94 USENIX Association Conference Proceedings USENIX Association June 1987 [[biblio-kernighan]] Kernighan & Pike, 1984 The UNIX Programming Environment B. W.Kernighan R.Pike Prentice-Hall Englewood Cliffs NJ 1984 [[biblio-macklem]] Macklem, 1994 The 4.4BSD NFS Implementation R.Macklem 6:1-14 4.4BSD System Manager's Manual O'Reilly & Associates, Inc. Sebastopol CA 1994 [[biblio-mckusick-2]] McKusick & Karels, 1988 Design of a General Purpose Memory Allocator for the 4.3BSD UNIX Kernel M. K.McKusick M. J.Karels 295-304 USENIX Assocation Conference Proceedings USENIX Assocation June 1998 [[biblio-mckusick-1]] McKusick et al, 1994 Berkeley Software Architecture Manual, 4.4BSD Edition M. K.McKusick M. J.Karels S. J.Leffler W. N.Joy R. S.Faber 5:1-42 4.4BSD Programmer's Supplementary Documents O'Reilly & Associates, Inc. Sebastopol CA 1994 [[biblio-ritchie]] Ritchie, 1988 Early Kernel Design private communication D. M.Ritchie March 1988 [[biblio-rosenblum]] Rosenblum & Ousterhout, 1992 The Design and Implementation of a Log-Structured File System M.Rosenblum K.Ousterhout 26-52 ACM Transactions on Computer Systems, 10, 1 Association for Computing Machinery February 1992 [[biblio-rozier]] Rozier et al, 1988 Chorus Distributed Operating Systems M.Rozier V.Abrossimov F.Armand I.Boule M.Gien M.Guillemont F.Herrmann C.Kaiser S.Langlois P.Leonard W.Neuhauser 305-370 USENIX Computing Systems, 1, 4 Fall 1988 [[biblio-tevanian]] Tevanian, 1987 Architecture-Independent Virtual Memory Management for Parallel and Distributed Environments: The Mach Approach Technical Report CMU-CS-88-106, A.Tevanian Department of Computer Science, Carnegie-Mellon University Pittsburgh PA December 1987 diff --git a/documentation/content/en/books/dev-model/_index.adoc b/documentation/content/en/books/dev-model/_index.adoc index 8e0a1b8226..3e3371ca07 100644 --- a/documentation/content/en/books/dev-model/_index.adoc +++ b/documentation/content/en/books/dev-model/_index.adoc @@ -1,1169 +1,1170 @@ --- title: A project model for the FreeBSD Project authors: - author: Niklas Saers copyright: Copyright © 2002-2005 Niklas Saers description: A formal study of the organization of the FreeBSD project trademarks: ["freebsd", "ibm", "ieee", "adobe", "intel", "linux", "microsoft", "opengroup", "sun", "netbsd", "general"] bookOrder: 45 tags: ["model", "project model", "FreeBSD"] +layout: single --- //// Copyright (c) 2002-2005 Niklas Saers All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. //// = A project model for the FreeBSD Project :doctype: book :toc: macro :toclevels: 2 :icons: font :sectnums: :sectnumlevels: 6 :partnums: :source-highlighter: rouge :experimental: :images-path: books/dev-model/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../images/{images-path} include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] include::../../../../../shared/asciidoctor.adoc[] endif::[] ''' toc::[] [[foreword]] [.abstract-title] Foreword Up until now, the FreeBSD project has released a number of described techniques to do different parts of work. However, a project model summarising how the project is structured is needed because of the increasing amount of project members. footnote:[This goes hand-in-hand with Brooks' law that adding another person to a late project will make it later since it will increase the communication needs . A project model is a tool to reduce the communication needs.] This paper will provide such a project model and is donated to the FreeBSD Documentation project where it can evolve together with the project so that it can at any point in time reflect the way the project works. It is based on [<>]. I would like to thank the following people for taking the time to explain things that were unclear to me and for proofreading the document. * Andrey A. Chernov mailto:ache@freebsd.org[ache@freebsd.org] * Bruce A. Mah mailto:bmah@freebsd.org[bmah@freebsd.org] * Dag-Erling Smørgrav mailto:des@freebsd.org[des@freebsd.org] * Giorgos Keramidas mailto:keramida@freebsd.org[keramida@freebsd.org] * Ingvil Hovig mailto:ingvil.hovig@skatteetaten.no[ingvil.hovig@skatteetaten.no] * Jesper Holck mailto:jeh.inf@cbs.dk[jeh.inf@cbs.dk] * John Baldwin mailto:jhb@freebsd.org[jhb@freebsd.org] * John Polstra mailto:jdp@freebsd.org[jdp@freebsd.org] * Kirk McKusick mailto:mckusick@freebsd.org[mckusick@freebsd.org] * Mark Linimon mailto:linimon@freebsd.org[linimon@freebsd.org] * Marleen Devos * Niels Jørgenssen mailto:nielsj@ruc.dk[nielsj@ruc.dk] * Nik Clayton mailto:nik@freebsd.org[nik@freebsd.org] * Poul-Henning Kamp mailto:phk@freebsd.org[phk@freebsd.org] * Simon L. Nielsen mailto:simon@freebsd.org[simon@freebsd.org] [[overview]] == Overview A project model is a means to reduce the communications overhead in a project. As shown by [<>], increasing the number of project participants increases the communication in the project exponentionally. FreeBSD has during the past few years increased both its mass of active users and committers, and the communication in the project has risen accordingly. This project model will serve to reduce this overhead by providing an up-to-date description of the project. During the Core elections in 2002, Mark Murray stated "I am opposed to a long rule-book, as that satisfies lawyer-tendencies, and is counter to the technocentricity that the project so badly needs." [<>]. This project model is not meant to be a tool to justify creating impositions for developers, but as a tool to facilitate coordination. It is meant as a description of the project, with an overview of how the different processes are executed. It is an introduction to how the FreeBSD project works. The FreeBSD project model will be described as of July 1st, 2004. It is based on the Niels Jørgensen's paper [<>], FreeBSD's official documents, discussions on FreeBSD mailing lists and interviews with developers. After providing definitions of terms used, this document will outline the organisational structure (including role descriptions and communication lines), discuss the methodology model and after presenting the tools used for process control, it will present the defined processes. Finally it will outline major sub-projects of the FreeBSD project. [<>] Section 1.2 and 1.3 give the vision and the architectural guidelines for the project. The vision is "To produce the best UNIX-like operating system package possible, with due respect to the original software tools ideology as well as usability, performance and stability." The architectural guidelines help determine whether a problem that someone wants to be solved is within the scope of the project [[definitions]] == Definitions [[ref-activity]] === Activity An "activity" is an element of work performed during the course of a project [<>]. It has an output and leads towards an outcome. Such an output can either be an input to another activity or a part of the process' delivery. [[def-process]] === Process A "process" is a series of activities that lead towards a particular outcome. A process can consist of one or more sub-processes. An example of a process is software design. [[ref-hat]] === Hat A "hat" is synonymous with role. A hat has certain responsibilities in a process and for the process outcome. The hat executes activities. It is well defined what issues the hat should be contacted about by the project members and people outside the project. [[ref-outcome]] === Outcome An "outcome" is the final output of the process. This is synonymous with deliverable, that is defined as "any measurable, tangible, verifiable outcome, result or item that must be produced to complete a project or part of a project. Often used more narrowly in reference to an external deliverable, which is a deliverable that is subject to approval by the project sponsor or customer" by [<>]. Examples of outcomes are a piece of software, a decision made or a report written. [[ref-freebsd]] === FreeBSD When saying "FreeBSD" we will mean the BSD derivative UNIX-like operating system FreeBSD, whereas when saying "the FreeBSD Project" we will mean the project organisation. [[model-orgstruct]] == Organisational structure While no-one takes ownership of FreeBSD, the FreeBSD organisation is divided into core, committers and contributors and is part of the FreeBSD community that lives around it. The FreeBSD Project's structure (in order of descending authority) [.informaltable] [cols="1,1", options="header"] |=== | Group | Number of people |Core members |9 |Committers |318 |Contributors |~3000 |=== Number of committers has been determined by going through CVS logs from January 1st, 2004 to December 31st, 2004 and contributors by going through the list of contributions and problem reports. The main resource in the FreeBSD community is its developers: the committers and contributors. It is with their contributions that the project can move forward. Regular developers are referred to as contributors. As of January 1st, 2003, there are an estimated 5500 contributors on the project. Committers are developers with the privilege of being able to commit changes. These are usually the most active developers who are willing to spend their time not only integrating their own code but integrating code submitted by the developers who do not have this privilege. They are also the developers who elect the core team, and they have access to closed discussions. The project can be grouped into four distinct separate parts, and most developers will focus their involvement in one part of FreeBSD. The four parts are kernel development, userland development, ports and documentation. When referring to the base system, both kernel and userland is meant. This split changes our table to look like this: The FreeBSD Project's structure with committers in categories [.informaltable] [cols="1,1,1", options="header"] |=== | Group | Category | Number of people |Core members | |9 |Committers |Base |164 | |Docs |45 | |Ports |166 | |Total |374 |Contributors | |~3000 |=== Number of committers per area has been determined by going through CVS logs from January 1st, 2004 to December 31st, 2004. Note that many committers work in multiple areas, making the total number higher than the real number of committers. The total number of active unique committers on June 2022 was 317. Committers fall into three groups: committers who are only concerned with one area of the project (for instance file systems), committers who are involved only with one sub-project, and committers who commit to different parts of the code, including sub-projects. Because some committers work on different parts, the total number in the committers section of the table is higher than in the above table. The kernel is the main building block of FreeBSD. While the userland applications are protected against faults in other userland applications, the entire system is vulnerable to errors in the kernel. This, combined with the vast amount of dependencies in the kernel and that it is not easy to see all the consequences of a kernel change, demands developers with a relative full understanding of the kernel. Multiple development efforts in the kernel also require a closer coordination than userland applications do. The core utilities, known as userland, provide the interface that identifies FreeBSD, both user interface, shared libraries and external interfaces to connecting clients. Currently, 162 people are involved in userland development and maintenance, many being maintainers for their own part of the code. Maintainership will be discussed in the <> section. Documentation is handled by <> and includes all documents surrounding the FreeBSD project, including the web pages. There were during 2004 101 people making commits to the FreeBSD Documentation Project. Ports is the collection of meta-data that is needed to make software packages build correctly on FreeBSD. An example of a port is the port for the web-browser Mozilla. It contains information about where to fetch the source, what patches to apply and how, and how the package should be installed on the system. This allows automated tools to fetch, build and install the package. As of this writing, there are more than 12600 ports available. footnote:[Statistics are generated by counting the number of entries in the file fetched by portsdb by April 1st, 2005. portsdb is a part of the port sysutils/portupgrade.] , ranging from web servers to games, programming languages and most of the application types that are in use on modern computers. Ports will be discussed further in the section <>. [[methodology-model]] == Methodology model [[development-model]] === Development model There is no defined model for how people write code in FreeBSD. However, Niels Jørgenssen has suggested a model of how written code is integrated into the project. Jørgenssen's model for change integration [.informaltable] [cols="1,1,1", options="header"] |=== | Stage | Next if successful | Next if unsuccessful |code |review | |review |pre-commit test |code |pre-commit test |development release |code |development release |parallel debugging |code |parallel debugging |production release |code |production release | |code |=== The "development release" is the FreeBSD-CURRENT ("-CURRENT") branch and the "production release" is the FreeBSD-STABLE branch ("-STABLE") [<>]. This is a model for one change, and shows that after coding, developers seek community review and try integrating it with their own systems. After integrating the change into the development release, called FreeBSD-CURRENT, it is tested by many users and developers in the FreeBSD community. After it has gone through enough testing, it is merged into the production release, called FreeBSD-STABLE. Unless each stage is finished successfully, the developer needs to go back and make modifications in the code and restart the process. To integrate a change with either -CURRENT or -STABLE is called making a commit. Jørgensen found that most FreeBSD developers work individually, meaning that this model is used in parallel by many developers on the different ongoing development efforts. A developer can also be working on multiple changes, so that while they are waiting for review or people to test one or more of their changes, they may be writing another change. As each commit represents an increment, this is a massively incremental model. The commits are in fact so frequent that during one year footnote:[The period from January 1st, 2004 to December 31st, 2004 was examined to find this number.] , 85427 commits were made, making a daily average of 233 commits. Within the "code" bracket in Jørgensen's model, each programmer has their own working style and follows their own development models. The bracket could very well have been called "development" as it includes requirements gathering and analysis, system and detailed design, implementation and verification. However, the only output from these stages is the source code or system documentation. From a stepwise model's perspective (such as the waterfall model), the other brackets can be seen as further verification and system integration. This system integration is also important to see if a change is accepted by the community. Up until the code is committed, the developer is free to choose how much to communicate about it to the rest of the project. In order for -CURRENT to work as a buffer (so that bright ideas that had some undiscovered drawbacks can be backed out) the minimum time a commit should be in -CURRENT before merging it to -STABLE is 3 days. Such a merge is referred to as an MFC (Merge From Current). It is important to notice the word "change". Most commits do not contain radical new features, but are maintenance updates. The only exceptions from this model are security fixes and changes to features that are deprecated in the -CURRENT branch. In these cases, changes can be committed directly to the -STABLE branch. In addition to many people working on the project, there are many related projects to the FreeBSD Project. These are either projects developing brand new features, sub-projects or projects whose outcome is incorporated into FreeBSD footnote:[For instance, the development of the Bluetooth stack started as a sub-project until it was deemed stable enough to be merged into the -CURRENT branch. Now it is a part of the core FreeBSD system.]. These projects fit into the FreeBSD Project just like regular development efforts: they produce code that is integrated with the FreeBSD Project. However, some of them (like Ports and Documentation) have the privilege of being applicable to both branches or commit directly to both -CURRENT and -STABLE. There is no standards to how design should be done, nor is design collected in a centralised repository. The main design is that of 4.4BSD. footnote:[According to Kirk McKusick, after 20 years of developing UNIX operating systems, the interfaces are for the most part figured out. There is therefore no need for much design. However, new applications of the system and new hardware leads to some implementations being more beneficial than those that used to be preferred. One example is the introduction of web browsing that made the normal TCP/IP connection a short burst of data rather than a steady stream over a longer period of time.] As design is a part of the "Code" bracket in Jørgenssen's model, it is up to every developer or sub-project how this should be done. Even if the design should be stored in a central repository, the output from the design stages would be of limited use as the differences of methodologies would make them poorly if at all interoperable. For the overall design of the project, the project relies on the sub-projects to negotiate fit interfaces between each other rather than to dictate interfacing. [[release-branches]] === Release branches The releases of FreeBSD are best illustrated by a tree with many branches where each major branch represents a major version. Minor versions are represented by branches of the major branches. In the following release tree, arrows that follow one-another in a particular direction represent a branch. Boxes with full lines and diamonds represent official releases. Boxes with dotted lines represent the development branch at that time. Security branches are represented by ovals. Diamonds differ from boxes in that they represent a fork, meaning a place where a branch splits into two branches where one of the branches becomes a sub-branch. For example, at 4.0-RELEASE the 4.0-CURRENT branch split into 4-STABLE and 5.0-CURRENT. At 4.5-RELEASE, the branch forked off a security branch called RELENG_4_5. .The FreeBSD release tree image::branches.png[Refer to table below for a screen-reader friendly version.] [.informaltable] [cols="1,1,1", options="header"] |=== | Major release | Forked from | Following minor releases |... | | |3.0 Current (development branch) | |Releng 3 branches: 3.0 Release to 3.5 Release, leading to 3.5.1 Release and the subsequent 3 Stable branch |4.0 Current (development branch) |3.1 Release |Releng 4 branches: 4.1 Release to 4.6 Release (and 4.6.2 Release), then 4.7 Release to 4.11 Release (all starting at 4.3 Release also leading to a Releng_4_n branch), and the subsequent 4 Release branch |5.0 Current (development branch) |4.0 Release |Releng 5 branches: 5.0 Release to 5.4 Release (all except 5.0 and 5.3 also leading to a Releng_5_n branch), and the subsequent 5 Release branch |6.0 Current (development branch) |5.3 Release | |... | | |=== The latest -CURRENT version is always referred to as -CURRENT, while the latest -STABLE release is always referred to as -STABLE. In this figure, -STABLE refers to 4-STABLE while -CURRENT refers to 5.0-CURRENT following 5.0-RELEASE. [<>] A "major release" is always made from the -CURRENT branch. However, the -CURRENT branch does not need to fork at that point in time, but can focus on stabilising. An example of this is that following 3.0-RELEASE, 3.1-RELEASE was also a continuation of the -CURRENT-branch, and -CURRENT did not become a true development branch until this version was released and the 3-STABLE branch was forked. When -CURRENT returns to becoming a development branch, it can only be followed by a major release. 5-STABLE is predicted to be forked off 5.0-CURRENT at around 5.3-RELEASE. It is not until 5-STABLE is forked that the development branch will be branded 6.0-CURRENT. A "minor release" is made from the -CURRENT branch following a major release, or from the -STABLE branch. Following and including, 4.3-RELEASEfootnote:[The first release this actually happened for was 4.5-RELEASE, but security branches were at the same time created for 4.3-RELEASE and 4.4-RELEASE.], when a minor release has been made, it becomes a "security branch". This is meant for organisations that do not want to follow the -STABLE branch and the potential new/changed features it offers, but instead require an absolutely stable environment, only updating to implement security updates. footnote:[There is a terminology overlap with respect to the word "stable", which leads to some confusion. The -STABLE branch is still a development branch, whose goal is to be useful for most people. If it is never acceptable for a system to get changes that are not announced at the time it is deployed, that system should run a security branch.] Each update to a security branch is called a "patchlevel". For every security enhancement that is done, the patchlevel number is increased, making it easy for people tracking the branch to see what security enhancements they have implemented. In cases where there have been especially serious security flaws, an entire new release can be made from a security branch. An example of this is 4.6.2-RELEASE. [[model-summary]] === Model summary To summarise, the development model of FreeBSD can be seen as the following tree: .The overall development model image::freebsd-code-model.png[Refer to paragraphs below for a screen-reader friendly version.] The tree of the FreeBSD development with ongoing development efforts and continuous integration. The tree symbolises the release versions with major versions spawning new main branches and minor versions being versions of the main branch. The top branch is the -CURRENT branch where all new development is integrated, and the -STABLE branch is the branch directly below it. Below the -STABLE branch are old, unsupported versions. Clouds of development efforts hang over the project where developers use the development models they see fit. The product of their work is then integrated into -CURRENT where it undergoes parallel debugging and is finally merged from -CURRENT into -STABLE. Security fixes are merged from -STABLE to the security branches. Many committers have a special area of responsibility. These roles are called hats. These hats can be either project roles, such as public relations officer, or maintainer for a certain area of the code. Because this is a project where people give voluntarily of their spare time, people with assigned hats are not always available. They must therefore appoint a deputy that can perform the hat's role in their absence. The other option is to have the role held by a group. Many of these hats are not formalised. Formalised hats have a charter stating the exact purpose of the hat along with its privileges and responsibilities. The writing of such charters is a new part of the project, and has thus yet to be completed for all hats. These hat descriptions are not such a formalisation, rather a summary of the role with links to the charter where available and contact addresses. [[sect-hats]] == Hats [[general-hats]] === General Hats [[role-contributor]] ==== Contributor A Contributor contributes to the FreeBSD project either as a developer, as an author, by sending problem reports, or in other ways contributing to the progress of the project. A contributor has no special privileges in the FreeBSD project. [<>] [[role-committer]] ==== Committer A person who has the required privileges to add their code or documentation to the repository. A committer has made a commit within the past 12 months. [<>] An active committer is a committer who has made an average of one commit per month during that time. It is worth noting that there are no technical barriers to prevent someone, once having gained commit privileges to the main- or a sub-project, to make commits in parts of that project's source the committer did not specifically get permission to modify. However, when wanting to make modifications to parts a committer has not been involved in before, they should read the logs to see what has happened in this area before, and also read the MAINTAINERS file to see if the maintainer of this part has any special requests on how changes in the code should be made. [[role-core]] ==== Core Team The core team is elected by the committers from the pool of committers and serves as the board of directors of the FreeBSD project. It promotes active contributors to committers, assigns people to well-defined hats, and is the final arbiter of decisions involving which way the project should be heading. As of July 1st, 2004, core consisted of 9 members. Elections are held every two years. [[role-maintainer]] ==== Maintainership Maintainership means that the person is responsible for what is allowed to go into that area of the code and has the final say should disagreements over the code occur. This involves proactive work aimed at stimulating contributions and reactive work in reviewing commits. With the FreeBSD source comes the MAINTAINERS file that contains a one-line summary of how each maintainer would like contributions to be made. Having this notice and contact information enables developers to focus on the development effort rather than being stuck in a slow correspondence should the maintainer be unavailable for some time. If the maintainer is unavailable for an unreasonably long period of time, and other people do a significant amount of work, maintainership may be switched without the maintainer's approval. This is based on the stance that maintainership should be demonstrated, not declared. Maintainership of a particular piece of code is a hat that is not held as a group. [[official-hats]] === Official Hats The official hats in the FreeBSD Project are hats that are more or less formalised and mainly administrative roles. They have the authority and responsibility for their area. The following list shows the responsibility lines and gives a description of each hat, including who it is held by. [[role-doc-manager]] ==== Documentation project manager <> architect is responsible for defining and following up documentation goals for the committers in the Documentation project, which they supervise. Hat held by: The DocEng team mailto:doceng@FreeBSD.org[doceng@FreeBSD.org]. The https://www.freebsd.org/internal/doceng/[ DocEng Charter]. [[role-postmaster]] ==== Postmaster The Postmaster is responsible for mail being correctly delivered to the committers' email address. They are also responsible for ensuring that the mailing lists work and should take measures against possible disruptions of mail such as having troll-, spam- and virus-filters. Hat currently held by: the Postmaster Team mailto:postmaster@FreeBSD.org[postmaster@FreeBSD.org]. [[role-release-coordination]] ==== Release Coordination The responsibilities of the Release Engineering Team are * Setting, publishing and following a release schedule for official releases * Documenting and formalising release engineering procedures * Creation and maintenance of code branches * Coordinating with the Ports and Documentation teams to have an updated set of packages and documentation released with the new releases * Coordinating with the Security team so that pending releases are not affected by recently disclosed vulnerabilities. Further information about the development process is available in the <> section. [[role-releng]] Hat held by: the Release Engineering team mailto:re@FreeBSD.org[re@FreeBSD.org]. The https://www.freebsd.org/releng/charter/[ Release Engineering Charter]. [[role-pr-cr]] ==== Public Relations & Corporate Liaison The Public Relations & Corporate Liaison's responsibilities are: * Making press statements when happenings that are important to the FreeBSD Project happen. * Being the official contact person for corporations that are working close with the FreeBSD Project. * Take steps to promote FreeBSD within both the Open Source community and the corporate world. * Handle the "freebsd-advocacy" mailing list. This hat is currently not occupied. [[role-security-officer]] ==== Security Officer The Security Officer's main responsibility is to coordinate information exchange with others in the security community and in the FreeBSD project. The Security Officer is also responsible for taking action when security problems are reported and promoting proactive development behavior when it comes to security. Because of the fear that information about vulnerabilities may leak out to people with malicious intent before a patch is available, only the Security Officer, consisting of an officer, a deputy and two <> members, receive sensitive information about security issues. However, to create or implement a patch, the Security Officer has the Security Officer Team mailto:security-team@FreeBSD.org[security-team@FreeBSD.org] to help do the work. [[role-repo-manager]] ==== Source Repository Manager The Source Repository Manager is the only one who is allowed to directly modify the repository without using the <> tool. It is their responsibility to ensure that technical problems that arise in the repository are resolved quickly. The source repository manager has the authority to back out commits if this is necessary to resolve a Git technical problem. Hat held by: the Source Repository Manager mailto:clusteradm@FreeBSD.org[clusteradm@FreeBSD.org]. [[role-election-manager]] ==== Election Manager The Election Manager is responsible for the <> process. The manager is responsible for running and maintaining the election system, and is the final authority should minor unforeseen events happen in the election process. Major unforeseen events have to be discussed with the <> Hat held only during elections. [[role-webmaster]] ==== Web site Management The Web site Management hat is responsible for coordinating the rollout of updated web pages on mirrors around the world, for the overall structure of the primary web site and the system it is running upon. The management needs to coordinate the content with <> and acts as maintainer for the "www" tree. Hat held by: the FreeBSD Webmasters mailto:www@FreeBSD.org[www@FreeBSD.org]. [[role-ports-manager]] ==== Ports Manager The Ports Manager acts as a liaison between <> and the core project, and all requests from the project should go to the ports manager. Hat held by: the Ports Management Team mailto:portmgr@FreeBSD.org[portmgr@FreeBSD.org]. The https://www.freebsd.org/portmgr/charter/[Portmgr charter]. [[role-standards]] ==== Standards The Standards hat is responsible for ensuring that FreeBSD complies with the standards it is committed to , keeping up to date on the development of these standards and notifying FreeBSD developers of important changes that allows them to take a proactive role and decrease the time between a standards update and FreeBSD's compliancy. Hat currently held by: Garrett Wollman mailto:wollman@FreeBSD.org[wollman@FreeBSD.org]. [[role-core-secretary]] ==== Core Secretary The Core Secretary's main responsibility is to write drafts to and publish the final Core Reports. The secretary also keeps the core agenda, thus ensuring that no balls are dropped unresolved. Hat currently held by: {carlavilla}. [[role-bugmeister]] ==== Bugmeister The Bugmeister is responsible for ensuring that the maintenance database is in working order, that the entries are correctly categorised and that there are no invalid entries. They supervise bugbusters. Hat currently held by: the Bugmeister Team mailto:bugmeister@FreeBSD.org[bugmeister@FreeBSD.org]. [[role-donations]] ==== Donations Liaison Officer The task of the donations liaison officer is to match the developers with needs with people or organisations willing to make a donation. Hat held by: the Donations Liaison Office mailto:donations@FreeBSD.org[donations@FreeBSD.org]. The https://www.freebsd.org/donations/[ Donations Liaison Charter]. [[role-admin]] ==== Admin (Also called "FreeBSD Cluster Admin") The admin team consists of the people responsible for administrating the computers that the project relies on for its distributed work and communication to be synchronised. It consists mainly of those people who have physical access to the servers. Hat held by: the Admin team mailto:admin@FreeBSD.org[admin@FreeBSD.org]. [[proc-depend-hats]] === Process dependent hats [[role-problem-originator]] ==== Report originator The person originally responsible for filing a Problem Report. [[role-bugbuster]] ==== Bugbuster A person who will either find the right person to solve the problem, or close the PR if it is a duplicate or otherwise not an interesting one. [[role-mentor]] ==== Mentor A mentor is a committer who takes it upon them to introduce a new committer to the project, both in terms of ensuring the new committer's setup is valid, that the new committer knows the available tools required in their work, and that the new committer knows what is expected of them in terms of behavior. [[role-vendor]] ==== Vendor The person(s) or organisation whom external code comes from and whom patches are sent to. [[role-reviewer]] ==== Reviewers People on the mailing list where the request for review is posted. The following section will describe the defined project processes. Issues that are not handled by these processes happen on an ad-hoc basis based on what has been customary to do in similar cases. [[model-processes]] == Processes [[proc-addrem-committer]] === Adding new and removing old committers The Core team has the responsibility of giving and removing commit privileges to contributors. This can only be done through a vote on the core mailing list. The ports and documentation sub-projects can give commit privileges to people working on these projects, but have to date not removed such privileges. Normally a contributor is recommended to core by a committer. For contributors or outsiders to contact core asking to be a committer is not well thought of and is usually rejected. If the area of particular interest for the developer potentially overlaps with other committers' area of maintainership, the opinion of those maintainers is sought. However, it is frequently this committer that recommends the developer. When a contributor is given committer status, they are assigned a mentor. The committer who recommended the new committer will, in the general case, take it upon themselves to be the new committers mentor. When a contributor is given their commit bit, a <>-signed email is sent from either <>, <>, or nik@freebsd.org to both admins@freebsd.org, the assigned mentor, the new committer, and core confirming the approval of a new account. The mentor then gathers a password line, <> public key, and PGP key from the new committer and sends them to <>. When the new account is created, the mentor activates the commit bit and guides the new committer through the rest of the initial process. .Process summary: adding a new committer image::proc-add-committer.png[Refer to paragraph below for a screen-reader friendly version.] When a contributor sends a piece of code, the receiving committer may choose to recommend that the contributor is given commit privileges. If they recommend this to core, core will vote on this recommendation. If the vote is in favour, a mentor is assigned the new committer and the new committer has to email their details to the administrators for an account to be created. After this, the new committer is all set to make their first commit. By tradition, this is by adding their name to the committers list. Recall that a committer is considered to be someone who has committed code during the past 12 months. However, it is not until after 18 months of inactivity have passed that commit privileges are eligible to be revoked. [<>] There are, however, no automatic procedures for doing this. For reactions concerning commit privileges not triggered by time, see <>. .Process summary: removing a committer image::proc-rm-committer.png[Refer to paragraph below for a screen-reader friendly version.] When Core decides to clean up the committers list, they check who has not made a commit for the past 18 months. Committers who have not done so have their commit bits revoked and their account removed by the administrators. It is also possible for committers to request that their commit bit be retired if for some reason they are no longer going to be actively committing to the project. In this case, it can also be restored at a later time by core, should the committer ask. Roles in this process: . <> . <> . <> . <> . <> [<>] [<>] [<>] [[committing]] === Committing code The committing of new or modified code is one of the most frequent processes in the FreeBSD project and will usually happen many times a day. Committing of code can only be done by a "committer". Committers commit either code written by themselves, code submitted to them, or code submitted through a <>. When code is written by the developer that is non-trivial, they should seek a code review from the community. This is done by sending mail to the relevant list asking for review. Before submitting the code for review, they should ensure it compiles correctly with the entire tree and that all relevant tests run. This is called "pre-commit test". When contributed code is received, it should be reviewed by the committer and tested the same way. When a change is committed to a part of the source that has been contributed from an outside <>, the maintainer should ensure that the patch is contributed back to the vendor. This is in line with the open source philosophy and makes it easier to stay in sync with outside projects as the patches do not have to be reapplied every time a new release is made. After the code has been available for review and no further changes are necessary, the code is committed into the development branch, -CURRENT. If the change applies for the -STABLE branch or the other branches as well, a "Merge From Current" ("MFC") countdown is set by the committer. After the number of days the committer chose when setting the MFC have passed, an email will automatically be sent to the committer reminding them to commit it to the -STABLE branch (and possibly security branches as well). Only security critical changes should be merged to security branches. Delaying the commit to -STABLE and other branches allows for "parallel debugging" where the committed code is tested on a wide range of configurations. This makes changes to -STABLE to contain fewer faults and thus giving the branch its name. .Process summary: A committer commits code image::proc-commit.png[Refer to paragraph below for a screen-reader friendly version.] When a committer has written a piece of code and wants to commit it, they first need to determine if it is trivial enough to go in without prior review or if it should first be reviewed by the developer community. If the code is trivial or has been reviewed and the committer is not the maintainer, they should consult the maintainer before proceeding. If the code is contributed by an outside vendor, the maintainer should create a patch that is sent back to the vendor. The code is then committed and then deployed by the users. Should they find problems with the code, this will be reported and the committer can go back to writing a patch. If a vendor is affected, they can choose to implement or ignore the patch. .Process summary: A contributor commits code image::proc-contrib.png[Refer to paragraphs below and above for a screen-reader friendly version.] The difference when a contributor makes a code contribution is that they submit the code through the Bugzilla interface. This report is picked up by the maintainer who reviews the code and commits it. Hats included in this process are: . <> . <> . <> . <> [<>] [<>] [[process-core-election]] === Core election Core elections are held at least every two years. footnote:[The first Core election was held September 2000] Nine core members are elected. New elections are held if the number of core members drops below seven. New elections can also be held should at least 1/3 of the active committers demand this. When an election is to take place, core announces this at least 6 weeks in advance, and appoints an election manager to run the elections. Only committers can be elected into core. The candidates need to submit their candidacy at least one week before the election starts, but can refine their statements until the voting starts. They are presented in the http://election.uk.freebsd.org/candidates.html[candidates list]. When writing their election statements, the candidates must answer a few standard questions submitted by the election manager. During elections, the rule that a committer must have committed during the 12 past months is followed strictly. Only these committers are eligible to vote. When voting, the committer may vote once in support of up to nine nominees. The voting is done over a period of four weeks with reminders being posted on "developers" mailing list that is available to all committers. The election results are released one week after the election ends, and the new core team takes office one week after the results have been posted. Should there be a voting tie, this will be resolved by the new, unambiguously elected core members. Votes and candidate statements are archived, but the archives are not publicly available. .Process summary: Core elections image::proc-elections.png[Refer to paragraph below for a screen-reader friendly version.] Core announces the election and selects an election manager who prepares the elections, and when ready, candidates can announce their candidacies through submitting their statements. The committers then vote. After the vote is over, the election results are announced and the new core team takes office. Hats in core elections are: * <> * <> * <> [<>] [<>] [<>] [[new-features]] === Development of new features Within the project there are sub-projects that are working on new features. These projects are generally done by one person [<>]. Every project is free to organise development as it sees fit. However, when the project is merged to the -CURRENT branch it must follow the project guidelines. When the code has been well tested in the -CURRENT branch and deemed stable enough and relevant to the -STABLE branch, it is merged to the -STABLE branch. The requirements of the project are given by developer wishes, requests from the community in terms of direct requests by mail, Problem Reports, commercial funding for the development of features, or contributions by the scientific community. The wishes that come within the responsibility of a developer are given to that developer who prioritises their time between the request and their wishes. A common way to do this is maintain a TODO-list maintained by the project. Items that do not come within someone's responsibility are collected on TODO-lists unless someone volunteers to take the responsibility. All requests, their distribution and follow-up are handled by the <> tool. Requirements analysis happens in two ways. The requests that come in are discussed on mailing lists, both within the main project and in the sub-project that the request belongs to or is spawned by the request. Furthermore, individual developers on the sub-project will evaluate the feasibility of the requests and determine the prioritisation between them. Other than archives of the discussions that have taken place, no outcome is created by this phase that is merged into the main project. As the requests are prioritised by the individual developers on the basis of doing what they find interesting, necessary, or are funded to do, there is no overall strategy or prioritisation of what requests to regard as requirements and following up their correct implementation. However, most developers have some shared vision of what issues are more important, and they can ask for guidelines from the release engineering team. The verification phase of the project is two-fold. Before committing code to the current-branch, developers request their code to be reviewed by their peers. This review is for the most part done by functional testing, but also code review is important. When the code is committed to the branch, a broader functional testing will happen, that may trigger further code review and debugging should the code not behave as expected. This second verification form may be regarded as structural verification. Although the sub-projects themselves may write formal tests such as unit tests, these are usually not collected by the main project and are usually removed before the code is committed to the current branch. footnote:[More and more tests are however performed when building the system (make world). These tests are however a very new addition and no systematic framework for these tests have yet been created.] [[model-maintenance]] === Maintenance It is an advantage to the project to for each area of the source have at least one person that knows this area well. Some parts of the code have designated maintainers. Others have de-facto maintainers, and some parts of the system do not have maintainers. The maintainer is usually a person from the sub-project that wrote and integrated the code, or someone who has ported it from the platform it was written for. footnote:[sendmail and named are examples of code that has been merged from other platforms.] The maintainer's job is to make sure the code is in sync with the project the code comes from if it is contributed code, and apply patches submitted by the community or write fixes to issues that are discovered. The main bulk of work that is put into the FreeBSD project is maintenance. [<>] has made a figure showing the life cycle of changes. Jørgenssen's model for change integration [.informaltable] [cols="1,1,1", options="header"] |=== | Stage | Next if successful | Next if unsuccessful |code |review | |review |pre-commit test |code |pre-commit test |development release |code |development release |parallel debugging |code |parallel debugging |production release |code |production release | |code |=== Here "development release" refers to the -CURRENT branch while "production release" refers to the -STABLE branch. The "pre-commit test" is the functional testing by peer developers when asked to do so or trying out the code to determine the status of the sub-project. "Parallel debugging" is the functional testing that can trigger more review, and debugging when the code is included in the -CURRENT branch. As of this writing, there were 269 committers in the project. When they commit a change to a branch, that constitutes a new release. It is very common for users in the community to track a particular branch. The immediate existence of a new release makes the changes widely available right away and allows for rapid feedback from the community. This also gives the community the response time they expect on issues that are of importance to them. This makes the community more engaged, and thus allows for more and better feedback that again spurs more maintenance and ultimately should create a better product. Before making changes to code in parts of the tree that has a history unknown to the committer, the committer is required to read the commit logs to see why certain features are implemented the way they are in order not to make mistakes that have previously either been thought through or resolved. [[model-pr]] === Problem reporting Before FreeBSD 10, FreeBSD included a problem reporting tool called `send-pr`. Problems include bug reports, feature requests, feature enhancements and notices of new versions of external software that are included in the project. Although `send-pr` is available, users and developers are encouraged to submit issues using our https://bugs.freebsd.org/submit/[ problem report form]. Problem reports are sent to an email address where it is inserted into the Problem Reports maintenance database. A <> classifies the problem and sends it to the correct group or maintainer within the project. After someone has taken responsibility for the report, the report is being analysed. This analysis includes verifying the problem and thinking out a solution for the problem. Often feedback is required from the report originator or even from the FreeBSD community. Once a patch for the problem is made, the originator may be asked to try it out. Finally, the working patch is integrated into the project, and documented if applicable. It there goes through the regular maintenance cycle as described in section <>. These are the states a problem report can be in: open, analyzed, feedback, patched, suspended and closed. The suspended state is for when further progress is not possible due to the lack of information or for when the task would require so much work that nobody is working on it at the moment. .Process summary: problem reporting image::proc-pr.png[Refer to paragraph below for a screen-reader friendly version.] A problem is reported by the report originator. It is then classified by a bugbuster and handed to the correct maintainer. They verify the problem and discuss the problem with the originator until they have enough information to create a working patch. This patch is then committed and the problem report is closed. The roles included in this process are: . <> . <> . <> [<>]. [<>] [[process-reactions]] === Reacting to misbehavior [<>] has a number of rules that committers should follow. However, it happens that these rules are broken. The following rules exist in order to be able to react to misbehavior. They specify what actions will result in how long a suspension of the committer's commit privileges. * Committing during code freezes without the approval of the Release Engineering team - 2 days * Committing to a security branch without approval - 2 days * Commit wars - 5 days to all participating parties * Impolite or inappropriate behavior - 5 days [<>] For the suspensions to be efficient, any single core member can implement a suspension before discussing it on the "core" mailing list. Repeat offenders can, with a 2/3 vote by core, receive harsher penalties, including permanent removal of commit privileges. (However, the latter is always viewed as a last resort, due to its inherent tendency to create controversy.) All suspensions are posted to the "developers" mailing list, a list available to committers only. It is important that you cannot be suspended for making technical errors. All penalties come from breaking social etiquette. Hats involved in this process: * <> * <> [[process-release-engineering]] === Release engineering The FreeBSD project has a Release Engineering team with a principal release engineer that is responsible for creating releases of FreeBSD that can be brought out to the user community via the net or sold in retail outlets. Since FreeBSD is available on multiple platforms and releases for the different architectures are made available at the same time, the team has one person in charge of each architecture. Also, there are roles in the team responsible for coordinating quality assurance efforts, building a package set and for having an updated set of documents. When referring to the release engineer, a representative for the release engineering team is meant. When a release is coming, the FreeBSD project changes shape somewhat. A release schedule is made containing feature- and code-freezes, release of interim releases and the final release. A feature-freeze means no new features are allowed to be committed to the branch without the release engineers' explicit consent. Code-freeze means no changes to the code (like bugs-fixes) are allowed to be committed without the release engineers' explicit consent. This feature- and code-freeze is known as stabilising. During the release process, the release engineer has the full authority to revert to older versions of code and thus "back out" changes should they find that the changes are not suitable to be included in the release. There are three different kinds of releases: . .0 releases are the first release of a major version. These are branched of the -CURRENT branch and have a significantly longer release engineering cycle due to the unstable nature of the -CURRENT branch . .X releases are releases of the -STABLE branch. They are scheduled to come out every 4 months. . .X.Y releases are security releases that follow the .X branch. These come out only when sufficient security fixes have been merged since the last release on that branch. New features are rarely included, and the security team is far more involved in these than in regular releases. For releases of the -STABLE-branch, the release process starts 45 days before the anticipated release date. During the first phase, the first 15 days, the developers merge what changes they have had in -CURRENT that they want to have in the release to the release branch. When this period is over, the code enters a 15 day code freeze in which only bug fixes, documentation updates, security-related fixes and minor device driver changes are allowed. These changes must be approved by the release engineer in advance. At the beginning of the last 15 day period a release candidate is created for widespread testing. Updates are less likely to be allowed during this period, except for important bug fixes and security updates. In this final period, all releases are considered release candidates. At the end of the release process, a release is created with the new version number, including binary distributions on web sites and the creation of CD-ROM images. However, the release is not considered "really released" until a <>-signed message stating exactly that, is sent to the mailing list freebsd-announce; anything labelled as a "release" before that may well be in-process and subject to change before the PGP-signed message is sent. footnote:[Many commercial vendors use these images to create CD-ROMs that are sold in retail outlets.]. The releases of the -CURRENT-branch (that is, all releases that end with ".0") are very similar, but with twice as long timeframe. It starts 8 weeks prior to the release with announcement of the release time line. Two weeks into the release process, the feature freeze is initiated and performance tweaks should be kept to a minimum. Four weeks prior to the release, an official beta version is made available. Two weeks prior to release, the code is officially branched into a new version. This version is given release candidate status, and as with the release engineering of -STABLE, the code freeze of the release candidate is hardened. However, development on the main development branch can continue. Other than these differences, the release engineering processes are alike. *.0 releases go into their own branch and are aimed mainly at early adopters. The branch then goes through a period of stabilisation, and it is not until the <> decides the demands to stability have been satisfied that the branch becomes -STABLE and -CURRENT targets the next major version. While this for the majority has been with *.1 versions, this is not a demand. Most releases are made when a given date that has been deemed a long enough time since the previous release comes. A target is set for having major releases every 18 months and minor releases every 4 months. The user community has made it very clear that security and stability cannot be sacrificed by self-imposed deadlines and target release dates. For slips of time not to become too long with regards to security and stability issues, extra discipline is required when committing changes to -STABLE. . Make release schedule . Feature freeze . Code freeze . Make branch . Release candidate . Stabilize release (loop back to previous step as many times as necessary; when release is considered stable, proceed with next step) . Build packages . Warn mirrors . Publish release [<>] [[tools]] == Tools The major support tools for supporting the development process are Bugzilla, Mailman, and OpenSSH. These are externally developed tools and are commonly used in the open source world. [[tool-git]] === Git Git is a system to handle multiple versions of text files and tracking who committed what changes and why. A project lives within a "repository" and different versions are considered different "branches". [[tool-bugzilla]] === Bugzilla Bugzilla is a maintenance database consisting of a set of tools to track bugs at a central site. It supports the bug tracking process for sending and handling bugs as well as querying and updating the database and editing bug reports. The project uses its web interface to send "Problem Reports" to the project's central Bugzilla server. The committers also have web and command-line clients. [[model-mailman]] === Mailman Mailman is a program that automates the management of mailing lists. The FreeBSD Project uses it to run 16 general lists, 60 technical lists, 4 limited lists and 5 lists with Git commit logs. It is also used for many mailing lists set up and used by other people and projects in the FreeBSD community. General lists are lists for the general public, technical lists are mainly for the development of specific areas of interest, and closed lists are for internal communication not intended for the general public. The majority of all the communication in the project goes through these 85 lists [<>, Appendix C]. [[tool-pgp]] === Pretty Good Privacy Pretty Good Privacy, better known as PGP, is a cryptosystem using a public key architecture to allow people to digitally sign and/or encrypt information in order to ensure secure communication between two parties. A signature is used when sending information out to many recipients, enabling them to verify that the information has not been tampered with before they received it. In the FreeBSD Project this is the primary means of ensuring that information has been written by the person who claims to have written it, and not altered in transit. [[tool-ssh2]] === Secure Shell Secure Shell is a standard for securely logging into a remote system and for executing commands on the remote system. It allows other connections, called tunnels, to be established and protected between the two involved systems. This standard exists in two primary versions, and only version two is used for the FreeBSD Project. The most common implementation of the standard is OpenSSH that is a part of the project's main distribution. Since its source is updated more often than FreeBSD releases, the latest version is also available in the ports tree. [[sub-projects]] == Sub-projects Sub-projects are formed to reduce the amount of communication needed to coordinate the group of developers. When a problem area is sufficiently isolated, most communication would be within the group focusing on the problem, requiring less communication with the groups they communicate with than were the group not isolated. [[sub-project-ports]] === The Ports Subproject A "port" is a set of meta-data and patches that are needed to fetch, compile and install correctly an external piece of software on a FreeBSD system. The amount of ports has grown at a tremendous rate, as shown by the following figure. .Number of ports added between 1995 and 2022 [[fig-ports]] image::portsstatus.svg <> shows the number of ports available to FreeBSD in the period 1995 to 2022. It looks like the curve has first grown exponentially, and then from the middle of 2001 to the middle of 2007 grown linearly at a rate of about 2000 ports/year, before its growth rate gets lower. As the external software described by the port often is under continued development, the amount of work required to maintain the ports is already large, and increasing. This has led to the ports part of the FreeBSD project gaining a more empowered structure, and is more and more becoming a sub-project of the FreeBSD project. Ports has its own core team with the <> as its leader, and this team can appoint committers without FreeBSD Core's approval. Unlike in the FreeBSD Project, where a lot of maintenance frequently is rewarded with a commit bit, the ports sub-project contains many active maintainers that are not committers. Unlike the main project, the ports tree is not branched. Every release of FreeBSD follows the current ports collection and has thus available updated information on where to find programs and how to build them. This, however, means that a port that makes dependencies on the system may need to have variations depending on what version of FreeBSD it runs on. With an unbranched ports repository it is not possible to guarantee that any port will run on anything other than -CURRENT and -STABLE, in particular older, minor releases. There is neither the infrastructure nor volunteer time needed to guarantee this. For efficiency of communication, teams depending on Ports, such as the release engineering team, have their own ports liaisons. [[sub-project-documentation]] === The FreeBSD Documentation Project The FreeBSD Documentation project was started January 1995. From the initial group of a project leader, four team leaders and 16 members, they are now a total of 44 committers. The documentation mailing list has just under 300 members, indicating that there is quite a large community around it. The goal of the Documentation project is to provide good and useful documentation of the FreeBSD project, thus making it easier for new users to get familiar with the system and detailing advanced features for the users. The main tasks in the Documentation project are to work on current projects in the "FreeBSD Documentation Set", and translate the documentation to other languages. Like the FreeBSD Project, documentation is split in the same branches. This is done so that there is always an updated version of the documentation for each version. Only documentation errors are corrected in the security branches. Like the ports sub-project, the Documentation project can appoint documentation committers without FreeBSD Core's approval. [<>]. The Documentation project has extref:{fdp-primer}[a primer]. This is used both to introduce new project members to the standard tools and syntaxes and to act as a reference when working on the project. :sectnums!: [bibliography] [[bibliography]] == References [[brooks]] [Brooks, 1995] Frederick P. Brooks. Copyright © 1975, 1995 Pearson Education Limited. 0201835959. Addison-Wesley Pub Co. The Mythical Man-Month. Essays on Software Engineering, Anniversary Edition (2nd Edition). [[thesis]] [Saers, 2003] Niklas Saers. Copyright © 2003. A project model for the FreeBSD Project. Candidatus Scientiarum thesis. http://niklas.saers.com/thesis. [[jorgensen2001]] [Jørgensen, 2001] Niels Jørgensen. Copyright © 2001. Putting it All in the Trunk. Incremental Software Development in the FreeBSD Open Source Project. http://www.dat.ruc.dk/~nielsj/research/papers/freebsd.pdf. [[ref-pmbok]] [PMI, 2000] Project Management Institute. Copyright © 1996, 2000 Project Management Institute. 1-880410-23-0. Project Management Institute. Newtown Square Pennsylvania USA . PMBOK Guide. A Guide to the Project Management Body of Knowledge, 2000 Edition. [[freebsd-bylaws]] [FreeBSD, 2000A] Copyright © 2002 The FreeBSD Project. Core Bylaws. https://www.freebsd.org/internal/bylaws/. [[freebsd-developer-handbook]] [FreeBSD, 2002A] Copyright © 2002 The FreeBSD Documentation Project. FreeBSD Developer's Handbook. extref:{developers-handbook}[Developers Handbook]. [[bsd-election2002]] [FreeBSD, 2002B] Copyright © 2002 The FreeBSD Project. Core team election 2002. http://election.uk.freebsd.org/candidates.html. [[freebsd-handle-pr]] [FreeBSD, 2002C] Dag-Erling Smørgrav and Hiten Pandya. Copyright © 2002 The FreeBSD Documentation Project. The FreeBSD Documentation Project. Problem Report Handling Guidelines. extref:{pr-guidelines}[Problem Report Handling Guidelines]. [[freebsd-send-pr]] [FreeBSD, 2002D] Dag-Erling Smørgrav. Copyright © 2002 The FreeBSD Documentation Project. The FreeBSD Documentation Project. Writing FreeBSD Problem Reports. extref:{problem-reports}[Writing FreeBSD Problem Reports]. [[freebsd-committer]] [FreeBSD, 2001] Copyright © 2001 The FreeBSD Documentation Project. The FreeBSD Documentation Project. Committers Guide. extref:{committers-guide}[Committer's Guide]. [[freebsd-releng]] [FreeBSD, 2002E] Murray Stokely. Copyright © 2002 The FreeBSD Documentation Project. The FreeBSD Documentation Project. FreeBSD Release Engineering. extref:{releng}[FreeBSD Release Engineering]. [[ref-bsd-handbook]] [FreeBSD, 2003A] The FreeBSD Documentation Project. FreeBSD Handbook. extref:{handbook}[FreeBSD Handbook]. [[freebsd-contributors]] [FreeBSD, 2002F] Copyright © 2002 The FreeBSD Documentation Project. The FreeBSD Documentation Project. Contributors to FreeBSD. extref:{contributors}[Contributors to FreeBSD]. [[freebsd-election]] [FreeBSD, 2002G] Copyright © 2002 The FreeBSD Project. The FreeBSD Project. Core team elections 2002. http://election.uk.freebsd.org. [[freebsd-expiration-policy]] [FreeBSD, 2002H] Copyright © 2002 The FreeBSD Project. The FreeBSD Project. Commit Bit Expiration Policy. 2002/04/06 15:35:30. https://www.freebsd.org/internal/expire-bits/. [[freebsd-new-account]] [FreeBSD, 2002I] Copyright © 2002 The FreeBSD Project. The FreeBSD Project. New Account Creation Procedure. 2002/08/19 17:11:27. https://www.freebsd.org/internal/new-account/. [[freebsd-doceng-charter]] [FreeBSD, 2003B] Copyright © 2002 The FreeBSD Documentation Project. The FreeBSD Documentation Project. FreeBSD DocEng Team Charter. 2003/03/16 12:17. https://www.freebsd.org/internal/doceng/. [[ref-freebsd-trenches]] [Lehey, 2002] Greg Lehey. Copyright © 2002 Greg Lehey. Greg Lehey. Two years in the trenches. The evolution of a software project. http://www.lemis.com/grog/In-the-trenches.pdf. diff --git a/documentation/content/en/books/faq/_index.adoc b/documentation/content/en/books/faq/_index.adoc index 80e9ee53b7..bc9a3c7524 100644 --- a/documentation/content/en/books/faq/_index.adoc +++ b/documentation/content/en/books/faq/_index.adoc @@ -1,3188 +1,3189 @@ --- title: Frequently Asked Questions for FreeBSD 12.X and 13.X authors: - author: The FreeBSD Documentation Project copyright: 1995-2022 The FreeBSD Documentation Project description: Frequently Asked Questions, and answers, covering all aspects of FreeBSD trademarks: ["freebsd", "ibm", "ieee", "adobe", "intel", "linux", "microsoft", "opengroup", "sun", "netbsd", "general"] bookOrder: 5 tags: ["FAQ", "FreeBSD FAQ"] +layout: single --- = Frequently Asked Questions for FreeBSD {rel2-relx} and {rel-relx} :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :partnums: :source-highlighter: rouge :experimental: :images-path: books/faq/ :rel-numbranch: 4 :rel-head: 14-CURRENT :rel-head-relx: 14.X :rel-head-releng: head/ :rel-relx: 13.X :rel-stable: 13-STABLE :rel-releng: stable/13/ :rel-relengdate: December 2018 :rel2-relx: 12.X :rel2-stable: 12-STABLE :rel2-releng: stable/12/ :rel2-relengdate: December 2018 ifdef::env-beastie[] ifdef::backend-html5[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [.abstract-title] Abstract This is the Frequently Asked Questions (FAQ) for FreeBSD versions {rel-relx} and {rel2-relx}. Every effort has been made to make this FAQ as informative as possible; if you have any suggestions as to how it may be improved, send them to the {freebsd-doc}. The latest version of this document is always available from the extref:{faq}[FreeBSD website]. It may also be downloaded as one large link:.[HTML] file with HTTP or as a variety of other formats from the https://download.freebsd.org/doc/[FreeBSD FTP server]. ''' toc::[] [[introduction]] == Introduction [[what-is-FreeBSD]] === What is FreeBSD? FreeBSD is a modern operating system for desktops, laptops, servers, and embedded systems with support for a large number of https://www.FreeBSD.org/platforms/[platforms]. It is based on U.C. Berkeley's "4.4BSD-Lite" release, with some "4.4BSD-Lite2" enhancements. It is also based indirectly on William Jolitz's port of U.C. Berkeley's "Net/2" to the i386(TM), known as "386BSD", though very little of the 386BSD code remains. FreeBSD is used by companies, Internet Service Providers, researchers, computer professionals, students and home users all over the world in their work, education and recreation. For more detailed information on FreeBSD, refer to the extref:{handbook}[FreeBSD Handbook]. [[FreeBSD-goals]] === What is the goal of the FreeBSD Project? The goal of the FreeBSD Project is to provide a stable and fast general purpose operating system that may be used for any purpose without strings attached. [[bsd-license-restrictions]] === Does the FreeBSD license have any restrictions? Yes. Those restrictions do not control how the code is used, but how to treat the FreeBSD Project itself. The license itself is available at https://www.FreeBSD.org/copyright/freebsd-license/[license] and can be summarized like this: * Do not claim that you wrote this. * Do not sue us if it breaks. * Do not remove or modify the license. Many of us have a significant investment in the project and would certainly not mind a little financial compensation now and then, but we definitely do not insist on it. We believe that our first and foremost "mission" is to provide code to any and all comers, and for whatever purpose, so that the code gets the widest possible use and provides the widest possible benefit. This, we believe, is one of the most fundamental goals of Free Software and one that we enthusiastically support. Code in our source tree which falls under the https://www.FreeBSD.org/copyright/COPYING[GNU General Public License (GPL)] or https://www.FreeBSD.org/copyright/COPYING.LIB[GNU Library General Public License (LGPL)] comes with slightly more strings attached, though at least on the side of enforced access rather than the usual opposite. Due to the additional complexities that can evolve in the commercial use of GPL software, we do, however, endeavor to replace such software with submissions under the more relaxed https://www.FreeBSD.org/copyright/freebsd-license/[FreeBSD license] whenever possible. [[replace-current-OS]] === Can FreeBSD replace my current operating system? For most people, yes. But this question is not quite that cut-and-dried. Most people do not actually use an operating system. They use applications. The applications are what really use the operating system. FreeBSD is designed to provide a robust and full-featured environment for applications. It supports a wide variety of web browsers, office suites, email readers, graphics programs, programming environments, network servers, and much more. Most of these applications can be managed through the https://www.FreeBSD.org/ports/[Ports Collection]. If an application is only available on one operating system, that operating system cannot just be replaced. Chances are, there is a very similar application on FreeBSD, however. As a solid office or Internet server or a reliable workstation, FreeBSD will almost certainly do everything you need. Many computer users across the world, including both novices and experienced UNIX(R) administrators, use FreeBSD as their only desktop operating system. Users migrating to FreeBSD from another UNIX(R)-like environment will find FreeBSD to be similar. Windows(R) and Mac OS(R) users may be interested in instead using https://www.ghostbsd.org/[GhostBSD], https://www.midnightbsd.org/[MidnightBSD] or https://www.nomadbsd.org/[NomadBSD] three FreeBSD-based desktop distributions. Non-UNIX(R) users should expect to invest some additional time learning the UNIX(R) way of doing things. This FAQ and the extref:{handbook}[FreeBSD Handbook] are excellent places to start. [[why-called-FreeBSD]] === Why is it called FreeBSD? * It may be used free of charge, even by commercial users. * Full source for the operating system is freely available, and the minimum possible restrictions have been placed upon its use, distribution and incorporation into other work (commercial or non-commercial). * Anyone who has an improvement or bug fix is free to submit their code and have it added to the source tree (subject to one or two obvious provisions). It is worth pointing out that the word "free" is being used in two ways here: one meaning "at no cost" and the other meaning "do whatever you like". Apart from one or two things you _cannot_ do with the FreeBSD code, for example pretending you wrote it, you can really do whatever you like with it. [[differences-to-other-bsds]] === What are the differences between FreeBSD and NetBSD, OpenBSD, and other open source BSD operating systems? James Howard wrote a good explanation of the history and differences between the various projects, called https://jameshoward.us/archive/bsd-family-tree/[The BSD Family Tree] which goes a fair way to answering this question. Some of the information is out of date, but the history portion in particular remains accurate. Most of the BSDs share patches and code, even today. All of the BSDs have common ancestry. The design goals of FreeBSD are described in <>, above. The design goals of the other most popular BSDs may be summarized as follows: * OpenBSD aims for operating system security above all else. The OpenBSD team wrote man:ssh[1] and man:pf[4], which have both been ported to FreeBSD. * NetBSD aims to be easily ported to other hardware platforms. * DragonFly BSD is a fork of FreeBSD 4.8 that has since developed many interesting features of its own, including the HAMMER file system and support for user-mode "vkernels". [[latest-version]] === What is the latest version of FreeBSD? At any point in the development of FreeBSD, there can be multiple parallel branches. {rel-relx} releases are made from the {rel-stable} branch, and {rel2-relx} releases are made from the {rel2-stable} branch. Up until the release of 12.0, the {rel2-relx} series was the one known as _-STABLE_. However, as of {rel-head-relx}, the {rel2-relx} branch will be designated for an "extended support" status and receive only fixes for major problems, such as security-related fixes. Releases are made <>. While many people stay more up-to-date with the FreeBSD sources (see the questions on <> and <>) than that, doing so is more of a commitment, as the sources are a moving target. More information on FreeBSD releases can be found on the https://www.FreeBSD.org/releng/#release-build[Release Engineering page] and in man:release[7]. [[current]] === What is _FreeBSD-CURRENT_? extref:{handbook}cutting-edge/[FreeBSD-CURRENT, current] is the development version of the operating system, which will in due course become the new FreeBSD-STABLE branch. As such, it is really only of interest to developers working on the system and die-hard hobbyists. See the extref:{handbook}cutting-edge/[relevant section, current] in the extref:{handbook}[Handbook] for details on running _-CURRENT_. Users not familiar with FreeBSD should not use FreeBSD-CURRENT. This branch sometimes evolves quite quickly and due to mistakes can be un-buildable at times. People that use FreeBSD-CURRENT are expected to be able to analyze, debug, and report problems. [[stable]] === What is the FreeBSD-STABLE concept? _FreeBSD-STABLE_ is the development branch from which major releases are made. Changes go into this branch at a slower pace and with the general assumption that they have first been tested in FreeBSD-CURRENT. However, at any given time, the sources for FreeBSD-STABLE may or may not be suitable for general use, as it may uncover bugs and corner cases that were not yet found in FreeBSD-CURRENT. Users who do not have the resources to perform testing should instead run the most recent release of FreeBSD. _FreeBSD-CURRENT_, on the other hand, has been one unbroken line since 2.0 was released. For more detailed information on branches see "extref:{releng}[FreeBSD Release Engineering: Creating the Release Branch, rel-branch]", the status of the branches and the upcoming release schedule can be found on the https://www.FreeBSD.org/releng[Release Engineering Information] page. Version {u-rel123-announce}[{rel123-current}] is the latest release from the {rel2-stable} branch; it was released on {rel123-current-date}. Version {u-rel131-announce}[{rel131-current}] is the latest release from the {rel-stable} branch; it was released on {rel131-current-date}. [[release-freq]] === When are FreeBSD releases made? The {re} releases a new major version of FreeBSD about every 18 months and a new minor version about every 8 months, on average. Release dates are announced well in advance, so that the people working on the system know when their projects need to be finished and tested. A testing period precedes each release, to ensure that the addition of new features does not compromise the stability of the release. Many users regard this caution as one of the best things about FreeBSD, even though waiting for all the latest goodies to reach _-STABLE_ can be a little frustrating. More information on the release engineering process (including a schedule of upcoming releases) can be found on the https://www.FreeBSD.org/releng/[release engineering] pages on the FreeBSD Web site. For people who need or want a little more excitement, binary snapshots are made weekly as discussed above. [[snapshot-freq]] === When are FreeBSD snapshots made? FreeBSD link:https://www.FreeBSD.org/snapshots/[snapshot] releases are made based on the current state of the _-CURRENT_ and _-STABLE_ branches. The goals behind each snapshot release are: * To test the latest version of the installation software. * To give people who would like to run _-CURRENT_ or _-STABLE_ but who do not have the time or bandwidth to follow it on a day-to-day basis an easy way of bootstrapping it onto their systems. * To preserve a fixed reference point for the code in question, just in case we break something really badly later. (Although Subversion normally prevents anything horrible like this happening.) * To ensure that all new features and fixes in need of testing have the greatest possible number of potential testers. No claims are made that any _-CURRENT_ snapshot can be considered "production quality" for any purpose. If a stable and fully tested system is needed, stick to full releases. Snapshot releases are directly available from link:https://www.FreeBSD.org/snapshots/[snapshot]. Official snapshots are generated on a regular basis for all actively developed branches. [[responsible]] === Who is responsible for FreeBSD? The key decisions concerning the FreeBSD project, such as the overall direction of the project and who is allowed to add code to the source tree, are made by a link:https://www.FreeBSD.org/administration#t-core[core team] of 9 people. There is a much larger team of more than 350 extref:{contributors}[committers, staff-committers] who are authorized to make changes directly to the FreeBSD source tree. However, most non-trivial changes are discussed in advance in the <>, and there are no restrictions on who may take part in the discussion. [[where-get]] === Where can I get FreeBSD? Every supported release of FreeBSD is available from the https://www.freebsd.org/where/[FreeBSD release locator page]: * For the latest {rel-stable} release, {rel131-current}-RELEASE, follow the link for link:https://www.freebsd.org/where/#download-rel131[the appropriate architecture and installation mode for {rel131-current}-RELEASE]. * For the latest {rel2-stable} release, {rel123-current}-RELEASE, follow the link for link:https://www.freebsd.org/where/#download-rel123[the appropriate architecture and installation mode for {rel123-current}-RELEASE]. * link:https://www.FreeBSD.org/snapshots/[Snapshot] releases are made monthly for the <> and <> branches, these being of service purely to bleeding-edge testers and developers. Information about obtaining FreeBSD on CD, DVD, and other media can be found in extref:{handbook}mirrors/[the Handbook, mirrors]. [[access-pr]] === How do I access the Problem Report database? The Problem Report database of all user change requests may be queried by using our web-based PR https://bugs.FreeBSD.org/search/[query] interface. The link:https://www.FreeBSD.org/support/bugreports[web-based problem report submission interface] can be used to submit problem reports through a web browser. Before submitting a problem report, read extref:{problem-reports}[Writing FreeBSD Problem Reports], an article on how to write good problem reports. [[support]] == Documentation and Support [[books]] === What good books are there about FreeBSD? The project produces a wide range of documentation, available online from this link: https://www.FreeBSD.org/docs/[https://www.FreeBSD.org/docs/]. [[doc-formats]] === Is the documentation available in other formats, such as PDF? Yes. link:https://download.freebsd.org/doc/[The documentation is also available in several formats] on the FreeBSD download site. Documentation directories are categorized according to: * The document's name, such as `faq`, or `handbook`. * The document's language, based on the locale names found under [.filename]#/usr/share/locale# on a FreeBSD system, with the encodings removed as all documentation now uses UTF-8. The current languages are: + [.informaltable] [cols="1,1", frame="none", options="header"] |=== | Name | Meaning |`en` |English |`bn-bd` |Bengali or Bangla (Bangladesh) |`da` |Danish |`de` |German |`el` |Greek |`es` |Spanish |`fr` |French |`hu` |Hungarian |`it` |Italian |`ja` |Japanese |`ko` |Korean |`mn` |Mongolian |`nl` |Dutch |`pl` |Polish |`pt-br` |Portuguese (Brazil) |`ru` |Russian |`tr` |Turkish |`zh-cn` |Simplified Chinese (China) |`zh-tw` |Traditional Chinese (Taiwan) |=== + [NOTE] ==== Some documents may not be available in all languages. ==== * The document's format. We produce the documentation in a number of different output formats. Each format has its own advantages and disadvantages. Some formats are better suited for online reading, while others are meant to be aesthetically pleasing when printed on paper. Having the documentation available in any of these formats ensures that our readers will be able to read the parts they are interested in, either on their monitor, or on paper after printing the documents. The currently available formats are: + [.informaltable] [cols="1,1", frame="none", options="header"] |=== | Format | Meaning |`html` |Depending on the document: one large HTML file containing the entire document or a collection of small linked HTML files, both together with images, stylesheets, and JavaScript |`pdf` |Adobe's Portable Document Format |=== * The compression and packaging scheme. .. Where the format is `html`, the files are bundled up using man:tar[1]. The resulting [.filename]#.tar# is then compressed using man:gzip[1]. .. The PDF format generates one file. For example, [.filename]#explaining-bsd_en.pdf#, [.filename]#faq_en.pdf#, and so on. After choosing the format, download the files, uncompress them if necessary, then copy the appropriate documents into place. For example, the HTML version of the FAQ can be found in [.filename]#doc/en/books/faq/faq_en.tar.gz# To download and uncompress that file, type: [source,shell] .... % fetch https://download.freebsd.org/doc/en/books/faq/faq_en.tar.gz % tar xvf faq_en.tar.gz .... If the file is compressed, tar will automatically detect the appropriate format and decompress it correctly, resulting in a collection of files. The main one is called [.filename]#index.html#, which will contain the whole document, starting with the table of contents linking to the other parts of the document. [[mailing]] === Where do I find info on the FreeBSD mailing lists? What FreeBSD news groups are available? Refer to the extref:{handbook}eresources/[Handbook entry on mailing-lists, eresources-mail] and the extref:{handbook}eresources/[Handbook entry on newsgroups, eresources-news]. [[irc]] === Are there FreeBSD IRC (Internet Relay Chat) channels? Yes, most major IRC networks host a FreeBSD chat channel and the FreeBSD wiki holds an up to date https://wiki.freebsd.org/IRC/Channels[list of IRC channels]. Each of these channels are distinct and are not connected to each other. Since their chat styles differ, try each to find one suited to your chat style. [[forums]] === Are there any web based forums to discuss FreeBSD? The official FreeBSD forums are located at https://forums.FreeBSD.org/[https://forums.FreeBSD.org/]. [[training]] === Where can I get commercial FreeBSD training and support? http://www.ixsystems.com[iXsystems, Inc.], parent company of the http://www.freebsdmall.com/[FreeBSD Mall], provides commercial FreeBSD and TrueOS software http://www.ixsystems.com/support[support], in addition to FreeBSD development and tuning solutions. BSD Certification Group, Inc. provides system administration certifications for DragonFly BSD, FreeBSD, NetBSD, and OpenBSD. Refer to http://www.BSDCertification.org[their site] for more information. Any other organizations providing training and support should contact the Project to be listed here. [[install]] == Installation [[which-architecture]] === Which platform should I download? I have a 64 bit capable Intel(R) CPU, but I only see amd64. amd64 is the term FreeBSD uses for 64-bit compatible x86 architectures (also known as "x86-64" or "x64"). Most modern computers should use amd64. Older hardware should use i386. When installing on a non-x86-compatible architecture, select the platform which best matches the hardware. [[floppy-download]] === Which file do I download to get FreeBSD? On the https://www.freebsd.org/where/[Getting FreeBSD] page, select `[iso]` next to the architecture that matches the hardware. Any of the following can be used: [.informaltable] [cols="1,1", frame="none", options="header"] |=== | file | description |[.filename]#disc1.iso# |Contains enough to install FreeBSD and a minimal set of packages. |[.filename]#dvd1.iso# |Similar to [.filename]#disc1.iso# but with additional packages. |[.filename]#memstick.img# |A bootable image sufficient for writing to a USB stick. |[.filename]#bootonly.iso# |A minimal image that requires network access during installation to completely install FreeBSD. |=== Full instructions on this procedure and a little bit more about installation issues in general can be found in the extref:{handbook}bsdinstall[Handbook entry on installing FreeBSD]. [[floppy-image-too-large]] === What do I do if the install image does not boot? This can be caused by not downloading the image in _binary_ mode when using FTP. Some FTP clients default their transfer mode to _ascii_ and attempt to change any end-of-line characters received to match the conventions used by the client's system. This will almost invariably corrupt the boot image. Check the SHA-256 checksum of the downloaded boot image: if it is not _exactly_ that on the server, then the download process is suspect. When using a command line FTP client, type _binary_ at the FTP command prompt after getting connected to the server and before starting the download of the image. [[install-instructions-location]] === Where are the instructions for installing FreeBSD? Installation instructions can be found at extref:{handbook}bsdinstall/[Handbook entry on installing FreeBSD]. [[custom-boot-floppy]] === How can I make my own custom release or install disk? Customized FreeBSD installation media can be created by building a custom release. Follow the instructions in the extref:{releng}[Release Engineering] article. [[windows-coexist]] === Can Windows(R) co-exist with FreeBSD? (x86-specific) If Windows(R) is installed first, then yes. FreeBSD's boot manager will then manage to boot Windows(R) and FreeBSD. If Windows(R) is installed afterwards, it will overwrite the boot manager. If that happens, see the next section. [[bootmanager-restore]] === Another operating system destroyed my Boot Manager. How do I get it back? (x86-specific) This depends upon the boot manager. The FreeBSD boot selection menu can be reinstalled using man:boot0cfg[8]. For example, to restore the boot menu onto the disk _ada0_: [source,shell] .... # boot0cfg -B ada0 .... The non-interactive MBR bootloader can be installed using man:gpart[8]: [source,shell] .... # gpart bootcode -b /boot/mbr ada0 .... For more complex situations, including GPT disks, see man:gpart[8]. [[need-complete-sources]] === Do I need to install the source? In general, no. There is nothing in the base system which requires the presence of the source to operate. Some ports, like package:sysutils/lsof[], will not build unless the source is installed. In particular, if the port builds a kernel module or directly operates on kernel structures, the source must be installed. [[need-kernel]] === Do I need to build a kernel? Usually not. The supplied `GENERIC` kernel contains the drivers an ordinary computer will need. man:freebsd-update[8], the FreeBSD binary upgrade tool, cannot upgrade custom kernels, another reason to stick with the `GENERIC` kernel when possible. For computers with very limited RAM, such as embedded systems, it may be worthwhile to build a smaller custom kernel containing just the required drivers. [[password-encryption]] === Should I use DES, Blowfish, or MD5 passwords and how do I specify which form my users receive? FreeBSD uses _SHA512_ by default. DES passwords are still available for backwards compatibility with operating systems that still use the less secure password format. FreeBSD also supports the Blowfish and MD5 password formats. Which password format to use for new passwords is controlled by the `passwd_format` login capability in [.filename]#/etc/login.conf#, which takes values of `des`, `blf` (if these are available) or `md5`. See the man:login.conf[5] manual page for more information about login capabilities. [[ffs-limits]] === What are the limits for FFS file systems? For FFS file systems, the largest file system is practically limited by the amount of memory required to man:fsck[8] the file system. man:fsck[8] requires one bit per fragment, which with the default fragment size of 4 KB equates to 32 MB of memory per TB of disk. This does mean that on architectures which limit userland processes to 2 GB (e.g., i386(TM)), the maximum man:fsck[8]'able filesystem is ~60 TB. If there was not a man:fsck[8] memory limit the maximum filesystem size would be 2 ^ 64 (blocks) * 32 KB => 16 Exa * 32 KB => 512 ZettaBytes. The maximum size of a single FFS file is approximately 2 PB with the default block size of 32 KB. Each 32 KB block can point to 4096 blocks. With triple indirect blocks, the calculation is 32 KB * 12 + 32 KB * 4096 + 32 KB * 4096^2 + 32 KB * 4096^3. Increasing the block size to 64 KB will increase the max file size by a factor of 16. [[archsw-readin-failed-error]] === Why do I get an error message, readin failed after compiling and booting a new kernel? The world and kernel are out of sync. This is not supported. Be sure to use `make buildworld` and `make buildkernel` to update the kernel. Boot the system by specifying the kernel directly at the second stage, pressing any key when the `|` shows up before loader is started. [[general-configuration-tool]] === Is there a tool to perform post-installation configuration tasks? Yes. bsdconfig provides a nice interface to configure FreeBSD post-installation. [[hardware]] == Hardware Compatibility [[compatibility-general]] === General [[which-hardware-to-get]] ==== I want to get a piece of hardware for my FreeBSD system. Which model/brand/type is best? This is discussed continually on the FreeBSD mailing lists but is to be expected since hardware changes so quickly. Read through the Hardware Notes for FreeBSD link:{u-rel123-hardware}[{rel123-current}] or link:{u-rel131-hardware}[{rel131-current}] and search the https://www.FreeBSD.org/search/#mailinglists[mailing list archives] before asking about the latest and greatest hardware. Chances are a discussion about that type of hardware took place just last week. Before purchasing a laptop, check the archives for {freebsd-questions}, or possibly a specific mailing list for a particular hardware type. [[memory-upper-limitation]] ==== What are the limits for memory? FreeBSD as an operating system generally supports as much physical memory (RAM) as the platform it is running on does. Keep in mind that different platforms have different limits for memory; for example i386(TM) without PAE supports at most 4 GB of memory (and usually less than that because of PCI address space) and i386(TM) with PAE supports at most 64 GB memory. As of FreeBSD 10, AMD64 platforms support up to 4 TB of physical memory. [[memory-i386-over-4gb]] ==== Why does FreeBSD report less than 4 GB memory when installed on an i386(TM) machine? The total address space on i386(TM) machines is 32-bit, meaning that at most 4 GB of memory is addressable (can be accessed). Furthermore, some addresses in this range are reserved by hardware for different purposes, for example for using and controlling PCI devices, for accessing video memory, and so on. Therefore, the total amount of memory usable by the operating system for its kernel and applications is limited to significantly less than 4 GB. Usually, 3.2 GB to 3.7 GB is the maximum usable physical memory in this configuration. To access more than 3.2 GB to 3.7 GB of installed memory (meaning up to 4 GB but also more than 4 GB), a special tweak called PAE must be used. PAE stands for Physical Address Extension and is a way for 32-bit x86 CPUs to address more than 4 GB of memory. It remaps the memory that would otherwise be overlaid by address reservations for hardware devices above the 4 GB range and uses it as additional physical memory (see man:pae[4]). Using PAE has some drawbacks; this mode of memory access is a little bit slower than the normal (without PAE) mode and loadable modules (see man:kld[4]) are not supported. This means all drivers must be compiled into the kernel. The most common way to enable PAE is to build a new kernel with the special ready-provided kernel configuration file called [.filename]#PAE#, which is already configured to build a safe kernel. Note that some entries in this kernel configuration file are too conservative and some drivers marked as unready to be used with PAE are actually usable. A rule of thumb is that if the driver is usable on 64-bit architectures (like AMD64), it is also usable with PAE. When creating a custom kernel configuration file, PAE can be enabled by adding the following line: [.programlisting] .... options PAE .... PAE is not much used nowadays because most new x86 hardware also supports running in 64-bit mode, known as AMD64 or Intel(R) 64. It has a much larger address space and does not need such tweaks. FreeBSD supports AMD64 and it is recommended that this version of FreeBSD be used instead of the i386(TM) version if 4 GB or more memory is required. [[compatibility-processors]] === Architectures and Processors [[architectures]] ==== Does FreeBSD support architectures other than the x86? Yes. FreeBSD divides support into multiple tiers. Tier 1 architectures, such as i386 or amd64; are fully supported. Tiers 2 and 3 are supported on a best-effort basis. A full explanation of the tier system is available in the extref:{committers-guide}[Committer's Guide., archs] A complete list of supported architectures can be found on the https://www.FreeBSD.org/platforms/[platforms page.] [[smp-support]] ==== Does FreeBSD support Symmetric Multiprocessing (SMP)? FreeBSD supports symmetric multi-processor (SMP) on all non-embedded platforms (e.g, i386, amd64, etc.). SMP is also supported in arm and MIPS kernels, although some CPUs may not support this. FreeBSD's SMP implementation uses fine-grained locking, and performance scales nearly linearly with number of CPUs. man:smp[4] has more details. [[microcode]] ==== What is microcode? How do I install Intel(R) CPU microcode updates? Microcode is a method of programmatically implementing hardware level instructions. This allows for CPU bugs to be fixed without replacing the on board chip. Install package:sysutils/devcpu-data[], then add: [.programlisting] .... microcode_update_enable="YES" .... to [.filename]#/etc/rc.conf# [[compatibility-peripherals]] === Peripherals [[supported-peripherals]] ==== What kind of peripherals does FreeBSD support? See a list of hardware known to work and any applicable restrictions in the Hardware Notes for FreeBSD link:{u-rel123-hardware}[{rel123-current}] or link:{u-rel131-hardware}[{rel131-current}]. [[compatibility-kbd-mice]] === Keyboards and Mice [[moused]] ==== Is it possible to use a mouse outside the X Window system? The default console driver, man:vt[4], provides the ability to use a mouse pointer in text consoles to cut & paste text. Run the mouse daemon, man:moused[8], and turn on the mouse pointer in the virtual console: [source,shell] .... # moused -p /dev/xxxx -t yyyy # vidcontrol -m on .... Where _xxxx_ is the mouse device name and _yyyy_ is a protocol type for the mouse. The mouse daemon can automatically determine the protocol type of most mice, except old serial mice. Specify the `auto` protocol to invoke automatic detection. If automatic detection does not work, see the man:moused[8] manual page for a list of supported protocol types. For a PS/2 mouse, add `moused_enable="YES"` to [.filename]#/etc/rc.conf# to start the mouse daemon at boot time. Additionally, to use the mouse daemon on all virtual terminals instead of just the console, add `allscreens_flags="-m on"` to [.filename]#/etc/rc.conf#. When the mouse daemon is running, access to the mouse must be coordinated between the mouse daemon and other programs such as X Windows. Refer to the FAQ <> for more details on this issue. [[text-mode-cut-paste]] ==== How do I cut and paste text with a mouse in the text console? It is not possible to remove data using the mouse. However, it is possible to copy and paste. Once the mouse daemon is running as described in the <>, hold down button 1 (left button) and move the mouse to select a region of text. Then, press button 2 (middle button) to paste it at the text cursor. Pressing button 3 (right button) will "extend" the selected region of text. If the mouse does not have a middle button, it is possible to emulate one or remap buttons using mouse daemon options. See the man:moused[8] manual page for details. [[mouse-wheel-buttons]] ==== My mouse has a fancy wheel and buttons. Can I use them in FreeBSD? The answer is, unfortunately, "It depends". These mice with additional features require specialized driver in most cases. Unless the mouse device driver or the user program has specific support for the mouse, it will act just like a standard two, or three button mouse. For the possible usage of wheels in the X Window environment, refer to <>. [[keyboard-delete-key]] ==== How do I use my delete key in sh and csh? For the Bourne Shell, add the following lines to [.filename]#~/.shrc#. See man:sh[1] and man:editrc[5]. [.programlisting] .... bind ^[[3~ ed-delete-next-char # for xterm .... For the C Shell, add the following lines to [.filename]#~/.cshrc#. See man:csh[1]. [.programlisting] .... bindkey ^[[3~ delete-char # for xterm .... [[compatibility-other]] === Other Hardware [[es1370-silent-pcm]] ==== Workarounds for no sound from my man:pcm[4] sound card? Some sound cards set their output volume to 0 at every boot. On FreeBSD 13 and earlier, run the following command every time the machine boots: [source,shell] .... # mixer pcm 100 vol 100 cd 100 .... Use the following command on FreeBSD 14 and later: [source,shell] .... # mixer pcm.volume=100 vol.volume=100 cd.volume=100 .... [[power-management-support]] ==== Does FreeBSD support power management on my laptop? FreeBSD supports the ACPI features found in modern hardware. Further information can be found in man:acpi[4]. [[troubleshoot]] == Troubleshooting [[pae]] === Why is FreeBSD finding the wrong amount of memory on i386(TM) hardware? The most likely reason is the difference between physical memory addresses and virtual addresses. The convention for most PC hardware is to use the memory area between 3.5 GB and 4 GB for a special purpose (usually for PCI). This address space is used to access PCI hardware. As a result real, physical memory cannot be accessed by that address space. What happens to the memory that should appear in that location is hardware dependent. Unfortunately, some hardware does nothing and the ability to use that last 500 MB of RAM is entirely lost. Luckily, most hardware remaps the memory to a higher location so that it can still be used. However, this can cause some confusion when watching the boot messages. On a 32-bit version of FreeBSD, the memory appears lost, since it will be remapped above 4 GB, which a 32-bit kernel is unable to access. In this case, the solution is to build a PAE enabled kernel. See the entry on memory limits for more information. On a 64-bit version of FreeBSD, or when running a PAE-enabled kernel, FreeBSD will correctly detect and remap the memory so it is usable. During boot, however, it may seem as if FreeBSD is detecting more memory than the system really has, due to the described remapping. This is normal and the available memory will be corrected as the boot process completes. [[signal11]] === Why do my programs occasionally die with Signal 11 errors? Signal 11 errors are caused when a process has attempted to access memory which the operating system has not granted it access to. If something like this is happening at seemingly random intervals, start investigating the cause. These problems can usually be attributed to either: . If the problem is occurring only in a specific custom application, it is probably a bug in the code. . If it is a problem with part of the base FreeBSD system, it may also be buggy code, but more often than not these problems are found and fixed long before us general FAQ readers get to use these bits of code (that is what -CURRENT is for). It is probably not a FreeBSD bug if the problem occurs compiling a program, but the activity that the compiler is carrying out changes each time. For example, if `make buildworld` fails while trying to compile [.filename]#ls.c# into [.filename]#ls.o# and, when run again, it fails in the same place, this is a broken build. Try updating source and try again. If the compile fails elsewhere, it is almost certainly due to hardware. In the first case, use a debugger such as man:gdb[1] to find the point in the program which is attempting to access a bogus address and fix it. In the second case, verify which piece of hardware is at fault. Common causes of this include: . The hard disks might be overheating: Check that the fans are still working, as the disk and other hardware might be overheating. . The processor running is overheating: This might be because the processor has been overclocked, or the fan on the processor might have died. In either case, ensure that the hardware is running at what it is specified to run at, at least while trying to solve this problem. If it is not, clock it back to the default settings.) + Regarding overclocking, it is far cheaper to have a slow system than a fried system that needs replacing! Also the community is not sympathetic to problems on overclocked systems. . Dodgy memory: if multiple memory SIMMS/DIMMS are installed, pull them all out and try running the machine with each SIMM or DIMM individually to narrow the problem down to either the problematic DIMM/SIMM or perhaps even a combination. . Over-optimistic motherboard settings: the BIOS settings, and some motherboard jumpers, provide options to set various timings. The defaults are often sufficient, but sometimes setting the wait states on RAM too low, or setting the "RAM Speed: Turbo" option will cause strange behavior. A possible idea is to set to BIOS defaults, after noting the current settings first. . Unclean or insufficient power to the motherboard. Remove any unused I/O boards, hard disks, or CD-ROMs, or disconnect the power cable from them, to see if the power supply can manage a smaller load. Or try another power supply, preferably one with a little more power. For instance, if the current power supply is rated at 250 Watts, try one rated at 300 Watts. Read the section on <> for a further explanation and a discussion on how memory testing software or hardware can still pass faulty memory. There is an extensive FAQ on this at http://www.bitwizard.nl/sig11/[the SIG11 problem FAQ]. Finally, if none of this has helped, it is possibly a bug in FreeBSD. Follow <> to send a problem report. [[trap-12-panic]] === My system crashes with either Fatal trap 12: page fault in kernel mode, or panic:, and spits out a bunch of information. What should I do? The FreeBSD developers are interested in these errors, but need more information than just the error message. Copy the full crash message. Then consult the FAQ section on <>, build a debugging kernel, and get a backtrace. This might sound difficult, but does not require any programming skills. Just follow the instructions. [[proc-table-full]] === What is the meaning of the error maxproc limit exceeded by uid %i, please see tuning(7) and login.conf(5)? The FreeBSD kernel will only allow a certain number of processes to exist at one time. The number is based on the `kern.maxusers` man:sysctl[8] variable. `kern.maxusers` also affects various other in-kernel limits, such as network buffers. If the machine is heavily loaded, increase `kern.maxusers`. This will increase these other system limits in addition to the maximum number of processes. To adjust the `kern.maxusers` value, see the extref:{handbook}config/[File/Process Limits, kern-maxfiles] section of the Handbook. While that section refers to open files, the same limits apply to processes. If the machine is lightly loaded but running a very large number of processes, adjust the `kern.maxproc` tunable by defining it in [.filename]#/boot/loader.conf#. The tunable will not get adjusted until the system is rebooted. For more information about tuning tunables, see man:loader.conf[5]. If these processes are being run by a single user, adjust `kern.maxprocperuid` to be one less than the new `kern.maxproc` value. It must be at least one less because one system program, man:init[8], must always be running. [[remote-fullscreen]] === Why do full screen applications on remote machines misbehave? The remote machine may be setting the terminal type to something other than `xterm` which is required by the FreeBSD console. Alternatively the kernel may have the wrong values for the width and height of the terminal. Check the value of the `TERM` environment variable is `xterm`. If the remote machine does not support that try `vt100`. Run `stty -a` to check what the kernel thinks the terminal dimensions are. If they are incorrect, they can be changed by running `stty rows _RR_ cols _CC_`. Alternatively, if the client machine has package:x11/xterm[] installed, then running `resize` will query the terminal for the correct dimensions and set them. [[connection-delay]] === Why does it take so long to connect to my computer via ssh or telnet? The symptom: there is a long delay between the time the TCP connection is established and the time when the client software asks for a password (or, in man:telnet[1]'s case, when a login prompt appears). The problem: more likely than not, the delay is caused by the server software trying to resolve the client's IP address into a hostname. Many servers, including the Telnet and SSH servers that come with FreeBSD, do this to store the hostname in a log file for future reference by the administrator. The remedy: if the problem occurs whenever connecting the client computer to any server, the problem is with the client. If the problem only occurs when someone connects to the server computer, the problem is with the server. If the problem is with the client, the only remedy is to fix the DNS so the server can resolve it. If this is on a local network, consider it a server problem and keep reading. If this is on the Internet, contact your ISP. If the problem is with the server on a local network, configure the server to resolve address-to-hostname queries for the local address range. See man:hosts[5] and man:named[8] for more information. If this is on the Internet, the problem may be that the local server's resolver is not functioning correctly. To check, try to look up another host such as `www.yahoo.com`. If it does not work, that is the problem. Following a fresh install of FreeBSD, it is also possible that domain and name server information is missing from [.filename]#/etc/resolv.conf#. This will often cause a delay in SSH, as the option `UseDNS` is set to `yes` by default in [.filename]#/etc/ssh/sshd_config#. If this is causing the problem, either fill in the missing information in [.filename]#/etc/resolv.conf# or set `UseDNS` to `no` in [.filename]#sshd_config# as a temporary workaround. [[file-table-full]] === Why does file: table is full show up repeatedly in man:dmesg[8]? This error message indicates that the number of available file descriptors have been exhausted on the system. Refer to the extref:{handbook}config/[kern.maxfiles, kern-maxfiles] section of the extref:{handbook}config/[Tuning Kernel Limits, configtuning-kernel-limits] section of the Handbook for a discussion and solution. [[computer-clock-skew]] === Why does the clock on my computer keep incorrect time? The computer has two or more clocks, and FreeBSD has chosen to use the wrong one. Run man:dmesg[8], and check for lines that contain `Timecounter`. The one with the highest quality value that FreeBSD chose. [source,shell] .... # dmesg | grep Timecounter Timecounter "i8254" frequency 1193182 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 Timecounter "TSC" frequency 2998570050 Hz quality 800 Timecounters tick every 1.000 msec .... Confirm this by checking the `kern.timecounter.hardware` man:sysctl[3]. [source,shell] .... # sysctl kern.timecounter.hardware kern.timecounter.hardware: ACPI-fast .... It may be a broken ACPI timer. The simplest solution is to disable the ACPI timer in [.filename]#/boot/loader.conf#: [.programlisting] .... debug.acpi.disabled="timer" .... Or the BIOS may modify the TSC clock-perhaps to change the speed of the processor when running from batteries, or going into a power saving mode, but FreeBSD is unaware of these adjustments, and appears to gain or lose time. In this example, the `i8254` clock is also available, and can be selected by writing its name to the `kern.timecounter.hardware` man:sysctl[3]. [source,shell] .... # sysctl kern.timecounter.hardware=i8254 kern.timecounter.hardware: TSC -> i8254 .... The computer should now start keeping more accurate time. To have this change automatically run at boot time, add the following line to [.filename]#/etc/sysctl.conf#: [.programlisting] .... kern.timecounter.hardware=i8254 .... [[indefinite-wait-buffer]] === What does the error swap_pager: indefinite wait buffer: mean? This means that a process is trying to page memory from disk, and the page attempt has hung trying to access the disk for more than 20 seconds. It might be caused by bad blocks on the disk drive, disk wiring, cables, or any other disk I/O-related hardware. If the drive itself is bad, disk errors will appear in [.filename]#/var/log/messages# and in the output of `dmesg`. Otherwise, check the cables and connections. [[lock-order-reversal]] === What is a lock order reversal? The FreeBSD kernel uses a number of resource locks to arbitrate contention for certain resources. When multiple kernel threads try to obtain multiple resource locks, there's always the potential for a deadlock, where two threads have each obtained one of the locks and blocks forever waiting for the other thread to release one of the other locks. This sort of locking problem can be avoided if all threads obtain the locks in the same order. A run-time lock diagnostic system called man:witness[4], enabled in FreeBSD-CURRENT and disabled by default for stable branches and releases, detects the potential for deadlocks due to locking errors, including errors caused by obtaining multiple resource locks with a different order from different parts of the kernel. The man:witness[4] framework tries to detect this problem as it happens, and reports it by printing a message to the system console about a `lock order reversal` (often referred to also as LOR). It is possible to get false positives, as man:witness[4] is conservative. A true positive report _does not_ mean that a system is dead-locked; instead it should be understood as a warning that a deadlock could have happened here. [NOTE] ==== Problematic LORs tend to get fixed quickly, so check the {freebsd-current} before posting to it. ==== [[called-with-non-sleepable-locks-held]] === What does 'Called ... with the following non-sleepable locks held' mean? This means that a function that may sleep was called while a mutex (or other unsleepable) lock was held. The reason this is an error is because mutexes are not intended to be held for long periods of time; they are supposed to only be held to maintain short periods of synchronization. This programming contract allows device drivers to use mutexes to synchronize with the rest of the kernel during interrupts. Interrupts (under FreeBSD) may not sleep. Hence it is imperative that no subsystem in the kernel block for an extended period while holding a mutex. To catch such errors, assertions may be added to the kernel that interact with the man:witness[4] subsystem to emit a warning or fatal error (depending on the system configuration) when a potentially blocking call is made while holding a mutex. In summary, such warnings are non-fatal, however with unfortunate timing they could cause undesirable effects ranging from a minor blip in the system's responsiveness to a complete system lockup. For additional information about locking in FreeBSD see man:locking[9]. [[touch-not-found]] === Why does buildworld/installworld die with the message touch: not found? This error does not mean that the man:touch[1] utility is missing. The error is instead probably due to the dates of the files being set sometime in the future. If the CMOS clock is set to local time, run `adjkerntz -i` to adjust the kernel clock when booting into single-user mode. [[applications]] == User Applications [[user-apps]] === Where are all the user applications? Refer to link:https://www.FreeBSD.org/ports/[the ports page] for info on software packages ported to FreeBSD. Most ports should work on all supported versions of FreeBSD. Those that do not are specifically marked as such. Each time a FreeBSD release is made, a snapshot of the ports tree at the time of release is also included in the [.filename]#ports/# directory. FreeBSD supports compressed binary packages to easily install and uninstall ports. Use man:pkg[7] to control the installation of packages. [[how-do-download-ports-tree]] === How do I download the Ports tree? Should I be using Git? See extref:{handbook}ports/[Installing the Ports Collection, ports-using-installation-methods] [[ports-4x]] === Why can I not build this port on my {rel2-relx} -, or {rel-relx} -STABLE machine? If the installed FreeBSD version lags significantly behind _-CURRENT_ or _-STABLE_, update the Ports Collection using the instructions in extref:{handbook}ports/[Using the Ports Collection, ports-using]. If the system is up-to-date, someone might have committed a change to the port which works for _-CURRENT_ but which broke the port for _-STABLE_. https://bugs.FreeBSD.org/submit/[Submit] a bug report, since the Ports Collection is supposed to work for both the _-CURRENT_ and _-STABLE_ branches. [[make-index]] === I just tried to build INDEX using make index, and it failed. Why? First, make sure that the Ports Collection is up-to-date. Errors that affect building [.filename]#INDEX# from an up-to-date copy of the Ports Collection are high-visibility and are thus almost always fixed immediately. There are rare cases where [.filename]#INDEX# will not build due to odd cases involving `OPTIONS_SET` being set in [.filename]#make.conf#. If you suspect that this is the case, try to make [.filename]#INDEX# with those variables turned off before reporting it to {freebsd-ports}. [[ports-update]] === I updated the sources, now how do I update my installed ports? FreeBSD does not include a port upgrading tool, but it does have some tools to make the upgrade process somewhat easier. Additional tools are available to simplify port handling and are described the extref:{handbook}ports/[Upgrading Ports, ports-using] section in the FreeBSD Handbook. [[ports-major-upgrade]] === Do I need to recompile every port each time I perform a major version update? Yes! While a recent system will run with software compiled under an older release, things will randomly crash and fail to work once other ports are installed or updated. When the system is upgraded, various shared libraries, loadable modules, and other parts of the system will be replaced with newer versions. Applications linked against the older versions may fail to start or, in other cases, fail to function properly. For more information, see extref:{handbook}cutting-edge/[the section on upgrades, freebsdupdate-upgrade] in the FreeBSD Handbook. [[ports-minor-upgrade]] === Do I need to recompile every port each time I perform a minor version update? In general, no. FreeBSD developers do their utmost to guarantee binary compatibility across all releases with the same major version number. Any exceptions will be documented in the Release Notes, and advice given there should be followed. [[minimal-sh]] === Why is /bin/sh so minimal? Why does FreeBSD not use bash or another shell? Many people need to write shell scripts which will be portable across many systems. That is why POSIX(R) specifies the shell and utility commands in great detail. Most scripts are written in Bourne shell (man:sh[1]), and because several important programming interfaces (man:make[1], man:system[3], man:popen[3], and analogues in higher-level scripting languages like Perl and Tcl) are specified to use the Bourne shell to interpret commands. As the Bourne shell is so often and widely used, it is important for it to be quick to start, be deterministic in its behavior, and have a small memory footprint. The existing implementation is our best effort at meeting as many of these requirements simultaneously as we can. To keep `/bin/sh` small, we have not provided many of the convenience features that other shells have. That is why other more featureful shells like `bash`, `scsh`, man:tcsh[1], and `zsh` are available. Compare the memory utilization of these shells by looking at the "VSZ" and "RSS" columns in a `ps -u` listing. [[kernelconfig]] == Kernel Configuration [[make-kernel]] === I would like to customize my kernel. Is it difficult? Not at all! Check out the extref:{handbook}kernelconfig/[kernel config section of the Handbook]. [NOTE] ==== The new [.filename]#kernel# will be installed to the [.filename]#/boot/kernel# directory along with its modules, while the old kernel and its modules will be moved to the [.filename]#/boot/kernel.old# directory. If a mistake is made in the configuration, simply boot the previous version of the kernel. ==== [[why-kernel-big]] === Why is my kernel so big? `GENERIC` kernels shipped with FreeBSD are compiled in _debug mode_. Kernels built in debug mode contain debug data in separate files that are used for debugging. FreeBSD releases prior to 11.0 store these debug files in the same directory as the kernel itself, [.filename]#/boot/kernel/#. In FreeBSD 11.0 and later the debug files are stored in [.filename]#/usr/lib/debug/boot/kernel/#. Note that there will be little or no performance loss from running a debug kernel, and it is useful to keep one around in case of a system panic. When running low on disk space, there are different options to reduce the size of [.filename]#/boot/kernel/# and [.filename]#/usr/lib/debug/#. To not install the symbol files, make sure the following line exists in [.filename]#/etc/src.conf#: [.programlisting] .... WITHOUT_KERNEL_SYMBOLS=yes .... For more information see man:src.conf[5]. If you want to avoid building debug files altogether, make sure that both of the following are true: * This line does not exist in the kernel configuration file: + [.programlisting] .... makeoptions DEBUG=-g .... * Do not run man:config[8] with `-g`. Either of the above settings will cause the kernel to be built in debug mode. To build and install only the specified modules, list them in [.filename]#/etc/make.conf#: [.programlisting] .... MODULES_OVERRIDE= accf_http ipfw .... Replace _accf_httpd ipfw_ with a list of needed modules. Only the listed modules will be built. This reduces the size of the kernel directory and decreases the amount of time needed to build the kernel. For more information, read [.filename]#/usr/share/examples/etc/make.conf#. Unneeded devices can be removed from the kernel to further reduce the size. See <> for more information. To put any of these options into effect, follow the instructions to extref:{handbook}kernelconfig/[build and install, kernelconfig-building] the new kernel. For reference, the FreeBSD 11 amd64 kernel ([.filename]#/boot/kernel/kernel#) is approximately 25 MB. [[generic-kernel-build-failure]] === Why does every kernel I try to build fail to compile, even GENERIC? There are a number of possible causes for this problem: * The source tree is different from the one used to build the currently running system. When attempting an upgrade, read [.filename]#/usr/src/UPDATING#, paying particular attention to the "COMMON ITEMS" section at the end. * The `make buildkernel` did not complete successfully. The `make buildkernel` target relies on files generated by the `make buildworld` target to complete its job correctly. * Even when building <>, it is possible that the source tree was fetched at a time when it was either being modified or it was broken. Only releases are guaranteed to be buildable, although <> builds fine the majority of the time. Try re-fetching the source tree and see if the problem goes away. Try using a different mirror in case the previous one is having problems. [[scheduler-in-use]] === Which scheduler is in use on a running system? The name of the scheduler currently being used is directly available as the value of the `kern.sched.name` sysctl: [source,shell] .... % sysctl kern.sched.name kern.sched.name: ULE .... [[scheduler-kern-quantum]] === What is kern.sched.quantum? `kern.sched.quantum` is the maximum number of ticks a process can run without being preempted in the 4BSD scheduler. [[disks]] == Disks, File Systems, and Boot Loaders [[adding-disks]] === How can I add my new hard disk to my FreeBSD system? See the extref:{handbook}disks/[Adding Disks, disks-adding] section in the FreeBSD Handbook. [[new-huge-disk]] === How do I move my system over to my huge new disk? The best way is to reinstall the operating system on the new disk, then move the user data over. This is highly recommended when tracking _-STABLE_ for more than one release or when updating a release instead of installing a new one. Install booteasy on both disks with man:boot0cfg[8] and dual boot until you are happy with the new configuration. Skip the next paragraph to find out how to move the data after doing this. Alternatively, partition and label the new disk with either man:sade[8] or man:gpart[8]. If the disks are MBR-formatted, booteasy can be installed on both disks with man:boot0cfg[8] so that the computer can dual boot to the old or new system after the copying is done. Once the new disk set up, the data cannot just be copied. Instead, use tools that understand device files and system flags, such as man:dump[8]. Although it is recommended to move the data while in single-user mode, it is not required. When the disks are formatted with UFS, never use anything but man:dump[8] and man:restore[8] to move the root file system. These commands should also be used when moving a single partition to another empty partition. The sequence of steps to use `dump` to move the data from one UFS partitions to a new partition is: [.procedure] ==== . `newfs` the new partition. . `mount` it on a temporary mount point. . `cd` to that directory. . `dump` the old partition, piping output to the new one. ==== For example, to move [.filename]#/dev/ada1s1a# with [.filename]#/mnt# as the temporary mount point, type: [source,shell] .... # newfs /dev/ada1s1a # mount /dev/ada1s1a /mnt # cd /mnt # dump 0af - / | restore rf - .... Rearranging partitions with `dump` takes a bit more work. To merge a partition like [.filename]#/var# into its parent, create the new partition large enough for both, move the parent partition as described above, then move the child partition into the empty directory that the first move created: [source,shell] .... # newfs /dev/ada1s1a # mount /dev/ada1s1a /mnt # cd /mnt # dump 0af - / | restore rf - # cd var # dump 0af - /var | restore rf - .... To split a directory from its parent, say putting [.filename]#/var# on its own partition when it was not before, create both partitions, then mount the child partition on the appropriate directory in the temporary mount point, then move the old single partition: [source,shell] .... # newfs /dev/ada1s1a # newfs /dev/ada1s1d # mount /dev/ada1s1a /mnt # mkdir /mnt/var # mount /dev/ada1s1d /mnt/var # cd /mnt # dump 0af - / | restore rf - .... The man:cpio[1] and man:pax[1] utilities are also available for moving user data. These are known to lose file flag information, so use them with caution. [[safe-softupdates]] === Which partitions can safely use Soft Updates? I have heard that Soft Updates on / can cause problems. What about Journaled Soft Updates? Short answer: Soft Updates can usually be safely used on all partitions. Long answer: Soft Updates has two characteristics that may be undesirable on certain partitions. First, a Soft Updates partition has a small chance of losing data during a system crash. The partition will not be corrupted as the data will simply be lost. Second, Soft Updates can cause temporary space shortages. When using Soft Updates, the kernel can take up to thirty seconds to write changes to the physical disk. When a large file is deleted the file still resides on disk until the kernel actually performs the deletion. This can cause a very simple race condition. Suppose one large file is deleted and another large file is immediately created. The first large file is not yet actually removed from the physical disk, so the disk might not have enough room for the second large file. This will produce an error that the partition does not have enough space, even though a large chunk of space has just been released. A few seconds later, the file creation works as expected. If a system should crash after the kernel accepts a chunk of data for writing to disk, but before that data is actually written out, data could be lost. This risk is extremely small, but generally manageable. These issues affect all partitions using Soft Updates. So, what does this mean for the root partition? Vital information on the root partition changes very rarely. If the system crashed during the thirty-second window after such a change is made, it is possible that data could be lost. This risk is negligible for most applications, but be aware that it exists. If the system cannot tolerate this much risk, do not use Soft Updates on the root file system! [.filename]#/# is traditionally one of the smallest partitions. If [.filename]#/tmp# is on [.filename]#/#, there may be intermittent space problems. Symlinking [.filename]#/tmp# to [.filename]#/var/tmp# will solve this problem. Finally, man:dump[8] does not work in live mode (-L) on a filesystem, with Journaled Soft Updates (SU+J). [[mount-foreign-fs]] === Can I mount other foreign file systems under FreeBSD? FreeBSD supports a variety of other file systems. UFS:: UFS CD-ROMs can be mounted directly on FreeBSD. Mounting disk partitions from Digital UNIX and other systems that support UFS may be more complex, depending on the details of the disk partitioning for the operating system in question. ext2/ext3:: FreeBSD supports `ext2fs`, `ext3fs`, and `ext4fs` partitions. See man:ext2fs[5] for more information. NTFS:: FUSE based NTFS support is available as a port (package:sysutils/fusefs-ntfs[]). For more information, see man:ntfs-3g[8]. FAT:: FreeBSD includes a read-write FAT driver. For more information, see man:mount_msdosfs[8]. ZFS:: FreeBSD includes a port of Sun(TM)'s ZFS driver. The current recommendation is to use it only on amd64 platforms with sufficient memory. For more information, see man:zfs[8]. FreeBSD includes the Network File System NFS and the FreeBSD Ports Collection provides several FUSE applications to support many other file systems. [[mount-dos]] === How do I mount a secondary DOS partition? The secondary DOS partitions are found after _all_ the primary partitions. For example, if `E` is the second DOS partition on the second SCSI drive, there will be a device file for "slice 5" in [.filename]#/dev#. To mount it: [source,shell] .... # mount -t msdosfs /dev/da1s5 /dos/e .... [[crypto-file-system]] === Is there a cryptographic file system for FreeBSD? Yes, man:gbde[8] and man:geli[8]. See the extref:{handbook}disks/[Encrypting Disk Partitions, disks-encrypting] section of the FreeBSD Handbook. [[grub-loader]] === How do I boot FreeBSD and Linux(R) using GRUB? To boot FreeBSD using GRUB, add the following to either [.filename]#/boot/grub/menu.lst# or [.filename]#/boot/grub/grub.conf#, depending upon which is used by the Linux(R) distribution. [.programlisting] .... title FreeBSD 9.1 root (hd0,a) kernel /boot/loader .... Where _hd0,a_ points to the root partition on the first disk. To specify the slice number, use something like this _(hd0,2,a)_. By default, if the slice number is omitted, GRUB searches the first slice which has the `a` partition. [[booteasy-loader]] === How do I boot FreeBSD and Linux(R) using BootEasy? Install LILO at the start of the Linux(R) boot partition instead of in the Master Boot Record. Then boot LILO from BootEasy. This is recommended when running Windows(R) and Linux(R) as it makes it simpler to get Linux(R) booting again if Windows(R) is reinstalled. [[changing-bootprompt]] === How do I change the boot prompt from ??? to something more meaningful? This cannot be accomplished with the standard boot manager without rewriting it. There are a number of other boot managers in the [.filename]#sysutils# category of the Ports Collection. [[removable-drives]] === How do I use a new removable drive? If the drive already has a file system on it, use a command like this: [source,shell] .... # mount -t msdosfs /dev/da0s1 /mnt .... If the drive will only be used with FreeBSD systems, partition it with UFS or ZFS. This will provide long filename support, improvement in performance, and stability. If the drive will be used by other operating systems, a more portable choice, such as msdosfs, is better. [source,shell] .... # dd if=/dev/zero of=/dev/da0 count=2 # gpart create -s GPT /dev/da0 # gpart add -t freebsd-ufs /dev/da0 .... Finally, create a new file system: [source,shell] .... # newfs /dev/da0p1 .... and mount it: [source,shell] .... # mount /dev/da0s1 /mnt .... It is a good idea to add a line to [.filename]#/etc/fstab# (see man:fstab[5]) so you can just type `mount /mnt` in the future: [.programlisting] .... /dev/da0p1 /mnt ufs rw,noauto 0 0 .... [[mount-cd-superblock]] === Why do I get Incorrect super block when mounting a CD? The type of device to mount must be specified. This is described in the Handbook section on extref:{handbook}disks/[Using Data CDs, mounting-cd]. [[cdrom-not-configured]] === Why do I get Device not configured when mounting a CD? This generally means that there is no CD in the drive, or the drive is not visible on the bus. Refer to the extref:{handbook}disks/[Using Data CDs, mounting-cd] section of the Handbook for a detailed discussion of this issue. [[cdrom-unicode-filenames]] === Why do all non-English characters in filenames show up as ? on my CDs when mounted in FreeBSD? The CD probably uses the "Joliet" extension for storing information about files and directories. This is discussed in the Handbook section on extref:{handbook}disks/[Using Data CD-ROMs, mounting-cd]. [[burncd-isofs]] === A CD burned under FreeBSD cannot be read under any other operating system. Why? This means a raw file was burned to the CD, rather than creating an ISO 9660 file system. Take a look at the Handbook section on extref:{handbook}disks/[Using Data CDs]. [[copy-cd]] === How can I create an image of a data CD? This is discussed in the Handbook section on extref:{handbook}disks/[Writing Data to an ISO File System, mkisofs]. For more on working with CD-ROMs, see the extref:{handbook}disks/[Creating CDs Section, creating-cds] in the Storage chapter in the Handbook. [[mount-audio-CD]] === Why can I not mount an audio CD? Trying to mount an audio CD will produce an error like `cd9660: /dev/cd0: Invalid argument`. This is because `mount` only works on file systems. Audio CDs do not have file systems; they just have data. Instead, use a program that reads audio CDs, such as the package:audio/xmcd[] package or port. [[multi-session-CD]] === How do I mount a multi-session CD? By default, man:mount[8] will attempt to mount the last data track (session) of a CD. To load an earlier session, use the `-s` command line argument. Refer to man:mount_cd9660[8] for specific examples. [[user-floppymount]] === How do I let ordinary users mount CD-ROMs, DVDs, USB drives, and other removable media? As `root` set the sysctl variable `vfs.usermount` to `1`. [source,shell] .... # sysctl vfs.usermount=1 .... To make this persist across reboots, add the line `vfs.usermount=1` to [.filename]#/etc/sysctl.conf# so that it is reset at system boot time. Users can only mount devices they have read permissions to. To allow users to mount a device permissions must be set in [.filename]#/etc/devfs.conf#. For example, to allow users to mount the first USB drive add: [.programlisting] .... # Allow all users to mount a USB drive. own /dev/da0 root:operator perm /dev/da0 0666 .... All users can now mount devices they could read onto a directory that they own: [source,shell] .... % mkdir ~/my-mount-point % mount -t msdosfs /dev/da0 ~/my-mount-point .... Unmounting the device is simple: [source,shell] .... % umount ~/my-mount-point .... Enabling `vfs.usermount`, however, has negative security implications. A better way to access MS-DOS(R) formatted media is to use the package:emulators/mtools[] package in the Ports Collection. [NOTE] ==== The device name used in the previous examples must be changed according to the configuration. ==== [[du-vs-df]] === The du and df commands show different amounts of disk space available. What is going on? This is due to how these commands actually work. `du` goes through the directory tree, measures how large each file is, and presents the totals. `df` just asks the file system how much space it has left. They seem to be the same thing, but a file without a directory entry will affect `df` but not `du`. When a program is using a file, and the file is deleted, the file is not really removed from the file system until the program stops using it. The file is immediately deleted from the directory listing, however. As an example, consider a file large enough to affect the output of `du` and `df`. A file being viewed with `more` can be deleted without causing an error. The entry is removed from the directory so no other program or user can access it. However, `du` shows that it is gone as it has walked the directory tree and the file is not listed. `df` shows that it is still there, as the file system knows that `more` is still using that space. Once the `more` session ends, `du` and `df` will agree. This situation is common on web servers. Many people set up a FreeBSD web server and forget to rotate the log files. The access log fills up [.filename]#/var#. The new administrator deletes the file, but the system still complains that the partition is full. Stopping and restarting the web server program would free the file, allowing the system to release the disk space. To prevent this from happening, set up man:newsyslog[8]. Note that Soft Updates can delay the freeing of disk space and it can take up to 30 seconds for the change to be visible. [[add-swap-space]] === How can I add more swap space? This section extref:{handbook}config/[of the Handbook, adding-swap-space] describes how to do this. [[manufacturer-disk-size]] === Why does FreeBSD see my disk as smaller than the manufacturer says it is? Disk manufacturers calculate gigabytes as a billion bytes each, whereas FreeBSD calculates them as 1,073,741,824 bytes each. This explains why, for example, FreeBSD's boot messages will report a disk that supposedly has 80 GB as holding 76,319 MB. Also note that FreeBSD will (by default) <> 8% of the disk space. [[disk-more-than-full]] === How is it possible for a partition to be more than 100% full? A portion of each UFS partition (8%, by default) is reserved for use by the operating system and the `root` user. man:df[1] does not count that space when calculating the `Capacity` column, so it can exceed 100%. Notice that the `Blocks` column is always greater than the sum of the `Used` and `Avail` columns, usually by a factor of 8%. For more details, look up `-m` in man:tunefs[8]. [[all-about-zfs]] == ZFS [[how-much-ram-for-zfs]] === What is the minimum amount of RAM one should have to run ZFS? A minimum of 4GB of RAM is required for comfortable usage, but individual workloads can vary widely. [[what-is-zil]] === What is the ZIL and when does it get used? The ZIL (ZFS intent log) is a write log used to implement posix write commitment semantics across crashes. Normally writes are bundled up into transaction groups and written to disk when filled ("Transaction Group Commit"). However syscalls like man:fsync[2] require a commitment that the data is written to stable storage before returning. The ZIL is needed for writes that have been acknowledged as written but which are not yet on disk as part of a transaction. The transaction groups are timestamped. In the event of a crash the last valid timestamp is found and missing data is merged in from the ZIL. [[need-ssd-for-zil]] === Do I need a SSD for ZIL? By default, ZFS stores the ZIL in the pool with all the data. If an application has a heavy write load, storing the ZIL in a separate device that has very fast synchronous, sequential write performance can improve overall system performance. For other workloads, a SSD is unlikely to make much of an improvement. [[what-is-l2arc]] === What is the L2ARC? The L2ARC is a read cache stored on a fast device such as an SSD. This cache is not persistent across reboots. Note that RAM is used as the first layer of cache and the L2ARC is only needed if there is insufficient RAM. L2ARC needs space in the ARC to index it. So, perversely, a working set that fits perfectly in the ARC will not fit perfectly any more if a L2ARC is used because part of the ARC is holding the L2ARC index, pushing part of the working set into the L2ARC which is slower than RAM. [[should-enable-dedup]] === Is enabling deduplication advisable? Generally speaking, no. Deduplication takes up a significant amount of RAM and may slow down read and write disk access times. Unless one is storing data that is very heavily duplicated, such as virtual machine images or user backups, it is possible that deduplication will do more harm than good. Another consideration is the inability to revert deduplication status. If data is written when deduplication is enabled, disabling dedup will not cause those blocks which were deduplicated to be replicated until they are next modified. Deduplication can also lead to some unexpected situations. In particular, deleting files may become much slower. [[zpool-fully-full]] === I cannot delete or create files on my ZFS pool. How can I fix this? This could happen because the pool is 100% full. ZFS requires space on the disk to write transaction metadata. To restore the pool to a usable state, truncate the file to delete: [source,shell] .... % truncate -s 0 unimportant-file .... File truncation works because a new transaction is not started, new spare blocks are created instead. [NOTE] ==== On systems with additional ZFS dataset tuning, such as deduplication, the space may not be immediately available. ==== [[zfs-ssd-trim]] === Is there TRIM support for ZFS on solid state drives? ZFS in FreeBSD 12.3 and 12.4: TRIM is enabled by default. To disable TRIM: add the line below to [.filename]#/etc/sysctl.conf#, then restart the system. [.programlisting] .... vfs.zfs.trim.enabled=0 .... OpenZFS in FreeBSD 13.0 and greater: see man:zpool-trim[8], and `autotrim` in man:zpoolprops[7]. [[admin]] == System Administration [[startup-config-files]] === Where are the system start-up configuration files? The primary configuration file is [.filename]#/etc/defaults/rc.conf# which is described in man:rc.conf[5]. System startup scripts such as [.filename]#/etc/rc# and [.filename]#/etc/rc.d#, which are described in man:rc[8], include this file. _Do not edit this file!_ Instead, to edit an entry in [.filename]#/etc/defaults/rc.conf#, copy the line into [.filename]#/etc/rc.conf# and change it there. For example, to start man:sshd[8], the included OpenSSH daemon: [source,shell] .... # echo 'sshd_enable="YES"' >> /etc/rc.conf .... Alternatively, use man:sysrc[8] to modify [.filename]#/etc/rc.conf#: [source,shell] .... # sysrc sshd_enable="YES" .... To start up local services, place shell scripts in the [.filename]#/usr/local/etc/rc.d# directory. These shell scripts should be set executable, the default file mode is `555`. [[adding-users]] === How do I add a user easily? Use the man:adduser[8] command, or the man:pw[8] command for more complicated situations. To remove the user, use the man:rmuser[8] command or, if necessary, man:pw[8]. [[root-not-found-cron-errors]] === Why do I keep getting messages like root: not found after editing /etc/crontab? This is normally caused by editing the system crontab. This is not the correct way to do things as the system crontab has a different format to the per-user crontabs. The system crontab has an extra field, specifying which user to run the command as. man:cron[8] assumes this user is the first word of the command to execute. Since no such command exists, this error message is displayed. To delete the extra, incorrect crontab: [source,shell] .... # crontab -r .... [[su-wheel-group]] === Why do I get the error, you are not in the correct group to su root when I try to su to root? This is a security feature. In order to `su` to `root`, or any other account with superuser privileges, the user account must be a member of the `wheel` group. If this feature were not there, anybody with an account on a system who also found out ``root``'s password would be able to gain superuser level access to the system. To allow someone to `su` to `root`, put them in the `wheel` group using `pw`: [source,shell] .... # pw groupmod wheel -m lisa .... The above example will add user `lisa` to the group `wheel`. [[rcconf-readonly]] === I made a mistake in rc.conf, or another startup file, and now I cannot edit it because the file system is read-only. What should I do? Restart the system using `boot -s` at the loader prompt to enter single-user mode. When prompted for a shell pathname, press kbd:[Enter] and run `mount -urw /` to re-mount the root file system in read/write mode. You may also need to run `mount -a -t ufs` to mount the file system where your favorite editor is defined. If that editor is on a network file system, either configure the network manually before mounting the network file systems, or use an editor which resides on a local file system, such as man:ed[1]. In order to use a full screen editor such as man:vi[1] or man:emacs[1], run `export TERM=xterm` so that these editors can load the correct data from the man:termcap[5] database. After performing these steps, edit [.filename]#/etc/rc.conf# to fix the syntax error. The error message displayed immediately after the kernel boot messages should indicate the number of the line in the file which is at fault. [[printer-setup]] === Why am I having trouble setting up my printer? See the extref:{handbook}printing/[Handbook entry on printing] for troubleshooting tips. [[keyboard-mappings]] === How can I correct the keyboard mappings for my system? Refer to the Handbook section on extref:{handbook}l10n/[using localization, using-localization], specifically the section on extref:{handbook}l10n/[console setup, setting-console]. [[user-quotas]] === Why can I not get user quotas to work properly? . It is possible that the kernel is not configured to use quotas. In this case, add the following line to the kernel configuration file and recompile the kernel: + [.programlisting] .... options QUOTA .... + Refer to the extref:{handbook}disks/[Handbook entry on quotas, quotas] for full details. . Do not turn on quotas on [.filename]#/#. . Put the quota file on the file system that the quotas are to be enforced on: + [.informaltable] [cols="1,1", frame="none", options="header"] |=== | File System | Quota file |[.filename]#/usr# |[.filename]#/usr/admin/quotas# |[.filename]#/home# |[.filename]#/home/admin/quotas# |... |... |=== [[sysv-ipc]] === Does FreeBSD support System V IPC primitives? Yes, FreeBSD supports System V-style IPC, including shared memory, messages and semaphores, in the [.filename]#GENERIC# kernel. With a custom kernel, support may be loaded with the [.filename]#sysvshm.ko#, [.filename]#sysvsem.ko# and [.filename]#sysvmsg.ko# kernel modules, or enabled in the custom kernel by adding the following lines to the kernel configuration file: [.programlisting] .... options SYSVSHM # enable shared memory options SYSVSEM # enable for semaphores options SYSVMSG # enable for messaging .... Recompile and install the kernel. [[sendmail-alternative]] === What other mail-server software can I use instead of Sendmail? The http://www.sendmail.org/[Sendmail] server is the default mail-server software for FreeBSD, but it can be replaced with another MTA installed from the Ports Collection. Available ports include package:mail/exim[], package:mail/postfix[], and package:mail/qmail[]. Search the mailing lists for discussions regarding the advantages and disadvantages of the available MTAs. [[forgot-root-pw]] === I have forgotten the root password! What do I do? Do not panic! Restart the system, type `boot -s` at the `Boot:` prompt to enter single-user mode. At the question about the shell to use, hit kbd:[Enter] which will display a # prompt. Enter `mount -urw /` to remount the root file system read/write, then run `mount -a` to remount all the file systems. Run `passwd root` to change the `root` password then run man:exit[1] to continue booting. [NOTE] ==== If you are still prompted to give the `root` password when entering the single-user mode, it means that the console has been marked as `insecure` in [.filename]#/etc/ttys#. In this case, it will be required to boot from a FreeBSD installation disk, choose the [.guimenuitem]#Live CD# or [.guimenuitem]#Shell# at the beginning of the install process and issue the commands mentioned above. Mount the specific partition in this case and then chroot to it. For example, replace `mount -urw /` with `mount /dev/ada0p1 /mnt; chroot /mnt` for a system on _ada0p1_. ==== [NOTE] ==== If the root partition cannot be mounted from single-user mode, it is possible that the partitions are encrypted and it is impossible to mount them without the access keys. For more information see the section about encrypted disks in the FreeBSD extref:{handbook}disks/[Handbook, disks-encrypting]. ==== [[CAD-reboot]] === How do I keep kbd:[Control] + kbd:[Alt] + kbd:[Delete] from rebooting the system? When using man:vt[4], the default console driver, this can be done by setting the following man:sysctl[8]: [source,shell] .... # sysctl kern.vt.kbd_reboot=0 .... [[dos-to-unix-txt]] === How do I reformat DOS text files to UNIX(R) ones? Use this man:perl[1] command: [source,shell] .... % perl -i.bak -npe 's/\r\n/\n/g' file(s) .... where _file(s)_ is one or more files to process. The modification is done in-place, with the original file stored with a [.filename]#.bak# extension. Alternatively, use man:tr[1]: [source,shell] .... % tr -d '\r' < dos-text-file > unix-file .... _dos-text-file_ is the file containing DOS text while _unix-file_ will contain the converted output. This can be quite a bit faster than using `perl`. Yet another way to reformat DOS text files is to use the package:converters/dosunix[] port from the Ports Collection. Consult its documentation about the details. [[reread-rc]] === How do I re-read [.filename]#/etc/rc.conf# and re-start [.filename]#/etc/rc# without a reboot? Go into single-user mode and then back to multi-user mode: [source,shell] .... # shutdown now # return # exit .... [[release-candidate]] === I tried to update my system to the latest _-STABLE_, but got _-BETAx_, _-RC_ or __-PRERELEASE__! What is going on? Short answer: it is just a name. _RC_ stands for "Release Candidate". It signifies that a release is imminent. In FreeBSD, _-PRERELEASE_ is typically synonymous with the code freeze before a release. (For some releases, the _-BETA_ label was used in the same way as _-PRERELEASE_.) Long answer: FreeBSD derives its releases from one of two places. Major, dot-zero, releases, such as 9.0-RELEASE are branched from the head of the development stream, commonly referred to as <>. Minor releases, such as 6.3-RELEASE or 5.2-RELEASE, have been snapshots of the active <> branch. Starting with 4.3-RELEASE, each release also now has its own branch which can be tracked by people requiring an extremely conservative rate of development (typically only security advisories). When a release is about to be made, the branch from which it will be derived from has to undergo a certain process. Part of this process is a code freeze. When a code freeze is initiated, the name of the branch is changed to reflect that it is about to become a release. For example, if the branch used to be called 6.2-STABLE, its name will be changed to 6.3-PRERELEASE to signify the code freeze and signify that extra pre-release testing should be happening. Bug fixes can still be committed to be part of the release. When the source code is in shape for the release the name will be changed to 6.3-RC to signify that a release is about to be made from it. Once in the RC stage, only the most critical bugs found can be fixed. Once the release (6.3-RELEASE in this example) and release branch have been made, the branch will be renamed to 6.3-STABLE. For more information on version numbers and the various Subversion branches, refer to the extref:{releng}[Release Engineering] article. [[kernel-chflag-failure]] === I tried to install a new kernel, and the man:chflags[1] failed. How do I get around this? Short answer: the security level is greater than 0. Reboot directly to single-user mode to install the kernel. Long answer: FreeBSD disallows changing system flags at security levels greater than 0. To check the current security level: [source,shell] .... # sysctl kern.securelevel .... The security level cannot be lowered in multi-user mode, so boot to single-user mode to install the kernel, or change the security level in [.filename]#/etc/rc.conf# then reboot. See the man:init[8] manual page for details on `securelevel`, and see [.filename]#/etc/defaults/rc.conf# and the man:rc.conf[5] manual page for more information on [.filename]#rc.conf#. [[kernel-securelevel-time]] === I cannot change the time on my system by more than one second! How do I get around this? Short answer: the system is at a security level greater than 1. Reboot directly to single-user mode to change the date. Long answer: FreeBSD disallows changing the time by more that one second at security levels greater than 1. To check the security level: [source,shell] .... # sysctl kern.securelevel .... The security level cannot be lowered in multi-user mode. Either boot to single-user mode to change the date or change the security level in [.filename]#/etc/rc.conf# and reboot. See the man:init[8] manual page for details on `securelevel`, and see [.filename]#/etc/defaults/rc.conf# and the man:rc.conf[5] manual page for more information on [.filename]#rc.conf#. [[statd-mem-leak]] === Why is rpc.statd using 256 MB of memory? No, there is no memory leak, and it is not using 256 MB of memory. For convenience, `rpc.statd` maps a large amount of memory into its address space. There is nothing terribly wrong with this from a technical standpoint; it just throws off things like man:top[1] and man:ps[1]. man:rpc.statd[8] maps its status file ([.filename]#/var/db/statd.status#) into its address space; to save worrying about remapping the status file later when it needs to grow, it maps the status file with a generous size. [[unsetting-schg]] === Why can I not unset the schg file flag? The system is running at securelevel greater than 0. Lower the securelevel and try again. For more information, see <> and the man:init[8] manual page. [[vnlru]] === What is vnlru? `vnlru` flushes and frees vnodes when the system hits the `kern.maxvnodes` limit. This kernel thread sits mostly idle, and only activates when there is a huge amount of RAM and users are accessing tens of thousands of tiny files. [[top-memory-states]] === What do the various memory states displayed by top mean? * `Active`: pages recently statistically used. * `Inactive`: pages recently statistically unused. * `Laundry`: pages recently statistically unused but known to be dirty, that is, whose contents needs to be paged out before they can be reused. * `Free`: pages without data content, which can be immediately reused. * `Wired`: pages that are fixed into memory, usually for kernel purposes, but also sometimes for special use in processes. Pages are most often written to disk (sort of a VM sync) when they are in the laundry state, but active or inactive pages can also be synced. This depends upon the CPU tracking of the modified bit being available, and in certain situations there can be an advantage for a block of VM pages to be synced, regardless of the queue they belong to. In most common cases, it is best to think of the laundry queue as a queue of relatively unused pages that might or might not be in the process of being written to disk. The inactive queue contains a mix of clean and dirty pages; clean pages near the head of the queue are reclaimed immediately to alleviate a free page shortage, and dirty pages are moved to the laundry queue for deferred processing. There are some other flags (e.g., busy flag or busy count) that might modify some of the described rules. [[free-memory-amount]] === How much free memory is available? There are a couple of kinds of "free memory". The most common is the amount of memory immediately available without reclaiming memory already in use. That is the size of the free pages queue plus some other reserved pages. This amount is exported by the `vm.stats.vm.v_free_count` man:sysctl[8], shown, for instance, by man:top[1]. Another kind of "free memory" is the total amount of virtual memory available to userland processes, which depends on the sum of swap space and usable memory. Other kinds of "free memory" descriptions are also possible, but it is relatively useless to define these, but rather it is important to make sure that the paging rate is kept low, and to avoid running out of swap space. [[var-empty]] === What is [.filename]#/var/empty#? [.filename]#/var/empty# is a directory that the man:sshd[8] program uses when performing privilege separation. The [.filename]#/var/empty# directory is empty, owned by `root` and has the `schg` flag set. This directory should not be deleted. [[newsyslog-expectations]] === I just changed [.filename]#/etc/newsyslog.conf#. How can I check if it does what I expect? To see what man:newsyslog[8] will do, use the following: [source,shell] .... % newsyslog -nrvv .... [[timezone]] === My time is wrong, how can I change the timezone? Use man:tzsetup[8]. [[X11]] == The X Window System and Virtual Consoles [[whatis-X]] === What is the X Window System? The X Window System (commonly `X11`) is the most widely available windowing system capable of running on UNIX(R) or UNIX(R) like systems, including FreeBSD. http://www.x.org/wiki/[The X.Org Foundation] administers the http://en.wikipedia.org/wiki/X_Window_System_core_protocol[X protocol standards], with the current reference implementation, version 11 release 7.7, so references are often shortened to `X11`. Many implementations are available for different architectures and operating systems. An implementation of the server-side code is properly known as an `X server`. [[running-X]] === I want to run Xorg, how do I go about it? To install Xorg do one of the following: Use the package:x11/xorg[] meta-port, which builds and installs every Xorg component. Use package:x11/xorg-minimal[], which builds and installs only the necessary Xorg components. Install Xorg from FreeBSD packages: [source,shell] .... # pkg install xorg .... After the installation of Xorg, follow the instructions from the extref:{handbook}x11/[X11 Configuration, x-config] section of the FreeBSD Handbook. [[running-X-securelevels]] === I tried to run X, but I get a 'No devices detected' error when I type startx. What do I do now? The system is probably running at a raised `securelevel`. It is not possible to start X at a raised `securelevel` because X requires write access to man:io[4]. For more information, see at the man:init[8] manual page. There are two solutions to the problem: set the `securelevel` back down to zero or run man:xdm[8] (or an alternative display manager) at boot time before the `securelevel` is raised. See <> for more information about running man:xdm[8] at boot time. [[x-and-moused]] === Why does my mouse not work with X? When using man:vt[4], the default console driver, FreeBSD can be configured to support a mouse pointer on each virtual screen. To avoid conflicting with X, man:vt[4] supports a virtual device called [.filename]#/dev/sysmouse#. All mouse events received from the real mouse device are written to the man:sysmouse[4] device via man:moused[8]. To use the mouse on one or more virtual consoles, _and_ use X, see <> and set up man:moused[8]. Then edit [.filename]#/etc/X11/xorg.conf# and make sure the following lines exist: [.programlisting] .... Section "InputDevice" Option "Protocol" "SysMouse" Option "Device" "/dev/sysmouse" ..... .... Starting with Xorg version 7.4, the `InputDevice` sections in [.filename]#xorg.conf# are ignored in favor of autodetected devices. To restore the old behavior, add the following line to the `ServerLayout` or `ServerFlags` section: [.programlisting] .... Option "AutoAddDevices" "false" .... Some people prefer to use [.filename]#/dev/mouse# under X. To make this work, [.filename]#/dev/mouse# should be linked to [.filename]#/dev/sysmouse# (see man:sysmouse[4]) by adding the following line to [.filename]#/etc/devfs.conf# (see man:devfs.conf[5]): [.programlisting] .... link sysmouse mouse .... This link can be created by restarting man:devfs[5] with the following command (as `root`): [source,shell] .... # service devfs restart .... [[x-and-wheel]] === My mouse has a fancy wheel. Can I use it in X? Yes, if X is configured for a 5 button mouse. To do this, add the lines `Buttons 5` and `ZAxisMapping 4 5` to the "InputDevice" section of [.filename]#/etc/X11/xorg.conf#, as seen in this example: [.programlisting] .... Section "InputDevice" Identifier "Mouse1" Driver "mouse" Option "Protocol" "auto" Option "Device" "/dev/sysmouse" Option "Buttons" "5" Option "ZAxisMapping" "4 5" EndSection .... The mouse can be enabled in Emacs by adding these lines to [.filename]#~/.emacs#: [.programlisting] .... ;; wheel mouse (global-set-key [mouse-4] 'scroll-down) (global-set-key [mouse-5] 'scroll-up) .... [[x-and-synaptic]] === My laptop has a Synaptics touchpad. Can I use it in X? Yes, after configuring a few things to make it work. In order to use the Xorg synaptics driver, first remove `moused_enable` from [.filename]#rc.conf#. To enable synaptics, add the following line to [.filename]#/boot/loader.conf#: [.programlisting] .... hw.psm.synaptics_support="1" .... Add the following to [.filename]#/etc/X11/xorg.conf#: [.programlisting] .... Section "InputDevice" Identifier "Touchpad0" Driver "synaptics" Option "Protocol" "psm" Option "Device" "/dev/psm0" EndSection .... Add the following into the "ServerLayout" section: [.programlisting] .... InputDevice "Touchpad0" "SendCoreEvents" .... [[no-remote-x11]] === How do I use remote X displays? For security reasons, the default setting is to not allow a machine to remotely open a window. To enable this feature, start X with the optional `-listen_tcp` argument: [source,shell] .... % startx -listen_tcp .... [[virtual-console]] === What is a virtual console and how do I make more? Virtual consoles provide several simultaneous sessions on the same machine without doing anything complicated like setting up a network or running X. When the system starts, it will display a login prompt on the monitor after displaying all the boot messages. Type in your login name and password to start working on the first virtual console. To start another session, perhaps to look at documentation for a program or to read mail while waiting for an FTP transfer to finish, hold down kbd:[Alt] and press kbd:[F2]. This will display the login prompt for the second virtual console. To go back to the original session, press kbd:[Alt+F1]. The default FreeBSD installation has eight virtual consoles enabled. kbd:[Alt+F1], kbd:[Alt+F2], kbd:[Alt+F3], and so on will switch between these virtual consoles. To enable more of virtual consoles, edit [.filename]#/etc/ttys# (see man:ttys[5]) and add entries for [.filename]#ttyv8# to [.filename]#ttyvc#, after the comment on "Virtual terminals": [.programlisting] .... # Edit the existing entry for ttyv8 in /etc/ttys and change # "off" to "on". ttyv8 "/usr/libexec/getty Pc" xterm on secure ttyv9 "/usr/libexec/getty Pc" xterm on secure ttyva "/usr/libexec/getty Pc" xterm on secure ttyvb "/usr/libexec/getty Pc" xterm on secure .... The more virtual terminals, the more resources that are used. This can be problematic on systems with 8 MB RAM or less. Consider changing `secure` to `insecure`. [IMPORTANT] ==== In order to run an X server, at least one virtual terminal must be left to `off` for it to use. This means that only eleven of the Alt-function keys can be used as virtual consoles so that one is left for the X server. ==== For example, to run X and eleven virtual consoles, the setting for virtual terminal 12 should be: [.programlisting] .... ttyvb "/usr/libexec/getty Pc" xterm off secure .... The easiest way to activate the virtual consoles is to reboot. [[vty-from-x]] === How do I access the virtual consoles from X? Use kbd:[Ctrl+Alt+Fn] to switch back to a virtual console. Press kbd:[Ctrl+Alt+F1] to return to the first virtual console. Once at a text console, use kbd:[Alt+Fn] to move between them. To return to the X session, switch to the virtual console running X. If X was started from the command line using `startx`, the X session will attach to the next unused virtual console, not the text console from which it was invoked. For eight active virtual terminals, X will run on the ninth, so use kbd:[Alt+F9]. [[xdm-boot]] === How do I start XDM on boot? There are two schools of thought on how to start man:xdm[8]. One school starts `xdm` from [.filename]#/etc/ttys# (see man:ttys[5]) using the supplied example, while the other sets `xdm_enable=yes` in [.filename]#/etc/rc.conf#. Both are equally valid, and one may work in situations where the other does not. In both cases the result is the same: X will pop up a graphical login prompt. The man:ttys[5] method has the advantage of documenting which vty X will start on and passing the responsibility of restarting the X server on logout to man:init[8]. The man:rc[8] method makes it easy to `kill xdm` if there is a problem starting the X server. When using the man:rc[8] method, `xdm_tty` (default `ttyv8`) can be set in [.filename]#/etc/rc.conf# to choose which vty man:xdm[8] opens on. [[xconsole-failure]] === Why do I get Couldn't open console when I run xconsole? When X is started with `startx`, the permissions on [.filename]#/dev/console# will _not_ get changed, resulting in things like `xterm -C` and `xconsole` not working. This is because of the way console permissions are set by default. On a multi-user system, one does not necessarily want just any user to be able to write on the system console. For users who are logging directly onto a machine with a VTY, the man:fbtab[5] file exists to solve such problems. In a nutshell, make sure an uncommented line of the form is in [.filename]#/etc/fbtab# (see man:fbtab[5]): [.programlisting] .... /dev/ttyv0 0600 /dev/console .... It will ensure that whomever logs in on [.filename]#/dev/ttyv0# will own the console. [[ps2-x]] === Why does my PS/2 mouse misbehave under X? The mouse and the mouse driver may have become out of synchronization. In rare cases, the driver may also erroneously report synchronization errors: [.programlisting] .... psmintr: out of sync (xxxx != yyyy) .... If this happens, disable the synchronization check code by setting the driver flags for the PS/2 mouse driver to `0x100`. This can be easiest achieved by adding `hint.psm.0.flags="0x100"` to [.filename]#/boot/loader.conf# and rebooting. [[mouse-button-reverse]] === How do I reverse the mouse buttons? Type `xmodmap -e "pointer = 3 2 1"`. Add this command to [.filename]#~/.xinitrc# or [.filename]#~/.xsession# to make it happen automatically. [[install-splash]] === How do I install a splash screen and where do I find them? The detailed answer for this question can be found in the extref:{handbook}[Boot Time Splash Screens, boot-splash] section of the FreeBSD Handbook. [[windows-keys]] === Can I use the kbd:[Windows] keys on my keyboard in X? Yes. Use man:xmodmap[1] to define which functions the keys should perform. Assuming all Windows keyboards are standard, the keycodes for these three keys are the following: * 115 - kbd:[Windows] key, between the left-hand kbd:[Ctrl] and kbd:[Alt] keys * 116 - kbd:[Windows] key, to the right of kbd:[AltGr] * 117 - kbd:[Menu], to the left of the right-hand kbd:[Ctrl] To have the left kbd:[Windows] key print a comma, try this. [source,shell] .... # xmodmap -e "keycode 115 = comma" .... To have the kbd:[Windows] key-mappings enabled automatically every time X is started, either put the `xmodmap` commands in [.filename]#~/.xinitrc# or, preferably, create a [.filename]#~/.xmodmaprc# and include the `xmodmap` options, one per line, then add the following line to [.filename]#~/.xinitrc#: [.programlisting] .... xmodmap $HOME/.xmodmaprc .... For example, to map the 3 keys to be kbd:[F13], kbd:[F14], and kbd:[F15], respectively. This would make it easy to map them to useful functions within applications or the window manager. To do this, put the following in [.filename]#~/.xmodmaprc#. [.programlisting] .... keycode 115 = F13 keycode 116 = F14 keycode 117 = F15 .... For the package:x11-wm/fvwm2[] desktop manager, one could map the keys so that kbd:[F13] iconifies or de-iconifies the window the cursor is in, kbd:[F14] brings the window the cursor is in to the front or, if it is already at the front, pushes it to the back, and kbd:[F15] pops up the main Workplace menu even if the cursor is not on the desktop, which is useful when no part of the desktop is visible. The following entries in [.filename]#~/.fvwmrc# implement the aforementioned setup: [.programlisting] .... Key F13 FTIWS A Iconify Key F14 FTIWS A RaiseLower Key F15 A A Menu Workplace Nop .... [[x-3d-acceleration]] === How can I get 3D hardware acceleration for OpenGL(R)? The availability of 3D acceleration depends on the version of Xorg and the type of video chip. For an nVidia chip, use the binary drivers provided for FreeBSD by installing one of the following ports: The latest versions of nVidia cards are supported by the package:x11/nvidia-driver[] port. Older drivers are available as: * package:x11/nvidia-driver-390[] * package:x11/nvidia-driver-340[] * package:x11/nvidia-driver-304[] nVidia provides detailed information on which card is supported by which driver on their web site: http://www.nvidia.com/object/IO_32667.html[http://www.nvidia.com/object/IO_32667.html]. For Matrox G200/G400, check the package:x11-drivers/xf86-video-mga[] port. For ATI Rage 128 and Radeon see man:ati[4], man:r128[4] and man:radeon[4]. [[networking]] == Networking [[diskless-booting]] === Where can I get information on diskless booting? "Diskless booting" means that the FreeBSD box is booted over a network, and reads the necessary files from a server instead of its hard disk. For full details, see extref:{handbook}advanced-networking/[the Handbook entry on diskless booting, network-diskless]. [[router]] === Can a FreeBSD box be used as a dedicated network router? Yes. Refer to the Handbook entry on extref:{handbook}advanced-networking/[advanced networking, advanced-networking], specifically the section on extref:{handbook}advanced-networking/[routing and gateways, network-routing]. [[natd]] === Does FreeBSD support NAT or Masquerading? Yes. For instructions on how to use NAT over a PPP connection, see the extref:{handbook}ppp-and-slip/[Handbook entry on PPP, userppp]. To use NAT over some other sort of network connection, look at the extref:{handbook}[natd, network-natd] section of the Handbook. [[ethernet-aliases]] === How can I set up Ethernet aliases? If the alias is on the same subnet as an address already configured on the interface, add `netmask 0xffffffff` to this command: [source,shell] .... # ifconfig ed0 alias 192.0.2.2 netmask 0xffffffff .... Otherwise, specify the network address and netmask as usual: [source,shell] .... # ifconfig ed0 alias 172.16.141.5 netmask 0xffffff00 .... More information can be found in the FreeBSD extref:{handbook}config/[Handbook, configtuning-virtual-hosts]. [[nfs-linux]] === Why can I not NFS-mount from a Linux(R) box? Some versions of the Linux(R) NFS code only accept mount requests from a privileged port; try to issue the following command: [source,shell] .... # mount -o -P linuxbox:/blah /mnt .... [[exports-errors]] === Why does mountd keep telling me it can't change attributes and that I have a bad exports list on my FreeBSD NFS server? The most frequent problem is not understanding the correct format of [.filename]#/etc/exports#. Review man:exports[5] and the extref:{handbook}network-servers/[NFS, network-nfs] entry in the Handbook, especially the section on extref:{handbook}[configuring NFS, configuring-nfs]. [[ip-multicast]] === How do I enable IP multicast support? Install the package:net/mrouted[] package or port and add `mrouted_enable="YES"` to [.filename]#/etc/rc.conf# start this service at boot time. [[fqdn-hosts]] === Why do I have to use the FQDN for hosts on my site? See the answer in the FreeBSD extref:{handbook}mail/[Handbook, mail-trouble]. [[network-permission-denied]] === Why do I get an error, Permission denied, for all networking operations? If the kernel is compiled with the `IPFIREWALL` option, be aware that the default policy is to deny all packets that are not explicitly allowed. If the firewall is unintentionally misconfigured, restore network operability by typing the following as `root`: [source,shell] .... # ipfw add 65534 allow all from any to any .... Consider setting `firewall_type="open"` in [.filename]#/etc/rc.conf#. For further information on configuring this firewall, see the extref:{handbook}firewalls/[Handbook chapter, firewalls-ipfw]. [[ipfw-fwd]] === Why is my `ipfw` “fwd” rule to redirect a service to another machine not working? Possibly because network address translation (NAT) is needed instead of just forwarding packets. A "fwd" rule only forwards packets, it does not actually change the data inside the packet. Consider this rule: [source,shell] .... 01000 fwd 10.0.0.1 from any to foo 21 .... When a packet with a destination address of _foo_ arrives at the machine with this rule, the packet is forwarded to _10.0.0.1_, but it still has the destination address of _foo_. The destination address of the packet is not changed to _10.0.0.1_. Most machines would probably drop a packet that they receive with a destination address that is not their own. Therefore, using a "fwd" rule does not often work the way the user expects. This behavior is a feature and not a bug. See the <>, the man:natd[8] manual, or one of the several port redirecting utilities in the link:https://www.FreeBSD.org/ports/[Ports Collection] for a correct way to do this. [[service-redirect]] === How can I redirect service requests from one machine to another? FTP and other service requests can be redirected with the package:sysutils/socket[] package or port. Replace the entry for the service in [.filename]#/etc/inetd.conf# to call `socket`, as seen in this example for ftpd: [.programlisting] .... ftp stream tcp nowait nobody /usr/local/bin/socket socket ftp.example.com ftp .... where _ftp.example.com_ and _ftp_ are the host and port to redirect to, respectively. [[bandwidth-mgr-tool]] === Where can I get a bandwidth management tool? There are three bandwidth management tools available for FreeBSD. man:dummynet[4] is integrated into FreeBSD as part of man:ipfw[4]. http://www.sonycsl.co.jp/person/kjc/programs.html[ALTQ] has been integrated into FreeBSD as part of man:pf[4]. Bandwidth Manager from http://www.etinc.com/[Emerging Technologies] is a commercial product. [[bpf-not-configured]] === Why do I get /dev/bpf0: device not configured? The running application requires the Berkeley Packet Filter (man:bpf[4]), but it was removed from a custom kernel. Add this to the kernel config file and build a new kernel: [.programlisting] .... device bpf # Berkeley Packet Filter .... [[mount-smb-share]] === How do I mount a disk from a Windows(R) machine that is on my network, like smbmount in Linux(R)? Use the SMBFS toolset. It includes a set of kernel modifications and a set of userland programs. The programs and information are available as man:mount_smbfs[8] in the base system. [[icmp-response-bw-limit]] === What are these messages about: Limiting icmp/open port/closed port response in my log files? This kernel message indicates that some activity is provoking it to send a large amount of ICMP or TCP reset (RST) responses. ICMP responses are often generated as a result of attempted connections to unused UDP ports. TCP resets are generated as a result of attempted connections to unopened TCP ports. Among others, these are the kinds of activities which may cause these messages: * Brute-force denial of service (DoS) attacks (as opposed to single-packet attacks which exploit a specific vulnerability). * Port scans which attempt to connect to a large number of ports (as opposed to only trying a few well-known ports). The first number in the message indicates how many packets the kernel would have sent if the limit was not in place, and the second indicates the limit. This limit is controlled using `net.inet.icmp.icmplim`. This example sets the limit to `300` packets per second: [source,shell] .... # sysctl net.inet.icmp.icmplim=300 .... To disable these messages without disabling response limiting, use `net.inet.icmp.icmplim_output` to disable the output: [source,shell] .... # sysctl net.inet.icmp.icmplim_output=0 .... Finally, to disable response limiting completely, set `net.inet.icmp.icmplim` to `0`. Disabling response limiting is discouraged for the reasons listed above. [[unknown-hw-addr-format]] === What are these arp: unknown hardware address format error messages? This means that some device on the local Ethernet is using a MAC address in a format that FreeBSD does not recognize. This is probably caused by someone experimenting with an Ethernet card somewhere else on the network. This is most commonly seen on cable modem networks. It is harmless, and should not affect the performance of the FreeBSD system. [[arp-wrong-iface]] === Why do I keep seeing messages like: 192.168.0.10 is on fxp1 but got reply from 00:15:17:67:cf:82 on rl0, and how do I disable it? A packet is coming from outside the network unexpectedly. To disable them, set `net.link.ether.inet.log_arp_wrong_iface` to `0`. [[ipv6-only]] === How do I compile an IPv6 only kernel? Configure your kernel with these settings: [source,shell] .... include GENERIC ident GENERIC-IPV6ONLY makeoptions MKMODULESENV+="WITHOUT_INET_SUPPORT=" nooptions INET nodevice gre .... [[security]] == Security [[sandbox]] === What is a sandbox? "Sandbox" is a security term. It can mean two things: * A process which is placed inside a set of virtual walls that are designed to prevent someone who breaks into the process from being able to break into the wider system. + The process is only able to run inside the walls. Since nothing the process does in regards to executing code is supposed to be able to breach the walls, a detailed audit of its code is not needed in order to be able to say certain things about its security. + The walls might be a user ID, for example. This is the definition used in the man:security[7] and man:named[8] man pages. + Take the `ntalk` service, for example (see man:inetd[8]). This service used to run as user ID `root`. Now it runs as user ID `tty`. The `tty` user is a sandbox designed to make it more difficult for someone who has successfully hacked into the system via `ntalk` from being able to hack beyond that user ID. * A process which is placed inside a simulation of the machine. It means that someone who is able to break into the process may believe that he can break into the wider machine but is, in fact, only breaking into a simulation of that machine and not modifying any real data. + The most common way to accomplish this is to build a simulated environment in a subdirectory and then run the processes in that directory chrooted so that [.filename]#/# for that process is this directory, not the real [.filename]#/# of the system). + Another common use is to mount an underlying file system read-only and then create a file system layer on top of it that gives a process a seemingly writeable view into that file system. The process may believe it is able to write to those files, but only the process sees the effects - other processes in the system do not, necessarily. + An attempt is made to make this sort of sandbox so transparent that the user (or hacker) does not realize that he is sitting in it. UNIX(R) implements two core sandboxes. One is at the process level, and one is at the userid level. Every UNIX(R) process is completely firewalled off from every other UNIX(R) process. One process cannot modify the address space of another. A UNIX(R) process is owned by a particular userid. If the user ID is not the `root` user, it serves to firewall the process off from processes owned by other users. The user ID is also used to firewall off on-disk data. [[securelevel]] === What is securelevel? `securelevel` is a security mechanism implemented in the kernel. When the securelevel is positive, the kernel restricts certain tasks; not even the superuser (`root`) is allowed to do them. The securelevel mechanism limits the ability to: * Unset certain file flags, such as `schg` (the system immutable flag). * Write to kernel memory via [.filename]#/dev/mem# and [.filename]#/dev/kmem#. * Load kernel modules. * Alter firewall rules. To check the status of the securelevel on a running system: [source,shell] .... # sysctl -n kern.securelevel .... The output contains the current value of the securelevel. If it is greater than 0, at least some of the securelevel's protections are enabled. The securelevel of a running system cannot be lowered as this would defeat its purpose. If a task requires that the securelevel be non-positive, change the `kern_securelevel` and `kern_securelevel_enable` variables in [.filename]#/etc/rc.conf# and reboot. For more information on securelevel and the specific things all the levels do, consult man:init[8]. [WARNING] ==== Securelevel is not a silver bullet; it has many known deficiencies. More often than not, it provides a false sense of security. One of its biggest problems is that in order for it to be at all effective, all files used in the boot process up until the securelevel is set must be protected. If an attacker can get the system to execute their code prior to the securelevel being set (which happens quite late in the boot process since some things the system must do at start-up cannot be done at an elevated securelevel), its protections are invalidated. While this task of protecting all files used in the boot process is not technically impossible, if it is achieved, system maintenance will become a nightmare since one would have to take the system down, at least to single-user mode, to modify a configuration file. This point and others are often discussed on the mailing lists, particularly the {freebsd-security}. Search the archives link:https://www.FreeBSD.org/search/[here] for an extensive discussion. A more fine-grained mechanism is preferred. ==== [[toor-account]] === What is this UID 0 toor account? Have I been compromised? Do not worry. `toor` is an "alternative" superuser account, where toor is root spelled backwards. It is intended to be used with a non-standard shell so the default shell for `root` does not need to change. This is important as shells which are not part of the base distribution, but are instead installed from ports or packages, are installed in [.filename]#/usr/local/bin# which, by default, resides on a different file system. If ``root``'s shell is located in [.filename]#/usr/local/bin# and the file system containing [.filename]#/usr/local/bin#) is not mounted, `root` will not be able to log in to fix a problem and will have to reboot into single-user mode in order to enter the path to a shell. Some people use `toor` for day-to-day `root` tasks with a non-standard shell, leaving `root`, with a standard shell, for single-user mode or emergencies. By default, a user cannot log in using `toor` as it does not have a password, so log in as `root` and set a password for `toor` before using it to login. [[serial]] == Serial Communications This section answers common questions about serial communications with FreeBSD. [[serial-console-prompt]] === How do I get the boot: prompt to show on the serial console? See extref:{handbook}serialcomms/[this section of the Handbook, serialconsole-setup]. [[found-serial]] === How do I tell if FreeBSD found my serial ports or modem cards? As the FreeBSD kernel boots, it will probe for the serial ports for which the kernel is configured. Either watch the boot messages closely or run this command after the system is up and running: [source,shell] .... % grep -E '^(sio|uart)[0-9]' < /var/run/dmesg.boot uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (115200,n,8,1) uart1: <16550 or compatible> port 0x2f8-2x3ff irq 3 on acpi0 .... This example shows two serial ports. The first is on IRQ4, port address `0x3f8`, and has a 16550-type UART chip. The second uses the same kind of chip but is on IRQ3 and is at port address `0x2f8`. Internal modem cards are treated just like serial ports, except that they always have a modem attached to the port. The [.filename]#GENERIC# kernel includes support for two serial ports using the same IRQ and port address settings in the above example. If these settings are not right for the system, or if there are more modem cards or serial ports than the kernel is configured for, reconfigure using the instructions in <> for more details. [[access-serial-ports]] === How do I access the serial ports on FreeBSD? (x86-specific) The third serial port, [.filename]#sio2#, or [.filename]#COM3#, is on [.filename]#/dev/cuad2# for dial-out devices, and on [.filename]#/dev/ttyd2# for dial-in devices. What is the difference between these two classes of devices? When opening [.filename]#/dev/ttydX# in blocking mode, a process will wait for the corresponding [.filename]#cuadX# device to become inactive, and then wait for the carrier detect line to go active. When the [.filename]#cuadX# device is opened, it makes sure the serial port is not already in use by the [.filename]#ttydX# device. If the port is available, it steals it from the [.filename]#ttydX# device. Also, the [.filename]#cuadX# device does not care about carrier detect. With this scheme and an auto-answer modem, remote users can log in and local users can still dial out with the same modem and the system will take care of all the conflicts. [[enable-multiport-serial]] === How do I enable support for a multi-port serial card? The section on kernel configuration provides information about configuring the kernel. For a multi-port serial card, place an man:sio[4] line for each serial port on the card in the man:device.hints[5] file. But place the IRQ specifiers on only one of the entries. All of the ports on the card should share one IRQ. For consistency, use the last serial port to specify the IRQ. Also, specify the following option in the kernel configuration file: [.programlisting] .... options COM_MULTIPORT .... The following [.filename]#/boot/device.hints# example is for an AST 4-port serial card on IRQ 12: [.programlisting] .... hint.sio.4.at="isa" hint.sio.4.port="0x2a0" hint.sio.4.flags="0x701" hint.sio.5.at="isa" hint.sio.5.port="0x2a8" hint.sio.5.flags="0x701" hint.sio.6.at="isa" hint.sio.6.port="0x2b0" hint.sio.6.flags="0x701" hint.sio.7.at="isa" hint.sio.7.port="0x2b8" hint.sio.7.flags="0x701" hint.sio.7.irq="12" .... The flags indicate that the master port has minor number `7` (`0x700`), and all the ports share an IRQ (`0x001`). [[default-serial-params]] === Can I set the default serial parameters for a port? See the extref:{handbook}serialcomms/[Serial Communications, serial-hw-config] section in the FreeBSD Handbook. [[cannot-tip]] === Why can I not run tip or cu? The built-in man:tip[1] and man:cu[1] utilities can only access the [.filename]#/var/spool/lock# directory via user `uucp` and group `dialer`. Use the `dialer` group to control who has access to the modem or remote systems by adding user accounts to `dialer`. Alternatively, everyone can be configured to run man:tip[1] and man:cu[1] by typing: [source,shell] .... # chmod 4511 /usr/bin/cu # chmod 4511 /usr/bin/tip .... [[misc]] == Miscellaneous Questions [[more-swap]] === FreeBSD uses a lot of swap space even when the computer has free memory left. Why? FreeBSD will proactively move entirely idle, unused pages of main memory into swap in order to make more main memory available for active use. This heavy use of swap is balanced by using the extra free memory for caching. Note that while FreeBSD is proactive in this regard, it does not arbitrarily decide to swap pages when the system is truly idle. Thus, the system will not be all paged out after leaving it idle overnight. [[top-freemem]] === Why does top show very little free memory even when I have very few programs running? The simple answer is that free memory is wasted memory. Any memory that programs do not actively allocate is used within the FreeBSD kernel as disk cache. The values shown by man:top[1] labeled as `Inact` and `Laundry` are cached data at different aging levels. This cached data means the system does not have to access a slow disk again for data it has accessed recently, thus increasing overall performance. In general, a low value shown for `Free` memory in man:top[1] is good, provided it is not _very_ low. [[chmod-symlinks]] === Why will `chmod` not change the permissions on symlinks? Symlinks do not have permissions, and by default, man:chmod[1] will follow symlinks to change the permissions on the source file, if possible. For the file, [.filename]#foo# with a symlink named [.filename]#bar#, this command will always succeed. [source,shell] .... % chmod g-w bar .... However, the permissions on [.filename]#bar# will not have changed. When changing modes of the file hierarchies rooted in the files instead of the files themselves, use either `-H` or `-L` together with `-R` to make this work. See man:chmod[1] and man:symlink[7] for more information. [WARNING] ==== `-R` does a _recursive_ man:chmod[1]. Be careful about specifying directories or symlinks to directories to man:chmod[1]. To change the permissions of a directory referenced by a symlink, use man:chmod[1] without any options and follow the symlink with a trailing slash ([.filename]#/#). For example, if [.filename]#foo# is a symlink to directory [.filename]#bar#, to change the permissions of [.filename]#foo# (actually [.filename]#bar#), do something like: [source,shell] .... % chmod 555 foo/ .... With the trailing slash, man:chmod[1] will follow the symlink, [.filename]#foo#, to change the permissions of the directory, [.filename]#bar#. ==== [[dos-binaries]] === Can I run DOS binaries under FreeBSD? Yes. A DOS emulation program, package:emulators/doscmd[], is available in the FreeBSD Ports Collection. If doscmd will not suffice, package:emulators/pcemu[] emulates an 8088 and enough BIOS services to run many DOS text-mode applications. It requires the X Window System. The Ports Collection also has package:emulators/dosbox[]. The main focus of this application is emulating old DOS games using the local file system for files. [[translation]] === What do I need to do to translate a FreeBSD document into my native language? See the extref:{fdp-primer}[Translation FAQ, translations] in the FreeBSD Documentation Project Primer. [[freebsd-mail-bounces]] === Why does my email to any address at FreeBSD.org bounce? The `FreeBSD.org` mail system implements some Postfix checks on incoming mail and rejects mail that is either from misconfigured relays or otherwise appears likely to be spam. Some of the specific requirements are: * The IP address of the SMTP client must "reverse-resolve" to a forward confirmed hostname. * The fully-qualified hostname given in the SMTP conversation (either HELO or EHLO) must resolve to the IP address of the client. Other advice to help mail reach its destination include: * Mail should be sent in plain text, and messages sent to mailing lists should generally be no more than 200KB in length. * Avoid excessive cross posting. Choose _one_ mailing list which seems most relevant and send it there. If you still have trouble with email infrastructure at `FreeBSD.org`, send a note with the details to mailto:postmaster@freebsd.org[postmaster@freebsd.org]; Include a date/time interval so that logs may be reviewed - and note that we only keep one week's worth of mail logs. (Be sure to specify the time zone or offset from UTC.) [[free-account]] === Where can I find a free FreeBSD account? While FreeBSD does not provide open access to any of their servers, others do provide open access UNIX(R) systems. The charge varies and limited services may be available. http://www.arbornet.org/[Arbornet, Inc], also known as _M-Net_, has been providing open access to UNIX(R) systems since 1983. Starting on an Altos running System III, the site switched to BSD/OS in 1991. In June of 2000, the site switched again to FreeBSD. _M-Net_ can be accessed via telnet and SSH and provides basic access to the entire FreeBSD software suite. However, network access is limited to members and patrons who donate to the system, which is run as a non-profit organization. _M-Net_ also provides an bulletin board system and interactive chat. [[daemon-name]] === What is the cute little red guy's name? He does not have one, and is just called "the BSD daemon". If you insist upon using a name, call him "beastie". Note that "beastie" is pronounced "BSD". More about the BSD daemon is available on his http://www.mckusick.com/beastie/index.html[home page]. [[use-beastie]] === Can I use the BSD daemon image? Perhaps. The BSD daemon is copyrighted by Marshall Kirk McKusick. Check his http://www.mckusick.com/beastie/mainpage/copyright.html[Statement on the Use of the BSD Daemon Figure] for detailed usage terms. In summary, the image can be used in a tasteful manner, for personal use, so long as appropriate credit is given. Before using the logo commercially, contact {mckusick} for permission. More details are available on the http://www.mckusick.com/beastie/index.html[BSD Daemon's home page]. [[daemon-images]] === Do you have any BSD daemon images I could use? Xfig and eps drawings are available under [.filename]#/usr/share/examples/BSD_daemon/#. [[glossary]] === I have seen an acronym or other term on the mailing lists and I do not understand what it means. Where should I look? Refer to the extref:{handbook}glossary/[FreeBSD Glossary]. [[bikeshed-painting]] === Why should I care what color the bikeshed is? The really, really short answer is that you should not. The somewhat longer answer is that just because you are capable of building a bikeshed does not mean you should stop others from building one just because you do not like the color they plan to paint it. This is a metaphor indicating that you need not argue about every little feature just because you know enough to do so. Some people have commented that the amount of noise generated by a change is inversely proportional to the complexity of the change. The longer and more complete answer is that after a very long argument about whether man:sleep[1] should take fractional second arguments, {phk} posted a long message entitled link:http://www.bikeshed.com[A bike shed (any color will do) on greener grass...]. The appropriate portions of that message are quoted below. **** “What is it about this bike shed?” some of you have asked me. It is a long story, or rather it is an old story, but it is quite short actually. C. Northcote Parkinson wrote a book in the early 1960s, called “Parkinson's Law”, which contains a lot of insight into the dynamics of management. [snip a bit of commentary on the book] In the specific example involving the bike shed, the other vital component is an atomic power-plant, I guess that illustrates the age of the book. Parkinson shows how you can go into the board of directors and get approval for building a multi-million or even billion dollar atomic power plant, but if you want to build a bike shed you will be tangled up in endless discussions. Parkinson explains that this is because an atomic plant is so vast, so expensive and so complicated that people cannot grasp it, and rather than try, they fall back on the assumption that somebody else checked all the details before it got this far. Richard P. Feynmann gives a couple of interesting, and very much to the point, examples relating to Los Alamos in his books. A bike shed on the other hand. Anyone can build one of those over a weekend, and still have time to watch the game on TV. So no matter how well prepared, no matter how reasonable you are with your proposal, somebody will seize the chance to show that he is doing his job, that he is paying attention, that he is here. In Denmark we call it “setting your fingerprint”. It is about personal pride and prestige, it is about being able to point somewhere and say “There! I did that.”. It is a strong trait in politicians, but present in most people given the chance. Just think about footsteps in wet cement. --Poul-Henning Kamp on freebsd-hackers, October 2, 1999 **** [[funnies]] == The FreeBSD Funnies [[very-very-cool]] === How cool is FreeBSD? _Q._ Has anyone done any temperature testing while running FreeBSD? I know Linux(R) runs cooler than DOS, but have never seen a mention of FreeBSD. It seems to run really hot. _A._ No, but we have done numerous taste tests on blindfolded volunteers who have also had 250 micrograms of LSD-25 administered beforehand. 35% of the volunteers said that FreeBSD tasted sort of orange, whereas Linux(R) tasted like purple haze. Neither group mentioned any significant variances in temperature. We eventually had to throw the results of this survey out entirely anyway when we found that too many volunteers were wandering out of the room during the tests, thus skewing the results. We think most of the volunteers are at Apple now, working on their new "scratch and sniff" GUI. It is a funny old business we are in! Seriously, FreeBSD uses the HLT (halt) instruction when the system is idle thus lowering its energy consumption and therefore the heat it generates. Also if you have ACPI (Advanced Configuration and Power Interface) configured, then FreeBSD can also put the CPU into a low power mode. [[letmeoutofhere]] === Who is scratching in my memory banks?? _Q._ Is there anything "odd" that FreeBSD does when compiling the kernel which would cause the memory to make a scratchy sound? When compiling (and for a brief moment after recognizing the floppy drive upon startup, as well), a strange scratchy sound emanates from what appears to be the memory banks. _A._ Yes! You will see frequent references to "daemons" in the BSD documentation, and what most people do not know is that this refers to genuine, non-corporeal entities that now possess your computer. The scratchy sound coming from your memory is actually high-pitched whispering exchanged among the daemons as they best decide how to deal with various system administration tasks. If the noise gets to you, a good `fdisk /mbr` from DOS will get rid of them, but do not be surprised if they react adversely and try to stop you. In fact, if at any point during the exercise you hear the satanic voice of Bill Gates coming from the built-in speaker, take off running and do not ever look back! Freed from the counterbalancing influence of the BSD daemons, the twin demons of DOS and Windows(R) are often able to re-assert total control over your machine to the eternal damnation of your soul. Now that you know, given a choice you would probably prefer to get used to the scratchy noises, no? [[changing-lightbulbs]] === How many FreeBSD hackers does it take to change a lightbulb? One thousand, one hundred and sixty-nine: Twenty-three to complain to -CURRENT about the lights being out; Four to claim that it is a configuration problem, and that such matters really belong on -questions; Three to submit PRs about it, one of which is misfiled under doc and consists only of "it's dark"; One to commit an untested lightbulb which breaks buildworld, then back it out five minutes later; Eight to flame the PR originators for not including patches in their PRs; Five to complain about buildworld being broken; Thirty-one to answer that it works for them, and they must have updated at a bad time; One to post a patch for a new lightbulb to -hackers; One to complain that he had patches for this three years ago, but when he sent them to -CURRENT they were just ignored, and he has had bad experiences with the PR system; besides, the proposed new lightbulb is non-reflexive; Thirty-seven to scream that lightbulbs do not belong in the base system, that committers have no right to do things like this without consulting the Community, and WHAT IS -CORE DOING ABOUT IT!? Two hundred to complain about the color of the bicycle shed; Three to point out that the patch breaks man:style[9]; Seventeen to complain that the proposed new lightbulb is under GPL; Five hundred and eighty-six to engage in a flame war about the comparative advantages of the GPL, the BSD license, the MIT license, the NPL, and the personal hygiene of unnamed FSF founders; Seven to move various portions of the thread to -chat and -advocacy; One to commit the suggested lightbulb, even though it shines dimmer than the old one; Two to back it out with a furious flame of a commit message, arguing that FreeBSD is better off in the dark than with a dim lightbulb; Forty-six to argue vociferously about the backing out of the dim lightbulb and demanding a statement from -core; Eleven to request a smaller lightbulb so it will fit their Tamagotchi if we ever decide to port FreeBSD to that platform; Seventy-three to complain about the SNR on -hackers and -chat and unsubscribe in protest; Thirteen to post "unsubscribe", "How do I unsubscribe?", or "Please remove me from the list", followed by the usual footer; One to commit a working lightbulb while everybody is too busy flaming everybody else to notice; Thirty-one to point out that the new lightbulb would shine 0.364% brighter if compiled with TenDRA (although it will have to be reshaped into a cube), and that FreeBSD should therefore switch to TenDRA instead of GCC; One to complain that the new lightbulb lacks fairings; Nine (including the PR originators) to ask "what is MFC?"; Fifty-seven to complain about the lights being out two weeks after the bulb has been changed. _{nik} adds:_ _I was laughing quite hard at this._ _And then I thought, "Hang on, shouldn't there be '1 to document it.' in that list somewhere?"_ _And then I was enlightened :-)_ _{tabthorpe}_ says: "None, _real_ FreeBSD hackers are not afraid of the dark!" [[dev-null]] === Where does data written to [.filename]#/dev/null# go? It goes into a special data sink in the CPU where it is converted to heat which is vented through the heatsink / fan assembly. This is why CPU cooling is increasingly important; as people get used to faster processors, they become careless with their data and more and more of it ends up in [.filename]#/dev/null#, overheating their CPUs. If you delete [.filename]#/dev/null# (which effectively disables the CPU data sink) your CPU may run cooler but your system will quickly become constipated with all that excess data and start to behave erratically. If you have a fast network connection you can cool down your CPU by reading data out of [.filename]#/dev/random# and sending it off somewhere; however you run the risk of overheating your network connection and [.filename]#/# or angering your ISP, as most of the data will end up getting converted to heat by their equipment, but they generally have good cooling, so if you do not overdo it you should be OK. _Paul Robinson adds:_ There are other methods. As every good sysadmin knows, it is part of standard practice to send data to the screen of interesting variety to keep all the pixies that make up your picture happy. Screen pixies (commonly mis-typed or re-named as "pixels") are categorized by the type of hat they wear (red, green or blue) and will hide or appear (thereby showing the color of their hat) whenever they receive a little piece of food. Video cards turn data into pixie-food, and then send them to the pixies - the more expensive the card, the better the food, so the better behaved the pixies are. They also need constant stimulation - this is why screen savers exist. To take your suggestions further, you could just throw the random data to console, thereby letting the pixies consume it. This causes no heat to be produced at all, keeps the pixies happy and gets rid of your data quite quickly, even if it does make things look a bit messy on your screen. Incidentally, as an ex-admin of a large ISP who experienced many problems attempting to maintain a stable temperature in a server room, I would strongly discourage people sending the data they do not want out to the network. The fairies who do the packet switching and routing get annoyed by it as well. [[punk-my-friend]] === My colleague sits at the computer too much, how can I prank her? Install package:games/sl[] and wait for her to mistype `sl` for `ls`. [[advanced]] == Advanced Topics [[learn-advanced]] === How can I learn more about FreeBSD's internals? See the extref:{arch-handbook}[FreeBSD Architecture Handbook]. Additionally, much general UNIX(R) knowledge is directly applicable to FreeBSD. [[how-to-contribute]] === How can I contribute to FreeBSD? What can I do to help? We accept all types of contributions: documentation, code, and even art. See the article on extref:{contributing}[Contributing to FreeBSD] for specific advice on how to do this. And thanks for the thought! [[define-snap-release]] === What are snapshots and releases? There are currently {rel-numbranch} active/semi-active branches in the FreeBSD http://svnweb.FreeBSD.org/base/[Subversion Repository]. (Earlier branches are only changed very rarely, which is why there are only {rel-numbranch} active branches of development): * {rel2-releng} AKA {rel2-stable} * {rel-releng} AKA {rel-stable} * {rel-head-releng} AKA _-CURRENT_ AKA {rel-head} `HEAD` is not an actual branch tag. It is a symbolic constant for the current, non-branched development stream known as _-CURRENT_. Right now, _-CURRENT_ is the {rel-head-relx} development stream; the {rel-stable} branch, {rel-releng}, forked off from _-CURRENT_ in {rel-relengdate} and the {rel2-stable} branch, {rel2-releng}, forked off from _-CURRENT_ in {rel2-relengdate}. [[kernel-panic-troubleshooting]] === How can I make the most of the data I see when my kernel panics? Here is typical kernel panic: [.programlisting] .... Fatal trap 12: page fault while in kernel mode fault virtual address = 0x40 fault code = supervisor read, page not present instruction pointer = 0x8:0xf014a7e5 stack pointer = 0x10:0xf4ed6f24 frame pointer = 0x10:0xf4ed6f28 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 80 (mount) interrupt mask = trap number = 12 panic: page fault .... This message is not enough. While the instruction pointer value is important, it is also configuration dependent as it varies depending on the kernel image. If it is a [.filename]#GENERIC# kernel image from one of the snapshots, it is possible for somebody else to track down the offending function, but for a custom kernel, only you can tell us where the fault occurred. To proceed: [.procedure] ==== . Write down the instruction pointer value. Note that the `0x8:` part at the beginning is not significant in this case: it is the `0xf0xxxxxx` part that we want. . When the system reboots, do the following: + [source,shell] .... % nm -n kernel.that.caused.the.panic | grep f0xxxxxx .... + where `f0xxxxxx` is the instruction pointer value. The odds are you will not get an exact match since the symbols in the kernel symbol table are for the entry points of functions and the instruction pointer address will be somewhere inside a function, not at the start. If you do not get an exact match, omit the last digit from the instruction pointer value and try again: + [source,shell] .... % nm -n kernel.that.caused.the.panic | grep f0xxxxx .... + If that does not yield any results, chop off another digit. Repeat until there is some sort of output. The result will be a possible list of functions which caused the panic. This is a less than exact mechanism for tracking down the point of failure, but it is better than nothing. ==== However, the best way to track down the cause of a panic is by capturing a crash dump, then using man:kgdb[1] to generate a stack trace on the crash dump. In any case, the method is this: [.procedure] ==== . Make sure that the following line is included in the kernel configuration file: + [.programlisting] .... makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols .... . Change to the [.filename]#/usr/src# directory: + [source,shell] .... # cd /usr/src .... . Compile the kernel: + [source,shell] .... # make buildkernel KERNCONF=MYKERNEL .... . Wait for man:make[1] to finish compiling. + [source,shell] .... # make installkernel KERNCONF=MYKERNEL .... . Reboot. ==== [NOTE] ==== If `KERNCONF` is not included, the [.filename]#GENERIC# kernel will instead be built and installed. ==== The man:make[1] process will have built two kernels. [.filename]#/usr/obj/usr/src/sys/MYKERNEL/kernel# and [.filename]#/usr/obj/usr/src/sys/MYKERNEL/kernel.debug#. [.filename]#kernel# was installed as [.filename]#/boot/kernel/kernel#, while [.filename]#kernel.debug# can be used as the source of debugging symbols for man:kgdb[1]. To capture a crash dump, edit [.filename]#/etc/rc.conf# and set `dumpdev` to point to either the swap partition or `AUTO`. This will cause the man:rc[8] scripts to use the man:dumpon[8] command to enable crash dumps. This command can also be run manually. After a panic, the crash dump can be recovered using man:savecore[8]; if `dumpdev` is set in [.filename]#/etc/rc.conf#, the man:rc[8] scripts will run man:savecore[8] automatically and put the crash dump in [.filename]#/var/crash#. [NOTE] ==== FreeBSD crash dumps are usually the same size as physical RAM. Therefore, make sure there is enough space in [.filename]#/var/crash# to hold the dump. Alternatively, run man:savecore[8] manually and have it recover the crash dump to another directory with more room. It is possible to limit the size of the crash dump by using `options MAXMEM=N` where _N_ is the size of kernel's memory usage in KBs. For example, for 1 GB of RAM, limit the kernel's memory usage to 128 MB, so that the crash dump size will be 128 MB instead of 1 GB. ==== Once the crash dump has been recovered , get a stack trace as follows: [source,shell] .... % kgdb /usr/obj/usr/src/sys/MYKERNEL/kernel.debug /var/crash/vmcore.0 (kgdb) backtrace .... Note that there may be several screens worth of information. Ideally, use man:script[1] to capture all of them. Using the unstripped kernel image with all the debug symbols should show the exact line of kernel source code where the panic occurred. The stack trace is usually read from the bottom up to trace the exact sequence of events that lead to the crash. man:kgdb[1] can also be used to print out the contents of various variables or structures to examine the system state at the time of the crash. [TIP] ==== If a second computer is available, man:kgdb[1] can be configured to do remote debugging, including setting breakpoints and single-stepping through the kernel code. ==== [NOTE] ==== If `DDB` is enabled and the kernel drops into the debugger, a panic and a crash dump can be forced by typing `panic` at the `ddb` prompt. It may stop in the debugger again during the panic phase. If it does, type `continue` and it will finish the crash dump. ==== [[dlsym-failure]] === Why has dlsym() stopped working for ELF executables? The ELF toolchain does not, by default, make the symbols defined in an executable visible to the dynamic linker. Consequently `dlsym()` searches on handles obtained from calls to `dlopen(NULL, flags)` will fail to find such symbols. To search, using `dlsym()`, for symbols present in the main executable of a process, link the executable using the `--export-dynamic` option to the ELF linker (man:ld[1]). [[change-kernel-address-space]] === How can I increase or reduce the kernel address space on i386? By default, the kernel address space is 1 GB (2 GB for PAE) for i386. When running a network-intensive server or using ZFS, this will probably not be enough. Add the following line to the kernel configuration file to increase available space and rebuild the kernel: [.programlisting] .... options KVA_PAGES=N .... To find the correct value of _N_, divide the desired address space size (in megabytes) by four. (For example, it is `512` for 2 GB.) [[acknowledgments]] == Acknowledgments This innocent little Frequently Asked Questions document has been written, rewritten, edited, folded, spindled, mutilated, eviscerated, contemplated, discombobulated, cogitated, regurgitated, rebuilt, castigated, and reinvigorated over the last decade, by a cast of hundreds if not thousands. Repeatedly. We wish to thank every one of the people responsible, and we encourage you to extref:{contributing}[join them] in making this FAQ even better. diff --git a/documentation/themes/beastie/layouts/books/single.html b/documentation/themes/beastie/layouts/books/single.html index ea5653a6ed..70793376f1 100644 --- a/documentation/themes/beastie/layouts/books/single.html +++ b/documentation/themes/beastie/layouts/books/single.html @@ -1,141 +1,137 @@ {{ define "main" }}
{{ if eq .Params.showBookMenu true }} {{ end }}
{{ if eq .Params.showBookMenu true }}
{{ end }}

{{ .Title }}

{{ if .Params.copyright }}
Copyright © {{ .Params.copyright }}
{{ end}} {{ if isset .Params "trademarks" }} {{ partial "trademarks" .Params.trademarks }} {{ end }}

{{ i18n "toc" }}

{{ .TableOfContents }}
{{ if $.Site.Params.isOnline }} {{ if .Params.add_split_page_link }}
[ {{ i18n "split-html" }} / {{ i18n "single-html" }} ]
{{ end }} {{ if .Params.add_single_page_link }}
[ {{ i18n "split-html" }} / {{ i18n "single-html" }} ]
{{ end }} {{ end }}
{{ .Content }}
{{ if .GitInfo }}

{{ i18n "last-modified" }}: {{ .GitInfo.AuthorDate | time.Format ":date_long" }} {{ i18n "by" }} {{ .GitInfo.AuthorName }}

{{ end }}
{{ if .Params.prev }} {{ end }}
{{ if $.Site.Params.isOnline }} {{ i18n "home" }} {{ else }} {{ i18n "home" }} {{ end }}
{{ if .Params.next }} {{ end }}
{{ end }}