diff --git a/documentation/content/en/articles/hubs/_index.adoc b/documentation/content/en/articles/hubs/_index.adoc index df91360857..21ad08e9d1 100644 --- a/documentation/content/en/articles/hubs/_index.adoc +++ b/documentation/content/en/articles/hubs/_index.adoc @@ -1,419 +1,419 @@ --- title: Mirroring FreeBSD authors: - author: Jun Kuriyama email: kuriyama@FreeBSD.org - author: Valentino Vaschetto email: logo@FreeBSD.org - author: Daniel Lang email: dl@leo.org - author: Ken Smith email: kensmith@FreeBSD.org description: The all in one guide for mirroring the FreeBSD website, FTP servers, and more trademarks: ["freebsd", "general"] tags: ["Mirroring", "FreeBSD", "Hub"] --- = Mirroring FreeBSD :doctype: article :toc: macro :toclevels: 1 :icons: font :sectnums: :source-highlighter: rouge :experimental: :images-path: articles/hubs/ ifdef::env-beastie[] ifdef::backend-html5[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] :imagesdir: ../../../images/{images-path} endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [.abstract-title] Abstract An in-progress article on how to mirror FreeBSD, aimed at hub administrators. ''' toc::[] [NOTE] ==== We are not accepting new community mirrors at this time. ==== [[mirror-contact]] == Contact Information The Mirror System Coordinators can be reached through email at mailto:mirror-admin@FreeBSD.org[mirror-admin@FreeBSD.org]. There is also a {freebsd-hubs}. [[mirror-requirements]] == Requirements for FreeBSD Mirrors [[mirror-diskspace]] === Disk Space Disk space is one of the most important requirements. Depending on the set of releases, architectures, and degree of completeness you want to mirror, a huge amount of disk space may be consumed. Also keep in mind that _official_ mirrors are probably required to be complete. The web pages should always be mirrored completely. Also note that the numbers stated here are reflecting the current state (at {rel120-current}-RELEASE/{rel113-current}-RELEASE). Further development and releases will only increase the required amount. Also make sure to keep some (ca. 10-20%) extra space around just to be sure. Here are some approximate figures: * Full FTP Distribution: 1.4 TB * CTM deltas: 10 GB * Web pages: 1GB The current disk usage of FTP Distribution can be found at link:ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes[ftp://ftp.FreeBSD.org/pub/FreeBSD/dir.sizes]. [[mirror-bandwidth]] === Network Connection/Bandwidth Of course, you need to be connected to the Internet. The required bandwidth depends on your intended use of the mirror. If you just want to mirror some parts of FreeBSD for local use at your site/intranet, the demand may be much smaller than if you want to make the files publicly available. If you intend to become an official mirror, the bandwidth required will be even higher. We can only give rough estimates here: * Local site, no public access: basically no minimum, but < 2 Mbps could make syncing too slow. * Unofficial public site: 34 Mbps is probably a good start. * Official site: > 100 Mbps is recommended, and your host should be connected as close as possible to your border router. [[mirror-system]] === System Requirements, CPU, RAM One thing this depends on the expected number of clients, which is determined by the server's policy. It is also affected by the types of services you want to offer. Plain FTP or HTTP services may not require a huge amount of resources. Watch out if you provide rsync. This can have a huge impact on CPU and memory requirements as it is considered a memory hog. The following are just examples to give you a very rough hint. For a moderately visited site that offers rsync, you might consider a current CPU with around 800MHz - 1 GHz, and at least 512MB RAM. This is probably the minimum you want for an _official_ site. For a frequently used site you definitely need more RAM (consider 2GB as a good start) and possibly more CPU, which could also mean that you need to go for a SMP system. You also want to consider a fast disk subsystem. Operations on the SVN repository require a fast disk subsystem (RAID is highly advised). A SCSI controller that has a cache of its own can also speed up things since most of these services incur a large number of small modifications to the disk. [[mirror-services]] === Services to Offer Every mirror site is required to have a set of core services available. In addition to these required services, there are a number of optional services that server administrators may choose to offer. This section explains which services you can provide and how to go about implementing them. [[mirror-serv-ftp]] ==== FTP (required for FTP Fileset) This is one of the most basic services, and it is required for each mirror offering public FTP distributions. FTP access must be anonymous, and no upload/download ratios are allowed (a ridiculous thing anyway). Upload capability is not required (and _must_ never be allowed for the FreeBSD file space). Also the FreeBSD archive should be available under the path [.filename]#/pub/FreeBSD#. There is a lot of software available which can be set up to allow anonymous FTP (in alphabetical order). * `/usr/libexec/ftpd`: FreeBSD's own ftpd can be used. Be sure to read man:ftpd[8]. * package:ftp/ncftpd[]: A commercial package, free for educational use. * package:ftp/oftpd[]: An ftpd designed with security as a main focus. * package:ftp/proftpd[]: A modular and very flexible ftpd. * package:ftp/pure-ftpd[]: Another ftpd developed with security in mind. * package:ftp/twoftpd[]: As above. * package:ftp/vsftpd[]: The "very secure" ftpd. FreeBSD's `ftpd`, `proftpd` and maybe `ncftpd` are among the most commonly used FTPds. The others do not have a large userbase among mirror sites. One thing to consider is that you may need flexibility in limiting how many simultaneous connections are allowed, thus limiting how much network bandwidth and system resources are consumed. [[mirror-serv-rsync]] ==== Rsync (optional for FTP Fileset) Rsync is often offered for access to the contents of the FTP area of FreeBSD, so other mirror sites can use your system as their source. The protocol is different from FTP in many ways. It is much more bandwidth friendly, as only differences between files are transferred instead of whole files when they change. Rsync does require a significant amount of memory for each instance. The size depends on the size of the synced module in terms of the number of directories and files. Rsync can use `rsh` and `ssh` (now default) as a transport, or use its own protocol for stand-alone access (this is the preferred method for public rsync servers). Authentication, connection limits, and other restrictions may be applied. There is just one software package available: * package:net/rsync[] [[mirror-serv-http]] ==== HTTP (required for Web Pages, Optional for FTP Fileset) If you want to offer the FreeBSD web pages, you will need to install a web server. You may optionally offer the FTP fileset via HTTP. The choice of web server software is left up to the mirror administrator. Some of the most popular choices are: * package:www/apache24[]: Apache is still one of the most widely deployed web servers on the Internet. It is used extensively by the FreeBSD Project. * package:www/boa[]: Boa is a single-tasking HTTP server. Unlike traditional web servers, it does not fork for each incoming connection, nor does it fork many copies of itself to handle multiple connections. Although, it should provide considerably great performance for purely static content. * package:www/cherokee[]: Cherokee is a very fast, flexible and easy to configure web server. It supports the widespread technologies nowadays: FastCGI, SCGI, PHP, CGI, SSL/TLS encrypted connections, vhosts, users authentication, on the fly encoding and load balancing. It also generates Apache compatible log files. * package:www/lighttpd[]: lighttpd is a secure, fast, compliant and very flexible web server which has been optimized for high-performance environments. It has a very low memory footprint compared to other web servers and takes care of cpu-load. * package:www/nginx[]: nginx is a high performance edge web server with a low memory footprint and key features to build a modern and efficient web infrastructure. Features include a HTTP server, HTTP and mail reverse proxy, caching, load balancing, compression, request throttling, connection multiplexing and reuse, SSL offload and HTTP media streaming. * package:www/thttpd[]: If you are going to be serving a large amount of static content you may find that using an application such as thttpd is more efficient than others. It is also optimized for excellent performance on FreeBSD. [[mirror-howto]] == How to Mirror FreeBSD Ok, now you know the requirements and how to offer the services, but not how to get it. :-) This section explains how to actually mirror the various parts of FreeBSD, what tools to use, and where to mirror from. [[mirror-ftp-rsync]] === Mirroring the FTP Site The FTP area is the largest amount of data that needs to be mirrored. It includes the _distribution sets_ required for network installation, the _branches_ which are actually snapshots of checked-out source trees, the _ISO Images_ to write CD-ROMs with the installation distribution, a live file system, and a snapshot of the ports tree. All of course for various FreeBSD versions, and various architectures. The best way to mirror the FTP area is rsync. You can install the port package:net/rsync[] and then use rsync to sync with your upstream host. rsync is already mentioned in crossref:hubs[mirror-serv-rsync, Rsync (optional for FTP Fileset)]. Since rsync access is not required, your preferred upstream site may not allow it. You may need to hunt around a little bit to find a site that allows rsync access. [NOTE] ==== Since the number of rsync clients will have a significant impact on the server machine, most admins impose limitations on their server. For a mirror, you should ask the site maintainer you are syncing from about their policy, and maybe an exception for your host (since you are a mirror). ==== A command line to mirror FreeBSD might look like: [source,shell] .... % rsync -vaHz --delete rsync://ftp4.de.FreeBSD.org/FreeBSD/ /pub/FreeBSD/ .... Consult the documentation for rsync, which is also available at http://rsync.samba.org/[http://rsync.samba.org/], about the various options to be used with rsync. If you sync the whole module (unlike subdirectories), be aware that the module-directory (here "FreeBSD") will not be created, so you cannot omit the target directory. Also you might want to set up a script framework that calls such a command via man:cron[8]. [[mirror-www]] === Mirroring the WWW Pages [WARNING] ==== Since doc migration to Hugo/Asciidoctor on 2021-01-25, mirroring the website with rsync no longer works. ==== There are ongoing studies to implement a website mirror with the extref:{handbook}mirrors/[official infrastructure]. For the former website mirrors, a way to achieve mirroring the website today is building the website locally with the corresponding address it will be hosted. [source,shell] .... % cd website && env HUGO_baseURL="https://www.XX.freebsd.org/" make .... Check for more details about the build tools on extref:{fdp-primer}overview/[FreeBSD Documentation Project Primer for New Contributors, overview-quick-start] book. //// [source,shell] .... % rsync -vaHz --delete rsync://bit0.us-west.freebsd.org/FreeBSD-www-data/ /usr/local/www/ .... //// [NOTE] ==== Notice the website was split into www.FreeBSD.org and docs.FreeBSD.org, and there are links between them; plus, at this moment, `HUGO_baseURL` variable won't cover all links, this way, mirroring the website is discouraged. ==== [[mirror-pkgs]] === Mirroring Packages Due to very high requirements of bandwidth, storage and administration the FreeBSD Project has decided not to allow public mirrors of packages. For sites with lots of machines, it might be advantagous to run a caching HTTP proxy for the man:pkg[8] process. Alternatively specific packages and their dependencies can be fetched by running something like the following: [source,shell] .... % pkg fetch -d -o /usr/local/mirror vim .... Once those packages have been fetched, the repository metadata must be generated by running: [source,shell] .... % pkg repo /usr/local/mirror .... Once the packages have been fetched and the metadata for the repository has been generated, serve the packages up to the client machines via HTTP. For additional information see the man pages for man:pkg[8], specifically the man:pkg-repo[8] page. [[mirror-how-often]] === How Often Should I Mirror? Every mirror should be updated at a minimum of once per day. Certainly a script with locking to prevent multiple runs happening at the same time will be needed to run from man:cron[8]. Since nearly every admin does this in their own way, specific instructions cannot be provided. It could work something like this: [.procedure] ==== . Put the command to run your mirroring application in a script. Use of a plain `/bin/sh` script is recommended. . Add some output redirections so diagnostic messages are logged to a file. . Test if your script works. Check the logs. . Use man:crontab[1] to add the script to the appropriate user's man:crontab[5]. This should be a different user than what your FTP daemon runs as so that if file permissions inside your FTP area are not world-readable those files cannot be accessed by anonymous FTP. This is used to "stage" releases - making sure all of the official mirror sites have all of the necessary release files on release day. ==== Here are some recommended schedules: * FTP fileset: daily * WWW pages: daily [[mirror-where]] == Where to Mirror From This is an important issue. So this section will spend some effort to explain the backgrounds. We will say this several times: under no circumstances should you mirror from `ftp.FreeBSD.org`. [[mirror-where-organization]] === A few Words About the Organization Mirrors are organized by country. All official mirrors have a DNS entry of the form `ftpN.CC.FreeBSD.org`. _CC_ (i.e., country code) is the _top level domain_ (TLD) of the country where this mirror is located. _N_ is a number, telling that the host would be the _Nth_ mirror in that country. (Same applies to `wwwN.CC.FreeBSD.org`, etc.) There are mirrors with no _CC_ part. These are the mirror sites that are very well connected and allow a large number of concurrent users. `ftp.FreeBSD.org` is actually two machines, one currently located in Denmark and the other in the United States. It is _NOT_ a master site and should never be used to mirror from. Lots of online documentation leads "interactive"users to `ftp.FreeBSD.org` so automated mirroring systems should find a different machine to mirror from. Additionally there exists a hierarchy of mirrors, which is described in terms of __tiers__. The master sites are not referred to but can be described as __Tier-0__. Mirrors that mirror from these sites can be considered __Tier-1__, mirrors of __Tier-1__-mirrors, are __Tier-2__, etc. Official sites are encouraged to be of a low __tier__, but the lower the tier the higher the requirements in terms as described in crossref:hubs[mirror-requirements, Requirements for FreeBSD Mirrors]. Also access to low-tier-mirrors may be restricted, and access to master sites is definitely restricted. The __tier__-hierarchy is not reflected by DNS and generally not documented anywhere except for the master sites. However, official mirrors with low numbers like 1-4, are usually _Tier-1_ (this is just a rough hint, and there is no rule). [[mirror-where-where]] === Ok, but Where Should I get the Stuff Now? Under no circumstances should you mirror from `ftp.FreeBSD.org`. The short answer is: from the site that is closest to you in Internet terms, or gives you the fastest access. [[mirror-where-simple]] ==== I Just Want to Mirror from Somewhere! If you have no special intentions or requirements, the statement in -crossref:hubs[mirror-where-where, Ok, but Where Should I get the Stuff Now?] applies. +crossref:hubs[mirror-where-where,"Ok, but Where Should I get the Stuff Now?"] applies. This means: [.procedure] ==== . Check for those which provide fastest access (number of hops, round-trip-times) and offer the services you intend to use (like rsync). . Contact the administrators of your chosen site stating your request, and asking about their terms and policies. . Set up your mirror as described above. ==== [[mirror-where-official]] ==== I am an Official Mirror, What is the Right Site for Me? In general the description in crossref:hubs[mirror-where-simple, I Just Want to Mirror from Somewhere!] still applies. Of course you may want to put some weight on the fact that your upstream should be of a low tier. There are some other considerations about _official_ mirrors that are described in crossref:hubs[mirror-official, Official Mirrors]. [[mirror-where-master]] ==== I Want to Access the Master Sites! If you have good reasons and good prerequisites, you may want and get access to one of the master sites. Access to these sites is generally restricted, and there are special policies for access. If you are already an _official_ mirror, this certainly helps you getting access. In any other case make sure your country really needs another mirror. If it already has three or more, ask the "zone administrator" (mailto:hostmaster@CC.FreeBSD.org[hostmaster@CC.FreeBSD.org]) or {freebsd-hubs} first. Whoever helped you become, an _official_ should have helped you gain access to an appropriate upstream host, either one of the master sites or a suitable Tier-1 site. If not, you can send email to mailto:mirror-admin@FreeBSD.org[mirror-admin@FreeBSD.org] to request help with that. There is one master site for the FTP fileset. [[mirror-where-master-ftp]] ===== ftp-master.FreeBSD.org This is the master site for the FTP fileset. `ftp-master.FreeBSD.org` provides rsync access, in addition to FTP. Refer to crossref:hubs[mirror-ftp-rsync, Mirroring the FTP Site]. Mirrors are also encouraged to allow rsync access for the FTP contents, since they are __Tier-1__-mirrors. [[mirror-official]] == Official Mirrors Official mirrors are mirrors that * a) have a `FreeBSD.org` DNS entry (usually a CNAME). * b) are listed as an official mirror in the FreeBSD documentation (like handbook). So far to distinguish official mirrors. Official mirrors are not necessarily __Tier-1__-mirrors. However you probably will not find a __Tier-1__-mirror, that is not also official. [[mirror-official-requirements]] === Special Requirements for Official (tier-1) Mirrors It is not so easy to state requirements for all official mirrors, since the project is sort of tolerant here. It is more easy to say, what _official tier-1 mirrors_ are required to. All other official mirrors can consider this a big __should__. Tier-1 mirrors are required to: * carry the complete fileset * allow access to other mirror sites * provide FTP and rsync access Furthermore, admins should be subscribed to the {freebsd-hubs}. See extref:{handbook}eresources[this link, eresources-mail] for details, how to subscribe. [IMPORTANT] ==== It is _very_ important for a hub administrator, especially Tier-1 hub admins, to check the https://www.FreeBSD.org/releng/[release schedule] for the next FreeBSD release. This is important because it will tell you when the next release is scheduled to come out, and thus giving you time to prepare for the big spike of traffic which follows it. It is also important that hub administrators try to keep their mirrors as up-to-date as possible (again, even more crucial for Tier-1 mirrors). If Mirror1 does not update for a while, lower tier mirrors will begin to mirror old data from Mirror1 and thus begins a downward spiral... Keep your mirrors up to date! ==== [[mirror-official-become]] === How to Become Official Then? Please contact the Cluster Administrators as documented at https://www.FreeBSD.org/administration/#t-clusteradm. [[mirror-statpages]] == Some Statistics from Mirror Sites Here are links to the stat pages of your favorite mirrors (aka the only ones who feel like providing stats). [[mirror-statpagesftp]] === FTP Site Statistics * ftp.is.FreeBSD.org - mailto:hostmaster@is.FreeBSD.org[hostmaster@is.FreeBSD.org] - http://www.rhnet.is/status/draupnir/draupnir.html[ (Bandwidth)] http://www.rhnet.is/status/ftp/ftp-notendur.html[(FTP processes)] http://www.rhnet.is/status/ftp/http-notendur.html[(HTTP processes)] * ftp2.ru.FreeBSD.org - mailto:mirror@macomnet.ru[mirror@macomnet.ru] - http://mirror.macomnet.net/mrtg/mirror.macomnet.net_195.128.64.25.html[(Bandwidth)] http://mirror.macomnet.net/mrtg/mirror.macomnet.net_proc.html[(HTTP and FTP users)] diff --git a/documentation/content/en/articles/vinum/_index.adoc b/documentation/content/en/articles/vinum/_index.adoc index 52656f88b7..bbb7bc069d 100644 --- a/documentation/content/en/articles/vinum/_index.adoc +++ b/documentation/content/en/articles/vinum/_index.adoc @@ -1,722 +1,722 @@ --- title: The vinum Volume Manager authors: - author: Greg Lehey description: The vinum Volume Manager in FreeBSD tags: ["vinum", "Volume Manager", "FreeBSD"] --- //// The Vinum Volume Manager By Greg Lehey (grog at lemis dot com) Added to the Handbook by Hiten Pandya and Tom Rhodes For the FreeBSD Documentation Project //// = The vinum Volume Manager :doctype: article :toc: macro :toclevels: 1 :icons: font :sectnums: :source-highlighter: rouge :experimental: :images-path: articles/vinum/ ifdef::env-beastie[] ifdef::backend-html5[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] :imagesdir: ../../../images/{images-path} endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] include::../../../../../shared/asciidoctor.adoc[] endif::[] ''' toc::[] [[vinum-synopsis]] == Synopsis No matter the type of disks, there are always potential problems. The disks can be too small, too slow, or too unreliable to meet the system's requirements. While disks are getting bigger, so are data storage requirements. Often a file system is needed that is bigger than a disk's capacity. Various solutions to these problems have been proposed and implemented. One method is through the use of multiple, and sometimes redundant, disks. In addition to supporting various cards and controllers for hardware Redundant Array of Independent Disks RAID systems, the base FreeBSD system includes the [.filename]#vinum# volume manager, a block device driver that implements virtual disk drives and addresses these three problems. [.filename]#vinum# provides more flexibility, performance, and reliability than traditional disk storage and implements `RAID`-0, `RAID`-1, and `RAID`-5 models, both individually and in combination. This chapter provides an overview of potential problems with traditional disk storage, and an introduction to the [.filename]#vinum# volume manager. [WARNING] ==== vinum is deprecated and is not present in FreeBSD 15.0 and later. Users are advised to migrate to man:gconcat[8], man:gmirror[8], man:gstripe[8], man:graid[8], or man:zfs[8]. ==== [NOTE] ==== Starting with FreeBSD 5, [.filename]#vinum# has been rewritten to fit into the extref:{handbook}geom[GEOM architecture, geom], while retaining the original ideas, terminology, and on-disk metadata. This rewrite is called _gvinum_ (for _GEOM vinum_). While this chapter uses the term [.filename]#vinum#, any command invocations should be performed with `gvinum`. The name of the kernel module has changed from the original [.filename]#vinum.ko# to [.filename]#geom_vinum.ko#, and all device nodes reside under [.filename]#/dev/gvinum# instead of [.filename]#/dev/vinum#. As of FreeBSD 6, the original [.filename]#vinum# implementation is no longer available in the code base. ==== [[vinum-access-bottlenecks]] == Access Bottlenecks Modern systems frequently need to access data in a highly concurrent manner. For example, large FTP or HTTP servers can maintain thousands of concurrent sessions and have multiple 100 Mbit/s connections to the outside world, well beyond the sustained transfer rate of most disks. Current disk drives can transfer data sequentially at up to 70 MB/s, but this value is of little importance in an environment where many independent processes access a drive, and where they may achieve only a fraction of these values. In such cases, it is more interesting to view the problem from the viewpoint of the disk subsystem. The important parameter is the load that a transfer places on the subsystem, or the time for which a transfer occupies the drives involved in the transfer. In any disk transfer, the drive must first position the heads, wait for the first sector to pass under the read head, and then perform the transfer. These actions can be considered to be atomic as it does not make any sense to interrupt them. [[vinum-latency]] Consider a typical transfer of about 10 kB: the current generation of high-performance disks can position the heads in an average of 3.5 ms. The fastest drives spin at 15,000 rpm, so the average rotational latency (half a revolution) is 2 ms. At 70 MB/s, the transfer itself takes about 150 μs, almost nothing compared to the positioning time. In such a case, the effective transfer rate drops to a little over 1 MB/s and is clearly highly dependent on the transfer size. The traditional and obvious solution to this bottleneck is "more spindles": rather than using one large disk, use several smaller disks with the same aggregate storage space. Each disk is capable of positioning and transferring independently, so the effective throughput increases by a factor close to the number of disks used. The actual throughput improvement is smaller than the number of disks involved. Although each drive is capable of transferring in parallel, there is no way to ensure that the requests are evenly distributed across the drives. Inevitably the load on one drive will be higher than on another. The evenness of the load on the disks is strongly dependent on the way the data is shared across the drives. In the following discussion, it is convenient to think of the disk storage as a large number of data sectors which are addressable by number, rather like the pages in a book. The most obvious method is to divide the virtual disk into groups of consecutive sectors the size of the individual physical disks and store them in this manner, rather like taking a large book and tearing it into smaller sections. This method is called _concatenation_ and has the advantage that the disks are not required to have any specific size relationships. It works well when the access to the virtual disk is spread evenly about its address space. When access is concentrated on a smaller area, the improvement is less marked. crossref:vinum[vinum-concat, Concatenated Organization] illustrates the sequence in which storage units are allocated in a concatenated organization. [[vinum-concat]] .Concatenated Organization image::vinum-concat.png[] An alternative mapping is to divide the address space into smaller, equal-sized components and store them sequentially on different devices. For example, the first 256 sectors may be stored on the first disk, the next 256 sectors on the next disk and so on. After filling the last disk, the process repeats until the disks are full. This mapping is called _striping_ or RAID-0. `RAID` offers various forms of fault tolerance, though RAID-0 is somewhat misleading as it provides no redundancy. Striping requires somewhat more effort to locate the data, and it can cause additional I/O load where a transfer is spread over multiple disks, but it can also provide a more constant load across the disks. crossref:vinum[vinum-striped, Striped Organization] illustrates the sequence in which storage units are allocated in a striped organization. [[vinum-striped]] .Striped Organization image::vinum-striped.png[] [[vinum-data-integrity]] == Data Integrity The final problem with disks is that they are unreliable. Although reliability has increased tremendously over the last few years, disk drives are still the most likely core component of a server to fail. When they do, the results can be catastrophic and replacing a failed disk drive and restoring data can result in server downtime. One approach to this problem is _mirroring_, or `RAID-1`, which keeps two copies of the data on different physical hardware. Any write to the volume writes to both disks; a read can be satisfied from either, so if one drive fails, the data is still available on the other drive. Mirroring has two problems: * It requires twice as much disk storage as a non-redundant solution. * Writes must be performed to both drives, so they take up twice the bandwidth of a non-mirrored volume. Reads do not suffer from a performance penalty and can even be faster. An alternative solution is _parity_, implemented in `RAID` levels 2, 3, 4 and 5. Of these, `RAID-5` is the most interesting. As implemented in [.filename]#vinum#, it is a variant on a striped organization which dedicates one block of each stripe to parity one of the other blocks. As implemented by [.filename]#vinum#, a `RAID-5` plex is similar to a striped plex, except that it implements `RAID-5` by including a parity block in each stripe. As required by `RAID-5`, the location of this parity block changes from one stripe to the next. The numbers in the data blocks indicate the relative block numbers. [[vinum-raid5-org]] .`RAID`-5 Organization image::vinum-raid5-org.png[] Compared to mirroring, `RAID-5` has the advantage of requiring significantly less storage space. Read access is similar to that of striped organizations, but write access is significantly slower, approximately 25% of the read performance. If one drive fails, the array can continue to operate in degraded mode where a read from one of the remaining accessible drives continues normally, but a read from the failed drive is recalculated from the corresponding block from all the remaining drives. [[vinum-objects]] == [.filename]#vinum# Objects To address these problems, [.filename]#vinum# implements a four-level hierarchy of objects: * The most visible object is the virtual disk, called a _volume_. Volumes have essentially the same properties as a UNIX(R) disk drive, though there are some minor differences. For one, they have no size limitations. * Volumes are composed of _plexes_, each of which represent the total address space of a volume. This level in the hierarchy provides redundancy. Think of plexes as individual disks in a mirrored array, each containing the same data. * Since [.filename]#vinum# exists within the UNIX(R) disk storage framework, it would be possible to use UNIX(R) partitions as the building block for multi-disk plexes. In fact, this turns out to be too inflexible as UNIX(R) disks can have only a limited number of partitions. Instead, [.filename]#vinum# subdivides a single UNIX(R) partition, the _drive_, into contiguous areas called _subdisks_, which are used as building blocks for plexes. * Subdisks reside on [.filename]#vinum#_drives_, currently UNIX(R) partitions. [.filename]#vinum# drives can contain any number of subdisks. With the exception of a small area at the beginning of the drive, which is used for storing configuration and state information, the entire drive is available for data storage. The following sections describe the way these objects provide the functionality required of [.filename]#vinum#. === Volume Size Considerations Plexes can include multiple subdisks spread over all drives in the [.filename]#vinum# configuration. As a result, the size of an individual drive does not limit the size of a plex or a volume. === Redundant Data Storage [.filename]#vinum# implements mirroring by attaching multiple plexes to a volume. Each plex is a representation of the data in a volume. A volume may contain between one and eight plexes. Although a plex represents the complete data of a volume, it is possible for parts of the representation to be physically missing, either by design (by not defining a subdisk for parts of the plex) or by accident (as a result of the failure of a drive). As long as at least one plex can provide the data for the complete address range of the volume, the volume is fully functional. === Which Plex Organization? [.filename]#vinum# implements both concatenation and striping at the plex level: * A _concatenated plex_ uses the address space of each subdisk in turn. Concatenated plexes are the most flexible as they can contain any number of subdisks, and the subdisks may be of different length. The plex may be extended by adding additional subdisks. They require less CPU time than striped plexes, though the difference in CPU overhead is not measurable. On the other hand, they are most susceptible to hot spots, where one disk is very active and others are idle. * A _striped plex_ stripes the data across each subdisk. The subdisks must all be the same size and there must be at least two subdisks to distinguish it from a concatenated plex. The greatest advantage of striped plexes is that they reduce hot spots. By choosing an optimum sized stripe, about 256 kB, the load can be evened out on the component drives. Extending a plex by adding new subdisks is so complicated that [.filename]#vinum# does not implement it. crossref:vinum[vinum-comparison, [.filename]#vinum# Plex Organizations] summarizes the advantages and disadvantages of each plex organization. [[vinum-comparison]] .[.filename]#vinum# Plex Organizations [cols="1,1,1,1,1", frame="none", options="header"] |=== | Plex type | Minimum subdisks | Can add subdisks | Must be equal size | Application |concatenated |1 |yes |no |Large data storage with maximum placement flexibility and moderate performance |striped |2 |no |yes |High performance in combination with highly concurrent access |=== [[vinum-examples]] == Some Examples [.filename]#vinum# maintains a _configuration database_ which describes the objects known to an individual system. Initially, the user creates the configuration database from one or more configuration files using man:gvinum[8]. [.filename]#vinum# stores a copy of its configuration database on each disk _device_ under its control. This database is updated on each state change, so that a restart accurately restores the state of each [.filename]#vinum# object. === The Configuration File The configuration file describes individual [.filename]#vinum# objects. The definition of a simple volume might be: [.programlisting] .... drive a device /dev/da3h volume myvol plex org concat sd length 512m drive a .... This file describes four [.filename]#vinum# objects: * The _drive_ line describes a disk partition (_drive_) and its location relative to the underlying hardware. It is given the symbolic name _a_. This separation of symbolic names from device names allows disks to be moved from one location to another without confusion. * The _volume_ line describes a volume. The only required attribute is the name, in this case _myvol_. * The _plex_ line defines a plex. The only required parameter is the organization, in this case _concat_. No name is necessary as the system automatically generates a name from the volume name by adding the suffix _.px_, where _x_ is the number of the plex in the volume. Thus this plex will be called _myvol.p0_. * The _sd_ line describes a subdisk. The minimum specifications are the name of a drive on which to store it, and the length of the subdisk. No name is necessary as the system automatically assigns names derived from the plex name by adding the suffix _.sx_, where _x_ is the number of the subdisk in the plex. Thus [.filename]#vinum# gives this subdisk the name _myvol.p0.s0_. After processing this file, man:gvinum[8] produces the following output: [.programlisting] .... # gvinum -> create config1 Configuration summary Drives: 1 (4 configured) Volumes: 1 (4 configured) Plexes: 1 (8 configured) Subdisks: 1 (16 configured) D a State: up Device /dev/da3h Avail: 2061/2573 MB (80%) V myvol State: up Plexes: 1 Size: 512 MB P myvol.p0 C State: up Subdisks: 1 Size: 512 MB S myvol.p0.s0 State: up PO: 0 B Size: 512 MB .... This output shows the brief listing format of man:gvinum[8]. It is represented graphically in crossref:vinum[vinum-simple-vol, A Simple [.filename]#vinum# Volume]. [[vinum-simple-vol]] .A Simple [.filename]#vinum# Volume image::vinum-simple-vol.png[] This figure, and the ones which follow, represent a volume, which contains the plexes, which in turn contains the subdisks. In this example, the volume contains one plex, and the plex contains one subdisk. This particular volume has no specific advantage over a conventional disk partition. It contains a single plex, so it is not redundant. The plex contains a single subdisk, so there is no difference in storage allocation from a conventional disk partition. The following sections illustrate various more interesting configuration methods. === Increased Resilience: Mirroring The resilience of a volume can be increased by mirroring. When laying out a mirrored volume, it is important to ensure that the subdisks of each plex are on different drives, so that a drive failure will not take down both plexes. The following configuration mirrors a volume: [.programlisting] .... drive b device /dev/da4h volume mirror plex org concat sd length 512m drive a plex org concat sd length 512m drive b .... In this example, it was not necessary to specify a definition of drive _a_ again, since [.filename]#vinum# keeps track of all objects in its configuration database. After processing this definition, the configuration looks like: [.programlisting] .... Drives: 2 (4 configured) Volumes: 2 (4 configured) Plexes: 3 (8 configured) Subdisks: 3 (16 configured) D a State: up Device /dev/da3h Avail: 1549/2573 MB (60%) D b State: up Device /dev/da4h Avail: 2061/2573 MB (80%) V myvol State: up Plexes: 1 Size: 512 MB V mirror State: up Plexes: 2 Size: 512 MB P myvol.p0 C State: up Subdisks: 1 Size: 512 MB P mirror.p0 C State: up Subdisks: 1 Size: 512 MB P mirror.p1 C State: initializing Subdisks: 1 Size: 512 MB S myvol.p0.s0 State: up PO: 0 B Size: 512 MB S mirror.p0.s0 State: up PO: 0 B Size: 512 MB S mirror.p1.s0 State: empty PO: 0 B Size: 512 MB .... crossref:vinum[vinum-mirrored-vol, A Mirrored [.filename]#vinum# Volume] shows the structure graphically. [[vinum-mirrored-vol]] .A Mirrored [.filename]#vinum# Volume image::vinum-mirrored-vol.png[] In this example, each plex contains the full 512 MB of address space. As in the previous example, each plex contains only a single subdisk. === Optimizing Performance The mirrored volume in the previous example is more resistant to failure than an unmirrored volume, but its performance is less as each write to the volume requires a write to both drives, using up a greater proportion of the total disk bandwidth. Performance considerations demand a different approach: instead of mirroring, the data is striped across as many disk drives as possible. The following configuration shows a volume with a plex striped across four disk drives: [.programlisting] .... drive c device /dev/da5h drive d device /dev/da6h volume stripe plex org striped 512k sd length 128m drive a sd length 128m drive b sd length 128m drive c sd length 128m drive d .... As before, it is not necessary to define the drives which are already known to [.filename]#vinum#. After processing this definition, the configuration looks like: [.programlisting] .... Drives: 4 (4 configured) Volumes: 3 (4 configured) Plexes: 4 (8 configured) Subdisks: 7 (16 configured) D a State: up Device /dev/da3h Avail: 1421/2573 MB (55%) D b State: up Device /dev/da4h Avail: 1933/2573 MB (75%) D c State: up Device /dev/da5h Avail: 2445/2573 MB (95%) D d State: up Device /dev/da6h Avail: 2445/2573 MB (95%) V myvol State: up Plexes: 1 Size: 512 MB V mirror State: up Plexes: 2 Size: 512 MB V striped State: up Plexes: 1 Size: 512 MB P myvol.p0 C State: up Subdisks: 1 Size: 512 MB P mirror.p0 C State: up Subdisks: 1 Size: 512 MB P mirror.p1 C State: initializing Subdisks: 1 Size: 512 MB P striped.p1 State: up Subdisks: 1 Size: 512 MB S myvol.p0.s0 State: up PO: 0 B Size: 512 MB S mirror.p0.s0 State: up PO: 0 B Size: 512 MB S mirror.p1.s0 State: empty PO: 0 B Size: 512 MB S striped.p0.s0 State: up PO: 0 B Size: 128 MB S striped.p0.s1 State: up PO: 512 kB Size: 128 MB S striped.p0.s2 State: up PO: 1024 kB Size: 128 MB S striped.p0.s3 State: up PO: 1536 kB Size: 128 MB .... [[vinum-striped-vol]] .A Striped [.filename]#vinum# Volume image::vinum-striped-vol.png[] This volume is represented in crossref:vinum[vinum-striped-vol, A Striped [.filename]#vinum# Volume]. The darkness of the stripes indicates the position within the plex address space, where the lightest stripes come first and the darkest last. === Resilience and Performance [[vinum-resilience]]With sufficient hardware, it is possible to build volumes which show both increased resilience and increased performance compared to standard UNIX(R) partitions. A typical configuration file might be: [.programlisting] .... volume raid10 plex org striped 512k sd length 102480k drive a sd length 102480k drive b sd length 102480k drive c sd length 102480k drive d sd length 102480k drive e plex org striped 512k sd length 102480k drive c sd length 102480k drive d sd length 102480k drive e sd length 102480k drive a sd length 102480k drive b .... The subdisks of the second plex are offset by two drives from those of the first plex. This helps to ensure that writes do not go to the same subdisks even if a transfer goes over two drives. -crossref:vinum[vinum-raid10-vol, A Mirrored, Striped [.filename]#vinum# Volume] represents the structure of this volume. +crossref:vinum[vinum-raid10-vol,"A Mirrored, Striped [.filename]#vinum# Volume"] represents the structure of this volume. [[vinum-raid10-vol]] .A Mirrored, Striped [.filename]#vinum# Volume image::vinum-raid10-vol.png[] [[vinum-object-naming]] == Object Naming [.filename]#vinum# assigns default names to plexes and subdisks, although they may be overridden. Overriding the default names is not recommended as it does not bring a significant advantage and it can cause confusion. Names may contain any non-blank character, but it is recommended to restrict them to letters, digits and the underscore characters. The names of volumes, plexes, and subdisks may be up to 64 characters long, and the names of drives may be up to 32 characters long. [.filename]#vinum# objects are assigned device nodes in the hierarchy [.filename]#/dev/gvinum#. The configuration shown above would cause [.filename]#vinum# to create the following device nodes: * Device entries for each volume. These are the main devices used by [.filename]#vinum#. The configuration above would include the devices [.filename]#/dev/gvinum/myvol#, [.filename]#/dev/gvinum/mirror#, [.filename]#/dev/gvinum/striped#, [.filename]#/dev/gvinum/raid5# and [.filename]#/dev/gvinum/raid10#. * All volumes get direct entries under [.filename]#/dev/gvinum/#. * The directories [.filename]#/dev/gvinum/plex#, and [.filename]#/dev/gvinum/sd#, which contain device nodes for each plex and for each subdisk, respectively. For example, consider the following configuration file: [.programlisting] .... drive drive1 device /dev/sd1h drive drive2 device /dev/sd2h drive drive3 device /dev/sd3h drive drive4 device /dev/sd4h volume s64 setupstate plex org striped 64k sd length 100m drive drive1 sd length 100m drive drive2 sd length 100m drive drive3 sd length 100m drive drive4 .... After processing this file, man:gvinum[8] creates the following structure in [.filename]#/dev/gvinum#: [.programlisting] .... drwxr-xr-x 2 root wheel 512 Apr 13 16:46 plex crwxr-xr-- 1 root wheel 91, 2 Apr 13 16:46 s64 drwxr-xr-x 2 root wheel 512 Apr 13 16:46 sd /dev/vinum/plex: total 0 crwxr-xr-- 1 root wheel 25, 0x10000002 Apr 13 16:46 s64.p0 /dev/vinum/sd: total 0 crwxr-xr-- 1 root wheel 91, 0x20000002 Apr 13 16:46 s64.p0.s0 crwxr-xr-- 1 root wheel 91, 0x20100002 Apr 13 16:46 s64.p0.s1 crwxr-xr-- 1 root wheel 91, 0x20200002 Apr 13 16:46 s64.p0.s2 crwxr-xr-- 1 root wheel 91, 0x20300002 Apr 13 16:46 s64.p0.s3 .... Although it is recommended that plexes and subdisks should not be allocated specific names, [.filename]#vinum# drives must be named. This makes it possible to move a drive to a different location and still recognize it automatically. Drive names may be up to 32 characters long. === Creating File Systems Volumes appear to the system to be identical to disks, with one exception. Unlike UNIX(R) drives, [.filename]#vinum# does not partition volumes, which thus do not contain a partition table. This has required modification to some disk utilities, notably man:newfs[8], so that it does not try to interpret the last letter of a [.filename]#vinum# volume name as a partition identifier. For example, a disk drive may have a name like [.filename]#/dev/ad0a# or [.filename]#/dev/da2h#. These names represent the first partition ([.filename]#a#) on the first (0) IDE disk ([.filename]#ad#) and the eighth partition ([.filename]#h#) on the third (2) SCSI disk ([.filename]#da#) respectively. By contrast, a [.filename]#vinum# volume might be called [.filename]#/dev/gvinum/concat#, which has no relationship with a partition name. To create a file system on this volume, use man:newfs[8]: [source,shell] .... # newfs /dev/gvinum/concat .... [[vinum-config]] == Configuring [.filename]#vinum# The [.filename]#GENERIC# kernel does not contain [.filename]#vinum#. It is possible to build a custom kernel which includes [.filename]#vinum#, but this is not recommended. The standard way to start [.filename]#vinum# is as a kernel module. man:kldload[8] is not needed because when man:gvinum[8] starts, it checks whether the module has been loaded, and if it is not, it loads it automatically. === Startup [.filename]#vinum# stores configuration information on the disk slices in essentially the same form as in the configuration files. When reading from the configuration database, [.filename]#vinum# recognizes a number of keywords which are not allowed in the configuration files. For example, a disk configuration might contain the following text: [.programlisting] .... volume myvol state up volume bigraid state down plex name myvol.p0 state up org concat vol myvol plex name myvol.p1 state up org concat vol myvol plex name myvol.p2 state init org striped 512b vol myvol plex name bigraid.p0 state initializing org raid5 512b vol bigraid sd name myvol.p0.s0 drive a plex myvol.p0 state up len 1048576b driveoffset 265b plexoffset 0b sd name myvol.p0.s1 drive b plex myvol.p0 state up len 1048576b driveoffset 265b plexoffset 1048576b sd name myvol.p1.s0 drive c plex myvol.p1 state up len 1048576b driveoffset 265b plexoffset 0b sd name myvol.p1.s1 drive d plex myvol.p1 state up len 1048576b driveoffset 265b plexoffset 1048576b sd name myvol.p2.s0 drive a plex myvol.p2 state init len 524288b driveoffset 1048841b plexoffset 0b sd name myvol.p2.s1 drive b plex myvol.p2 state init len 524288b driveoffset 1048841b plexoffset 524288b sd name myvol.p2.s2 drive c plex myvol.p2 state init len 524288b driveoffset 1048841b plexoffset 1048576b sd name myvol.p2.s3 drive d plex myvol.p2 state init len 524288b driveoffset 1048841b plexoffset 1572864b sd name bigraid.p0.s0 drive a plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 0b sd name bigraid.p0.s1 drive b plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 4194304b sd name bigraid.p0.s2 drive c plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 8388608b sd name bigraid.p0.s3 drive d plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 12582912b sd name bigraid.p0.s4 drive e plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 16777216b .... The obvious differences here are the presence of explicit location information and naming, both of which are allowed but discouraged, and the information on the states. [.filename]#vinum# does not store information about drives in the configuration information. It finds the drives by scanning the configured disk drives for partitions with a [.filename]#vinum# label. This enables [.filename]#vinum# to identify drives correctly even if they have been assigned different UNIX(R) drive IDs. [[vinum-rc-startup]] ==== Automatic Startup _Gvinum_ always features an automatic startup once the kernel module is loaded, via man:loader.conf[5]. To load the _Gvinum_ module at boot time, add `geom_vinum_load="YES"` to [.filename]#/boot/loader.conf#. When [.filename]#vinum# is started with `gvinum start`, [.filename]#vinum# reads the configuration database from one of the [.filename]#vinum# drives. Under normal circumstances, each drive contains an identical copy of the configuration database, so it does not matter which drive is read. After a crash, however, [.filename]#vinum# must determine which drive was updated most recently and read the configuration from this drive. It then updates the configuration, if necessary, from progressively older drives. [[vinum-root]] == Using [.filename]#vinum# for the Root File System For a machine that has fully-mirrored file systems using [.filename]#vinum#, it is desirable to also mirror the root file system. Setting up such a configuration is less trivial than mirroring an arbitrary file system because: * The root file system must be available very early during the boot process, so the [.filename]#vinum# infrastructure must already be available at this time. * The volume containing the root file system also contains the system bootstrap and the kernel. These must be read using the host system's native utilities, such as the BIOS, which often cannot be taught about the details of [.filename]#vinum#. In the following sections, the term "root volume" is generally used to describe the [.filename]#vinum# volume that contains the root file system. === Starting up [.filename]#vinum# Early Enough for the Root File System [.filename]#vinum# must be available early in the system boot as man:loader[8] must be able to load the vinum kernel module before starting the kernel. This can be accomplished by putting this line in [.filename]#/boot/loader.conf#: [.programlisting] .... geom_vinum_load="YES" .... === Making a [.filename]#vinum#-based Root Volume Accessible to the Bootstrap The current FreeBSD bootstrap is only 7.5 KB of code and does not understand the internal [.filename]#vinum# structures. This means that it cannot parse the [.filename]#vinum# configuration data or figure out the elements of a boot volume. Thus, some workarounds are necessary to provide the bootstrap code with the illusion of a standard `a` partition that contains the root file system. For this to be possible, the following requirements must be met for the root volume: * The root volume must not be a stripe or `RAID`-5. * The root volume must not contain more than one concatenated subdisk per plex. Note that it is desirable and possible to use multiple plexes, each containing one replica of the root file system. The bootstrap process will only use one replica for finding the bootstrap and all boot files, until the kernel mounts the root file system. Each single subdisk within these plexes needs its own `a` partition illusion, for the respective device to be bootable. It is not strictly needed that each of these faked `a` partitions is located at the same offset within its device, compared with other devices containing plexes of the root volume. However, it is probably a good idea to create the [.filename]#vinum# volumes that way so the resulting mirrored devices are symmetric, to avoid confusion. To set up these `a` partitions for each device containing part of the root volume, the following is required: [.procedure] ==== . The location, offset from the beginning of the device, and size of this device's subdisk that is part of the root volume needs to be examined, using the command: + [source,shell] .... # gvinum l -rv root .... + [.filename]#vinum# offsets and sizes are measured in bytes. They must be divided by 512 to obtain the block numbers that are to be used by `bsdlabel`. . Run this command for each device that participates in the root volume: + [source,shell] .... # bsdlabel -e devname .... + _devname_ must be either the name of the disk, like [.filename]#da0# for disks without a slice table, or the name of the slice, like [.filename]#ad0s1#. + If there is already an `a` partition on the device from a pre-[.filename]#vinum# root file system, it should be renamed to something else so that it remains accessible (just in case), but will no longer be used by default to bootstrap the system. A currently mounted root file system cannot be renamed, so this must be executed either when being booted from a "Fixit" media, or in a two-step process where, in a mirror, the disk that is not been currently booted is manipulated first. + The offset of the [.filename]#vinum# partition on this device (if any) must be added to the offset of the respective root volume subdisk on this device. The resulting value will become the `offset` value for the new `a` partition. The `size` value for this partition can be taken verbatim from the calculation above. The `fstype` should be `4.2BSD`. The `fsize`, `bsize`, and `cpg` values should be chosen to match the actual file system, though they are fairly unimportant within this context. + That way, a new `a` partition will be established that overlaps the [.filename]#vinum# partition on this device. `bsdlabel` will only allow for this overlap if the [.filename]#vinum# partition has properly been marked using the `vinum` fstype. . A faked `a` partition now exists on each device that has one replica of the root volume. It is highly recommendable to verify the result using a command like: + [source,shell] .... # fsck -n /dev/devnamea .... ==== It should be remembered that all files containing control information must be relative to the root file system in the [.filename]#vinum# volume which, when setting up a new [.filename]#vinum# root volume, might not match the root file system that is currently active. So in particular, [.filename]#/etc/fstab# and [.filename]#/boot/loader.conf# need to be taken care of. At next reboot, the bootstrap should figure out the appropriate control information from the new [.filename]#vinum#-based root file system, and act accordingly. At the end of the kernel initialization process, after all devices have been announced, the prominent notice that shows the success of this setup is a message like: [source,shell] .... Mounting root from ufs:/dev/gvinum/root .... === Example of a [.filename]#vinum#-based Root Setup After the [.filename]#vinum# root volume has been set up, the output of `gvinum l -rv root` could look like: [source,shell] .... ... Subdisk root.p0.s0: Size: 125829120 bytes (120 MB) State: up Plex root.p0 at offset 0 (0 B) Drive disk0 (/dev/da0h) at offset 135680 (132 kB) Subdisk root.p1.s0: Size: 125829120 bytes (120 MB) State: up Plex root.p1 at offset 0 (0 B) Drive disk1 (/dev/da1h) at offset 135680 (132 kB) .... The values to note are `135680` for the offset, relative to partition [.filename]#/dev/da0h#. This translates to 265 512-byte disk blocks in `bsdlabel`'s terms. Likewise, the size of this root volume is 245760 512-byte blocks. [.filename]#/dev/da1h#, containing the second replica of this root volume, has a symmetric setup. The bsdlabel for these devices might look like: [source,shell] .... ... 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 245760 281 4.2BSD 2048 16384 0 # (Cyl. 0*- 15*) c: 71771688 0 unused 0 0 # (Cyl. 0 - 4467*) h: 71771672 16 vinum # (Cyl. 0*- 4467*) .... It can be observed that the `size` parameter for the faked `a` partition matches the value outlined above, while the `offset` parameter is the sum of the offset within the [.filename]#vinum# partition `h`, and the offset of this partition within the device or slice. This is a typical setup that is necessary to avoid the problem described in -crossref:vinum[vinum-root-panic, Nothing Boots, the Bootstrap Panics]. +crossref:vinum[vinum-root-panic,"Nothing Boots, the Bootstrap Panics"]. The entire `a` partition is completely within the `h` partition containing all the [.filename]#vinum# data for this device. In the above example, the entire device is dedicated to [.filename]#vinum# and there is no leftover pre-[.filename]#vinum# root partition. === Troubleshooting The following list contains a few known pitfalls and solutions. ==== System Bootstrap Loads, but System Does Not Boot If for any reason the system does not continue to boot, the bootstrap can be interrupted by pressing kbd:[space] at the 10-seconds warning. The loader variable `vinum.autostart` can be examined by typing `show` and manipulated using `set` or `unset`. If the [.filename]#vinum# kernel module was not yet in the list of modules to load automatically, type `load geom_vinum`. When ready, the boot process can be continued by typing `boot -as` which `-as` requests the kernel to ask for the root file system to mount (`-a`) and make the boot process stop in single-user mode (`-s`), where the root file system is mounted read-only. That way, even if only one plex of a multi-plex volume has been mounted, no data inconsistency between plexes is being risked. At the prompt asking for a root file system to mount, any device that contains a valid root file system can be entered. If [.filename]#/etc/fstab# is set up correctly, the default should be something like `ufs:/dev/gvinum/root`. A typical alternate choice would be something like `ufs:da0d` which could be a hypothetical partition containing the pre-[.filename]#vinum# root file system. Care should be taken if one of the alias `a` partitions is entered here, that it actually references the subdisks of the [.filename]#vinum# root device, because in a mirrored setup, this would only mount one piece of a mirrored root device. If this file system is to be mounted read-write later on, it is necessary to remove the other plex(es) of the [.filename]#vinum# root volume since these plexes would otherwise carry inconsistent data. ==== Only Primary Bootstrap Loads If [.filename]#/boot/loader# fails to load, but the primary bootstrap still loads (visible by a single dash in the left column of the screen right after the boot process starts), an attempt can be made to interrupt the primary bootstrap by pressing kbd:[space]. This will make the bootstrap stop in extref:{handbook}boot[stage two, boot-boot1]. An attempt can be made here to boot off an alternate partition, like the partition containing the previous root file system that has been moved away from `a`. [[vinum-root-panic]] ==== Nothing Boots, the Bootstrap Panics This situation will happen if the bootstrap had been destroyed by the [.filename]#vinum# installation. Unfortunately, [.filename]#vinum# accidentally leaves only 4 KB at the beginning of its partition free before starting to write its [.filename]#vinum# header information. However, the stage one and two bootstraps plus the bsdlabel require 8 KB. So if a [.filename]#vinum# partition was started at offset 0 within a slice or disk that was meant to be bootable, the [.filename]#vinum# setup will trash the bootstrap. Similarly, if the above situation has been recovered, by booting from a "Fixit" media, and the bootstrap has been re-installed using `bsdlabel -B` as described in extref:{handbook}boot[stage two, boot-boot1], the bootstrap will trash the [.filename]#vinum# header, and [.filename]#vinum# will no longer find its disk(s). Though no actual [.filename]#vinum# configuration data or data in [.filename]#vinum# volumes will be trashed, and it would be possible to recover all the data by entering exactly the same [.filename]#vinum# configuration data again, the situation is hard to fix. It is necessary to move the entire [.filename]#vinum# partition by at least 4 KB, to have the [.filename]#vinum# header and the system bootstrap no longer collide. diff --git a/documentation/content/en/books/fdp-primer/asciidoctor-primer/_index.adoc b/documentation/content/en/books/fdp-primer/asciidoctor-primer/_index.adoc index d5aae7675a..67ba576464 100644 --- a/documentation/content/en/books/fdp-primer/asciidoctor-primer/_index.adoc +++ b/documentation/content/en/books/fdp-primer/asciidoctor-primer/_index.adoc @@ -1,363 +1,375 @@ --- title: Chapter 6. Asciidoctor Primer prev: books/fdp-primer/doc-build next: books/fdp-primer/rosetta description: A brief introduction to Asciidoctor tags: ["AsciiDoc", "Asciidoctor", "Primer", "Introduction", "Guide"] showBookMenu: true weight: 7 params: path: "/books/fdp-primer/asciidoctor-primer/" --- [[asciidoctor-primer]] = Asciidoctor Primer :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumoffset: 6 :partnums: :source-highlighter: rouge :experimental: :images-path: books/fdp-primer/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] Most FDP documentation is written with AsciiDoc. This chapter explains what that means, how to read and understand the documentation source, and the techniques used. To get a complete reference of the Asciidoctor capabilities please consult the link:https://docs.asciidoctor.org/home/[Asciidoctor documentation]. Some of the examples used in this chapter have been taken from the link:https://docs.asciidoctor.org/asciidoc/latest/syntax-quick-reference[AsciiDoc Syntax Quick Reference]. [[asciidoctor-primer-overview]] == Overview In the original days of computers, electronic text was simple. There were a few character sets like ASCII or EBCDIC, but that was about it. Text was text, and what you saw really was what you got. No frills, no formatting, no intelligence. Inevitably, this was not enough. When text is in a machine-usable format, machines are expected to be able to use and manipulate it intelligently. Authors want to indicate that certain phrases should be emphasized, or added to a glossary, or made into hyperlinks. Filenames could be shown in a “typewriter” style font for viewing on screen, but as “italics” when printed, or any of a myriad of other options for presentation. It was once hoped that Artificial Intelligence (AI) would make this easy. The computer would read the document and automatically identify key phrases, filenames, text that the reader should type in, examples, and more. Unfortunately, real life has not happened quite like that, and computers still require assistance before they can meaningfully process text. More precisely, they need help identifying what is what. Consider this text: To remove [.filename]#/tmp/foo#, use man:rm[1]. [source,shell] ---- % rm /tmp/foo ---- It is easy for the reader to see which parts are filenames, which are commands to be typed in, which parts are references to manual pages, and so on. But the computer processing the document cannot reliably determine this. For this we need markup. The previous example is actually represented in this document like this: .... To remove */tmp/foo*, use man:rm[1]. [source,shell] ---- % rm /tmp/foo ---- .... [[asciidoctor-headings]] == Headings Asciidoctor supports six headings levels. If the document type is `article` only one level 0 (`=`) can be used. If the document type is `book` then there can be multiple level 0 (`=`) headings. This is an example of headings in an `article`. .... = Document Title (Level 0) == Level 1 Section Title === Level 2 Section Title ==== Level 3 Section Title ===== Level 4 Section Title ====== Level 5 Section Title == Another Level 1 Section Title .... [WARNING] ==== Section levels cannot be skipped when nesting sections. The following syntax is not correct. .... = Document Title == Level 1 ==== Level 3 .... ==== [[asciidoctor-paragraphs]] == Paragraphs Paragraphs don't require special markup in AsciiDoc. A paragraph is defined by one or more consecutive lines of text. To create a new paragraph leave one blank line. For example, this is a heading with two paragraphs. .... = This is the heading This is the first paragraph. This is also the first paragraph. And this is the second paragraph. .... [[asciidoctor-lists]] == Lists Asciidoctor supports a few types of lists, the most common are `ordered` and `unordered`. To get more information about lists, see link:https://docs.asciidoctor.org/asciidoc/latest/syntax-quick-reference/#lists[AsciiDoc Syntax Quick Reference]. [[asciidoctor-ordered-lists]] === Ordered lists To create an ordered list use the `.` character. For example, this is an ordered list. .... . First item . Second item .. Subsecond item . Third item .... And this would be rendered as. . First item . Second item .. Subsecond item . Third item [[asciidoctor-unordered-lists]] === Unordered lists To create an unordered list use the `*` character. For example, this is an unordered list. .... * First item * Second item ** Subsecond item * Third item .... And this would be rendered as. * First item * Second item ** Subsecond item * Third item [[asciidoctor-links]] == Links [[asciidoctor-links-external]] === External links To point to another website the `link` macro should be used. .... link:https://www.FreeBSD.org[FreeBSD] .... [NOTE] ==== As the Asciidoctor documentation describes, the `link` macro is not required when the target starts with a URL scheme like `https`. However, it is a good practice to do this anyway to ensure that Asciidoctor renders the link correctly, especially in non-latin languages like Japanese. ==== [[asciidoctor-links-internal]] === Links to another book or article To point to another book or article the Asciidoctor variables should be used. For example, if we are in the `cups` article and we want to point to `ipsec-must` these steps should be used. . Include the [.filename]#urls.adoc# file from [.filename]#~/doc/shared# folder. + .... \include::shared/{lang}/urls.adoc[] .... + . Then create a link using the Asciidoctor variable to the `ipsec-must` article. + .... extref:{ipsec-must}[IPSec-Must article] .... + And this would be rendered as. + extref:{ipsec-must}[IPSec-Must article] [NOTE] ==== The `extref` macro is defined as an extension. It is designed to render the correct link across the different outputs ==== === Links to the same file or to another file in the same book Books are structured in different directories to keep a sane layout. To create a link from one subdirectory of a book to another subdirectory of the same book, use the `crossref` macro: .... crossref:doc-build[documentation-makefile, This link] .... And this would be rendered as crossref:doc-build[documentation-makefile, This link] [NOTE] ==== The `crossref` macro is defined as an extension. It is designed to render the correct link across the different outputs ==== [NOTE] ==== Use the `crossref` macro for intra-document links too. Although it might be inconvenient to write the name of the current document, it ensures the correct link is rendered across the different outputs ==== [WARNING] ==== Do not use either the `xref` macro or its shortcut `<<` `>>`. They do not work well in all output formats. ==== +[IMPORTANT] +==== +When the second parameter of the `crossref` macro contains a comma, it must be enclosed in double quotes. +Otherwise, the comma is interpreted as a parameter separator, and the text after the comma is stripped. + +For example: +.... +crossref:plist[documentation-comma, "First, second"] +.... +Without quotes, only `First` would be used as the link text, and the rest would be lost. +==== + [[asciidoctor-images-icons]] == Images and Icons Images and icons play a crucial role in enhancing the overall user experience. These visual elements are strategically integrated to convey information, clarify concepts, and provide a visually engaging interface. [[asciidoctor-images]] === Images Images help illustrate complex concepts, making them more accessible to users. The first step will be to add the image in the images directory in the path: * [.filename]#~/website/static/images/# for the website. * [.filename]#~/documentation/static/images/# for the documentation. For example, to add a new image to the FreeBSD installation process, the image will be saved to the path [.filename]#~/documentation/static/images/books/handbook/bsdinstall/new-image3.png#. The next step will be to configure the Asciidoctor attributes `images-path` and `imagesdir`. We are going to use as an example the header of the extref:{freebsd-releng}[FreeBSD Release Engineering] article. [source,asciidoc] .... = FreeBSD Release Engineering :doctype: article [...] :images-path: articles/freebsd-releng/ <1> [...] :imagesdir: ../../../images/{images-path} <2> [...] .... <.> Makes reference to the path inside [.filename]#/static/images# folder. <.> Makes reference to the Asciidoctor attribute. Once the image is in the correct path and the Asciidoctor attributes have been configured in the document, the `image` macro can be used. This is an example: .... image::new-image3.png[New step in the FreeBSD install process] .... [TIP] ==== To improve accessibility, it is mandatory to add descriptive text to each image. ==== [[asciidoctor-icons]] === Icons Icons serve as intuitive symbols for quick recognition and navigation. The first step to use icons is to add the `icons` property to the Asciidoctor properties section, at the top of each document. .... :icons: font .... Once the Asciidoctor icon property has been set an icon supported by link:https://fontawesome.com/v4/icons/[Font Awesome] can be added. This is an example about how to use the `envelope` icon: .... icon:envelope[link=mailto:test@example.com, title="contact"] .... [TIP] ==== To improve the accessibility of the website, the `title` attribute is mandatory. ==== [[asciidoctor-conclusion]] == Conclusion This is the conclusion of this Asciidoctor primer. For reasons of space and complexity, several things have not been covered in depth (or at all). diff --git a/documentation/content/en/books/handbook/basics/_index.adoc b/documentation/content/en/books/handbook/basics/_index.adoc index 8a488681d8..88c348a858 100644 --- a/documentation/content/en/books/handbook/basics/_index.adoc +++ b/documentation/content/en/books/handbook/basics/_index.adoc @@ -1,1962 +1,1962 @@ --- title: Chapter 3. FreeBSD Basics part: Part I. Getting Started prev: books/handbook/bsdinstall next: books/handbook/ports description: Basic commands and functionality of the FreeBSD operating system tags: ["basics", "virtual consoles", "users", "management", "permissions", "directory structure", "disk organization", "mounting", "processes", "daemons", "shell", "editor", "manual pages", "devices"] showBookMenu: true weight: 5 params: path: "/books/handbook/basics/" --- [[basics]] = FreeBSD Basics :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumoffset: 3 :partnums: :source-highlighter: rouge :experimental: :images-path: books/handbook/basics/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[basics-synopsis]] == Synopsis This chapter covers the basic commands and functionality of the FreeBSD operating system. Much of this material is relevant for any UNIX(R)-like operating system. New FreeBSD users are encouraged to read through this chapter carefully. Read this chapter to learn: * How to use and configure virtual consoles. * How to create and manage users and groups on FreeBSD. * How UNIX(R) file permissions and FreeBSD file flags work. * The default FreeBSD file system layout. * The FreeBSD disk organization. * How to mount and unmount file systems. * What processes, daemons, and signals are. * What a shell is, and how to change the default login environment. * How to use basic text editors. * What devices and device nodes are. * How to read manual pages for more information. [[consoles]] == Virtual Consoles and Terminals Unless FreeBSD has been configured to automatically start a graphical environment during startup, the system will boot into a command line login prompt, as seen in this example: [.programlisting] .... FreeBSD/amd64 (pc3.example.org) (ttyv0) login: .... The first line contains some information about the system. The `amd64` indicates that FreeBSD is running on a 64-bit x86 system. The hostname is `pc3.example.org`, and `ttyv0` indicates that this is the "system console". The second line is the login prompt. Since FreeBSD is a multiuser system, it needs some way to distinguish between different users. This is accomplished by requiring every user to log into the system before gaining access to the programs on the system. Every user has a unique "username" and a personal "password". To log into the system console, type the username that was configured during system installation, as described in crossref:bsdinstall[bsdinstall-addusers,Add Users], and press kbd:[Enter]. Then enter the password associated with the username and press kbd:[Enter]. The password is _not echoed_ for security reasons. Once the correct password is input, the message of the day (MOTD) will be displayed followed by a command prompt. Depending upon the shell that was selected when the user was created, this prompt will be a `+#+`, `$`, or `%` character. The prompt indicates that the user is now logged into the FreeBSD system console and ready to try the available commands. [[consoles-virtual]] === Virtual Consoles While the system console can be used to interact with the system, a user working from the command line at the keyboard of a FreeBSD system will typically instead log into a virtual console. This is because system messages are configured by default to display on the system console. These messages will appear over the command or file that the user is working on, making it difficult to concentrate on the work at hand. By default, FreeBSD is configured to provide several virtual consoles for inputting commands. Each virtual console has its own login prompt and shell and it is easy to switch between virtual consoles. This essentially provides the command line equivalent of having several windows open at the same time in a graphical environment. The key combinations kbd:[Alt+F1] through kbd:[Alt+F8] have been reserved by FreeBSD for switching between virtual consoles. Use kbd:[Alt+F1] to switch to the system console (`ttyv0`), kbd:[Alt+F2] to access the first virtual console (`ttyv1`), kbd:[Alt+F3] to access the second virtual console (`ttyv2`), and so on. When using Xorg as a graphical console, the combination becomes kbd:[Ctrl+Alt+F1] to return to a text-based virtual console. When switching from one console to the next, FreeBSD manages the screen output. The result is an illusion of having multiple virtual screens and keyboards that can be used to type commands for FreeBSD to run. The programs that are launched in one virtual console do not stop running when the user switches to a different virtual console. Refer to man:kbdcontrol[1], man:vidcontrol[1], man:atkbd[4], man:syscons[4], and man:vt[4] for a more technical description of the FreeBSD console and its keyboard drivers. In FreeBSD, the number of available virtual consoles is configured in this section of `/etc/ttys`: [.programlisting] .... # name getty type status comments # ttyv0 "/usr/libexec/getty Pc" xterm on secure # Virtual terminals ttyv1 "/usr/libexec/getty Pc" xterm on secure ttyv2 "/usr/libexec/getty Pc" xterm on secure ttyv3 "/usr/libexec/getty Pc" xterm on secure ttyv4 "/usr/libexec/getty Pc" xterm on secure ttyv5 "/usr/libexec/getty Pc" xterm on secure ttyv6 "/usr/libexec/getty Pc" xterm on secure ttyv7 "/usr/libexec/getty Pc" xterm on secure ttyv8 "/usr/X11R6/bin/xdm -nodaemon" xterm off secure .... To disable a virtual console, put a comment symbol (`+#+`) at the beginning of the line representing that virtual console. For example, to reduce the number of available virtual consoles from eight to four, put a `+#+` in front of the last four lines representing virtual consoles `ttyv5` through `ttyv8`. _Do not_ comment out the line for the system console `ttyv0`. Note that the last virtual console (`ttyv8`) is used to access the graphical environment if Xorg has been installed and configured as described in crossref:x11[x11,The X Window System]. For a detailed description of every column in this file and the available options for the virtual consoles, refer to man:ttys[5]. [[consoles-singleuser]] === Single User Mode The FreeBSD boot menu provides an option labelled as "Boot Single User". If this option is selected, the system will boot into a special mode known as "single user mode". This mode is typically used to repair a system that will not boot or to reset the `root` password when it is not known. While in single user mode, networking and other virtual consoles are not available. However, full `root` access to the system is available, and by default, the `root` password is not needed. For these reasons, physical access to the keyboard is needed to boot into this mode and determining who has physical access to the keyboard is something to consider when securing a FreeBSD system. The settings which control single user mode are found in this section of `/etc/ttys`: [.programlisting] .... # name getty type status comments # # If console is marked "insecure", then init will ask for the root password # when going to single-user mode. console none unknown off secure .... By default, the status is set to `secure`. This assumes that who has physical access to the keyboard is either not important or it is controlled by a physical security policy. If this setting is changed to `insecure`, the assumption is that the environment itself is insecure because anyone can access the keyboard. When this line is changed to `insecure`, FreeBSD will prompt for the `root` password when a user selects to boot into single user mode. [NOTE] ==== _Be careful when changing this setting to `insecure`!_ If the `root` password is forgotten, booting into single user mode is still possible, but may be difficult for someone who is not familiar with the FreeBSD booting process. ==== [[consoles-vidcontrol]] === Changing Console Video Modes The FreeBSD console default video mode may be adjusted to 1024x768, 1280x1024, or any other size supported by the graphics chip and monitor. To use a different video mode load the `VESA` module: [source,shell] .... # kldload vesa .... To determine which video modes are supported by the hardware, use man:vidcontrol[1]. To get a list of supported video modes issue the following: [source,shell] .... # vidcontrol -i mode .... The output of this command lists the video modes that are supported by the hardware. To select a new video mode, specify the mode using man:vidcontrol[1] as the `root` user: [source,shell] .... # vidcontrol MODE_279 .... If the new video mode is acceptable, it can be permanently set on boot by adding it to `/etc/rc.conf`: [.programlisting] .... allscreens_flags="MODE_279" .... [[users-synopsis]] == Users and Basic Account Management FreeBSD allows multiple users to use the computer at the same time. While only one user can sit in front of the screen and use the keyboard at any one time, any number of users can log in to the system through the network. To use the system, each user should have their own user account. This chapter describes: * The different types of user accounts on a FreeBSD system. * How to add, remove, and modify user accounts. * How to set limits to control the resources that users and groups are allowed to access. * How to create groups and add users as members of a group. [[users-introduction]] === Account Types Since all access to the FreeBSD system is achieved using accounts and all processes are run by users, user and account management is important. There are three main types of accounts: system accounts, user accounts, and the superuser account. [[users-system]] ==== System Accounts System accounts are used to run services such as DNS, mail, and web servers. The reason for this is security; if all services ran as the superuser, they could act without restriction. Examples of system accounts are `daemon`, `operator`, `bind`, `news`, and `www`. `nobody` is the generic unprivileged system account. However, the more services that use `nobody`, the more files and processes that user will become associated with, and hence the more privileged that user becomes. [[users-user]] ==== User Accounts User accounts are assigned to real people and are used to log in and use the system. Every person accessing the system should have a unique user account. This allows the administrator to find out who is doing what and prevents users from clobbering the settings of other users. Each user can set up their own environment to accommodate their use of the system, by configuring their default shell, editor, key bindings, and language settings. Every user account on a FreeBSD system has certain information associated with it: User name:: The user name is typed at the `login:` prompt. Each user must have a unique user name. There are a number of rules for creating valid user names which are documented in man:passwd[5]. It is recommended to use user names that consist of eight or fewer, all lower case characters in order to maintain backwards compatibility with applications. Password:: Each account has an associated password. User ID (UID):: The User ID (UID) is a number used to uniquely identify the user to the FreeBSD system. Commands that allow a user name to be specified will first convert it to the UID. It is recommended to use a UID less than 65535, since higher values may cause compatibility issues with some software. Group ID (GID):: The Group ID (GID) is a number used to uniquely identify the primary group that the user belongs to. Groups are a mechanism for controlling access to resources based on a user's GID rather than their UID. This can significantly reduce the size of some configuration files and allows users to be members of more than one group. It is recommended to use a GID of 65535 or lower as higher GIDs may break some software. Login class:: Login classes are an extension to the group mechanism that provide additional flexibility when tailoring the system to different users. Login classes are discussed further in crossref:security[users-limiting,Configuring Login Classes]. Password change time:: By default, passwords do not expire. However, password expiration can be enabled on a per-user basis, forcing some or all users to change their passwords after a certain amount of time has elapsed. Account expiration time:: By default, FreeBSD does not expire accounts. When creating accounts that need a limited lifespan, such as student accounts in a school, specify the account expiry date using man:pw[8]. After the expiry time has elapsed, the account cannot be used to log in to the system, although the account's directories and files will remain. User's full name:: The user name uniquely identifies the account to FreeBSD, but does not necessarily reflect the user's real name. Similar to a comment, this information can contain spaces, uppercase characters, and be more than 8 characters long. Home directory:: The home directory is the full path to a directory on the system. This is the user's starting directory when the user logs in. A common convention is to put all user home directories under `/home/username` or `/usr/home/username`. Each user stores their personal files and subdirectories in their own home directory. User shell:: The shell provides the user's default environment for interacting with the system. There are many different kinds of shells and experienced users will have their own preferences, which can be reflected in their account settings. [[users-superuser]] ==== The Superuser Account The superuser account, usually called `root`, is used to manage the system with no limitations on privileges. For this reason, it should not be used for day-to-day tasks like sending and receiving mail, general exploration of the system, or programming. The superuser, unlike other user accounts, can operate without limits, and misuse of the superuser account may result in spectacular disasters. User accounts are unable to destroy the operating system by mistake, so it is recommended to login as a user account and to only become the superuser when a command requires extra privilege. Always double and triple-check any commands issued as the superuser, since an extra space or missing character can mean irreparable data loss. There are several ways to gain superuser privilege. While one can log in as `root`, this is highly discouraged. Instead, use man:su[1] to become the superuser. If `-` is specified when running this command, the user will also inherit the root user's environment. The user running this command must be in the `wheel` group or else the command will fail. The user must also know the password for the `root` user account. In this example, the user only becomes superuser in order to run `make install` as this step requires superuser privilege. Once the command completes, the user types `exit` to leave the superuser account and return to the privilege of their user account. .Install a Program As the Superuser [example] ==== [source,shell] .... % configure % make % su - Password: # make install # exit % .... ==== The built-in man:su[1] framework works well for single systems or small networks with just one system administrator. An alternative is to install the package:security/sudo[] package or port. This software provides activity logging and allows the administrator to configure which users can run which commands as the superuser. [[users-modifying]] === Managing Accounts FreeBSD provides a variety of different commands to manage user accounts. The most common commands are summarized in crossref:basics[users-modifying-utilities,Utilities for Managing User Accounts], followed by some examples of their usage. See the manual page for each utility for more details and usage examples. [[users-modifying-utilities]] .Utilities for Managing User Accounts [cols="25h,~"] |=== | Command | Summary |man:adduser[8] |The recommended command-line application for adding new users. |man:rmuser[8] |The recommended command-line application for removing users. |man:chpass[1] |A flexible tool for changing user database information. |man:passwd[1] |The command-line tool to change user passwords. |man:pw[8] |A powerful and flexible tool for modifying all aspects of user accounts. |man:bsdconfig[8] |A system configuration utility with account management support. |=== [[users-adduser]] ==== Adding a user The recommended program for adding new users is man:adduser[8]. When a new user is added, this program automatically updates `/etc/passwd` and `/etc/group`. It also creates a home directory for the new user, copies in the default configuration files from `/usr/share/skel`, and can optionally mail the new user a welcome message. This utility must be run as the superuser. The man:adduser[8] utility is interactive and walks through the steps for creating a new user account. As seen in crossref:basics[users-modifying-adduser, Adding a User on FreeBSD], either input the required information or press kbd:[Return] to accept the default value shown in square brackets. In this example, the user has been invited into the `wheel` group, allowing them to become the superuser with man:su[1]. When finished, the utility will prompt to either create another user or to exit. [[users-modifying-adduser]] .Adding a User on FreeBSD [example] ==== [source,shell] .... # adduser .... The output should be similar to the following: [.programlisting] .... Username: jru Full name: J. Random User Uid (Leave empty for default): Login group [jru]: Login group is jru. Invite jru into other groups? []: wheel Login class [default]: Shell (sh csh tcsh zsh nologin) [sh]: zsh Home directory [/home/jru]: Home directory permissions (Leave empty for default): Use password-based authentication? [yes]: Use an empty password? (yes/no) [no]: Use a random password? (yes/no) [no]: Enter password: Enter password again: Lock out the account after creation? [no]: Username : jru Password : **** Full Name : J. Random User Uid : 1001 Class : Groups : jru wheel Home : /home/jru Shell : /usr/local/bin/zsh Locked : no OK? (yes/no): yes adduser: INFO: Successfully added (jru) to the user database. Add another user? (yes/no): no Goodbye! .... ==== [NOTE] ==== Since the password is not echoed when typed, be careful to not mistype the password when creating the user account. ==== [[users-rmuser]] ==== Removing a user To completely remove a user from the system, run man:rmuser[8] as the superuser. This command performs the following steps: [.procedure] ==== . Removes the user's man:crontab[1] entry, if one exists. . Removes any man:at[1] jobs belonging to the user. . Sends a SIGKILL signal to all processes owned by the user. . Removes the user from the system's local password file. . Removes the user's home directory (if it is owned by the user), including handling of symbolic links in the path to the actual home directory. . Removes the incoming mail files belonging to the user from `/var/mail`. . Removes all files owned by the user from `/tmp`, `/var/tmp`, and `/var/tmp/vi.recover`. . Removes the username from all groups to which it belongs in `/etc/group`. (If a group becomes empty and the group name is the same as the username, the group is removed; this complements man:adduser[8]'s per-user unique groups.) . Removes all message queues, shared memory segments and semaphores owned by the user. ==== man:rmuser[8] cannot be used to remove superuser accounts since that is almost always an indication of massive destruction. By default, an interactive mode is used, as shown in the following example. .`rmuser` Interactive Account Removal [example] ==== [source,shell] .... # rmuser jru .... The output should be similar to the following: [.programlisting] .... Matching password entry: jru:*:1001:1001::0:0:J. Random User:/home/jru:/usr/local/bin/zsh Is this the entry you wish to remove? y Remove user's home directory (/home/jru)? y Removing user (jru): mailspool home passwd. .... ==== [[users-chpass]] ==== Change user information Any user can use man:chpass[1] to change their default shell and personal information associated with their user account. The superuser can use this utility to change additional account information for any user. When passed no options, aside from an optional username, man:chpass[1] displays an editor containing user information. When the user exits from the editor, the user database is updated with the new information. [NOTE] ==== This utility will prompt for the user's password when exiting the editor, unless the utility is run as the superuser. ==== In crossref:basics[users-modifying-chpass-su,Using `chpass` as Superuser], the superuser has typed `chpass jru` and is now viewing the fields that can be changed for this user. If `jru` runs this command instead, only the last six fields will be displayed and available for editing. This is shown in crossref:basics[users-modifying-chpass-ru,Using `chpass` as Regular User]. [[users-modifying-chpass-su]] .Using `chpass` as Superuser [example] ==== [source,shell] .... # chpass jru .... The output should be similar to the following: [.programlisting] .... # Changing user database information for jru. Login: jru Password: * Uid [#]: 1001 Gid [# or name]: 1001 Change [month day year]: Expire [month day year]: Class: Home directory: /home/jru Shell: /usr/local/bin/zsh Full Name: J. Random User Office Location: Office Phone: Home Phone: Other information: .... ==== [[users-modifying-chpass-ru]] .Using `chpass` as Regular User [example] ==== [source,shell] .... #Changing user database information for jru. Shell: /usr/local/bin/zsh Full Name: J. Random User Office Location: Office Phone: Home Phone: Other information: .... ==== [NOTE] ==== The commands man:chfn[1] and man:chsh[1] are links to man:chpass[1], as are man:ypchpass[1], man:ypchfn[1], and man:ypchsh[1]. Since NIS support is automatic, specifying the `yp` before the command is not necessary. How to configure NIS is covered in crossref:network-servers[network-servers,Network Servers]. ==== [[users-passwd]] ==== Change user password Any user can easily change their password using man:passwd[1]. To prevent accidental or unauthorized changes, this command will prompt for the user's original password before a new password can be set: .Changing The Password [example] ==== [source,shell] .... % passwd .... The output should be similar to the following: [.programlisting] .... Changing local password for jru. Old password: New password: Retype new password: passwd: updating the database... passwd: done .... ==== The superuser can change any user's password by specifying the username when running man:passwd[1]. When this utility is run as the superuser, it will not prompt for the user's current password. This allows the password to be changed when a user cannot remember the original password. .Changing Another User's Password as the Superuser [example] ==== [source,shell] .... # passwd jru .... The output should be similar to the following: [.programlisting] .... Changing local password for jru. New password: Retype new password: passwd: updating the database... passwd: done .... ==== [NOTE] ==== As with man:chpass[1], man:yppasswd[1] is a link to man:passwd[1], so NIS works with either command. ==== [[users-pw]] ==== Create, remove, modify and display system users and groups The man:pw[8] utility can create, remove, modify, and display users and groups. It functions as a front end to the system user and group files. man:pw[8] has a very powerful set of command line options that make it suitable for use in shell scripts, but new users may find it more complicated than the other commands presented in this section. [[users-groups]] === Managing Groups A group is a list of users. A group is identified by its group name and GID. In FreeBSD, the kernel uses the UID of a process, and the list of groups it belongs to, to determine what the process is allowed to do. Most of the time, the GID of a user or process usually means the first group in the list. The group name to GID mapping is listed in `/etc/group`. This is a plain text file with four colon-delimited fields. The first field is the group name, the second is the encrypted password, the third the GID, and the fourth the comma-delimited list of members. For a complete description of the syntax, refer to man:group[5]. The superuser can modify `/etc/group` using a text editor, although editing the group file using man:vigr[8] is preferred because it can catch some common mistakes. Alternatively, man:pw[8] can be used to add and edit groups. For example, to add a group called `teamtwo` and then confirm that it exists: [WARNING] ==== Care must be taken when using the operator group, as unintended superuser-like access privileges may be granted, including but not limited to shutdown, reboot, and access to all items in `/dev` in the group. ==== .Adding a Group Using man:pw[8] [example] ==== [source,shell] .... # pw groupadd teamtwo # pw groupshow teamtwo .... The output should be similar to the following: [.programlisting] .... teamtwo:*:1100: .... ==== In this example, `1100` is the GID of `teamtwo`. Right now, `teamtwo` has no members. This command will add `jru` as a member of `teamtwo`. .Adding User Accounts to a New Group Using man:pw[8] [example] ==== [source,shell] .... # pw groupmod teamtwo -M jru # pw groupshow teamtwo .... The output should be similar to the following: [.programlisting] .... teamtwo:*:1100:jru .... ==== The argument to `-M` is a comma-delimited list of users to be added to a new (empty) group or to replace the members of an existing group. To the user, this group membership is different from (and in addition to) the user's primary group listed in the password file. This means that the user will not show up as a member when using `groupshow` with man:pw[8], but will show up when the information is queried via man:id[1] or a similar tool. When man:pw[8] is used to add a user to a group, it only manipulates `/etc/group` and does not attempt to read additional data from `/etc/passwd`. .Adding a New Member to a Group Using man:pw[8] [example] ==== [source,shell] .... # pw groupmod teamtwo -m db # pw groupshow teamtwo .... The output should be similar to the following: [.programlisting] .... teamtwo:*:1100:jru,db .... ==== In this example, the argument to `-m` is a comma-delimited list of users who are to be added to the group. Unlike the previous example, these users are appended to the group and do not replace existing users in the group. .Using man:id[1] to Determine Group Membership [example] ==== [source,shell] .... % id jru .... The output should be similar to the following: [.programlisting] .... uid=1001(jru) gid=1001(jru) groups=1001(jru), 1100(teamtwo) .... ==== In this example, `jru` is a member of the groups `jru` and `teamtwo`. For more information about this command and the format of `/etc/group`, refer to man:pw[8] and man:group[5]. [[permissions]] == Permissions In FreeBSD, every file and directory has an associated set of permissions and several utilities are available for viewing and modifying these permissions. Understanding how permissions work is necessary to make sure that users are able to access the files that they need and are unable to improperly access the files used by the operating system or owned by other users. This section discusses the traditional UNIX(R) permissions used in FreeBSD. For finer-grained file system access control, refer to crossref:security[fs-acl,Access Control Lists]. In UNIX(R), basic permissions are assigned using three types of access: read, write, and execute. These access types are used to determine file access to the file's owner, group, and others (everyone else). The read, write, and execute permissions can be represented as the letters `r`, `w`, and `x`. They can also be represented as binary numbers as each permission is either on or off (`0`). When represented as a number, the order is always read as `rwx`, where `r` has an on value of `4`, `w` has an on value of `2` and `x` has an on value of `1`. Table 4.1 summarizes the possible numeric and alphabetic possibilities. When reading the "Directory Listing" column, a `-` is used to represent a permission that is set to off. .UNIX(R) Permissions [cols="1,1,1", frame="none", options="header"] |=== | Value | Permission | Directory Listing |0 |No read, no write, no execute |`---` |1 |No read, no write, execute |`--x` |2 |No read, write, no execute |`-w-` |3 |No read, write, execute |`-wx` |4 |Read, no write, no execute |`r--` |5 |Read, no write, execute |`r-x` |6 |Read, write, no execute |`rw-` |7 |Read, write, execute |`rwx` |=== Use the `-l` argument with man:ls[1] to view a long directory listing that includes a column of information about a file's permissions for the owner, group, and everyone else. For example, `ls -l` in an arbitrary directory may show: [source,shell] .... % ls -l .... The output should be similar to the following: [.programlisting] .... total 530 -rw-r--r-- 1 root wheel 512 Sep 5 12:31 myfile -rw-r--r-- 1 root wheel 512 Sep 5 12:31 otherfile -rw-r--r-- 1 root wheel 7680 Sep 5 12:31 email.txt .... Focusing on the line for `myfile`, the first `(leftmost)` character indicates whether this file is a regular file, a directory, a special character device, a socket, or any other special pseudo-file device. In this example, the `-` indicates a regular file. The next three characters, `rw-` in this example, give the permissions for the owner of the file. The next three characters, `r--`, give the permissions for the group that the file belongs to. The final three characters, `r--`, give the permissions for the rest of the world. A dash means that the permission is turned off. In this example, the permissions are set so the owner can read and write to the file, the group can read the file, and the rest of the world can only read the file. According to the table above, the permissions for this file would be `644`, where each digit represents the three parts of the file's permission. How does the system control permissions on devices? FreeBSD treats most hardware devices as a file that programs can open, read, and write data to. These special device files are stored in `/dev/`. Directories are also treated as files. They have read, write, and execute permissions. The executable bit for a directory has a slightly different meaning than that of files. When a directory is marked executable, it means it is possible to change into that directory using man:cd[1]. This also means that it is possible to access the files within that directory, subject to the permissions on the files themselves. In order to perform a directory listing, the read permission must be set on the directory. In order to delete a file that one knows the name of, it is necessary to have write _and_ execute permissions to the directory containing the file. There are more permission bits, but they are primarily used in special circumstances such as setuid binaries and sticky directories. For more information on file permissions and how to set them, refer to man:chmod[1]. === Symbolic Permissions Symbolic permissions use characters instead of octal values to assign permissions to files or directories. Symbolic permissions use the syntax of (who) (action) (permissions), where the following values are available: [.informaltable] [cols="1,1,1", frame="none", options="header"] |=== | Option | Letter | Represents |(who) |u |User |(who) |g |Group owner |(who) |o |Other |(who) |a |All ("world") |(action) |+ |Adding permissions |(action) |- |Removing permissions |(action) |= |Explicitly set permissions |(permissions) |r |Read |(permissions) |w |Write |(permissions) |x |Execute |(permissions) |t |Sticky bit |(permissions) |s |Set UID or GID |=== These values are used with man:chmod[1], but with letters instead of numbers. For example, the following command would block both members of the group associated with _FILE_ and all other users from accessing _FILE_: [source,shell] .... % chmod go= FILE .... A comma separated list can be provided when more than one set of changes to a file must be made. For example, the following command removes the group and "world" write permission on _FILE_, and adds the execute permissions for everyone: [source,shell] .... % chmod go-w,a+x FILE .... === FreeBSD File Flags In addition to file permissions, FreeBSD supports the use of "file flags". These flags add an additional level of security and control over files, but not directories. With file flags, even `root` can be prevented from removing or altering files. File flags are modified using man:chflags[1]. For example, to enable the system undeletable flag on the file `file1`, issue the following command: [source,shell] .... # chflags sunlink file1 .... To disable the system undeletable flag, put a "no" in front of the `sunlink`: [source,shell] .... # chflags nosunlink file1 .... To view the flags of a file, use `-lo` with man:ls[1]: [source,shell] .... # ls -lo file1 .... [.programlisting] .... -rw-r--r-- 1 trhodes trhodes sunlnk 0 Mar 1 05:54 file1 .... Several file flags may only be added or removed by the `root` user. In other cases, the file owner may set its file flags. Refer to man:chflags[1] and man:chflags[2] for more information. === The setuid, setgid, and sticky Permissions Other than the permissions already discussed, there are three other specific settings that all administrators should know about. They are the `setuid`, `setgid`, and `sticky` permissions. These settings are important for some UNIX(R) operations as they provide functionality not normally granted to normal users. To understand them, the difference between the real user ID and effective user ID must be noted. The real user ID is the UID who owns or starts the process. The effective UID is the user ID the process runs as. As an example, man:passwd[1] runs with the real user ID when a user changes their password. However, in order to update the password database, the command runs as the effective ID of the `root` user. This allows users to change their passwords without seeing a `Permission Denied` error. The setuid permission may be added symbolically by adding the `s` permission for the user as in the following example: [source,shell] .... # chmod u+s suidexample.sh .... The setuid permission may also be set by prefixing a permission set with the number four (4) as shown in the following example: [source,shell] .... # chmod 4755 suidexample.sh .... The permissions on `suidexample.sh` now look like the following: [.programlisting] .... -rwsr-xr-x 1 trhodes trhodes 63 Aug 29 06:36 suidexample.sh .... Note that a `s` is now part of the permission set designated for the file owner, replacing the executable bit. This allows utilities which need elevated permissions, such as man:passwd[1]. [NOTE] ==== The `nosuid` man:mount[8] option will cause such binaries to silently fail without alerting the user. That option is not completely reliable as a `nosuid` wrapper may be able to circumvent it. ==== To view this in real time, open two terminals. On one, type `passwd` as a normal user. While it waits for a new password, check the process table and look at the user information for man:passwd[1]: In terminal A: [source,shell] .... Changing local password for trhodes Old Password: .... In terminal B: [source,shell] .... # ps aux | grep passwd .... [source,shell] .... trhodes 5232 0.0 0.2 3420 1608 0 R+ 2:10AM 0:00.00 grep passwd root 5211 0.0 0.2 3620 1724 2 I+ 2:09AM 0:00.01 passwd .... Although man:passwd[1] is run as a normal user, it is using the effective UID of `root`. The `setgid` permission performs the same function as the `setuid` permission; except that it alters the group settings. When an application or utility executes with this setting, it will be granted the permissions based on the group that owns the file, not the user who started the process. To set the `setgid` permission on a file symbolically, add the `s` permission for the group with man:chmod[1]: [source,shell] .... # chmod g+s sgidexample.sh .... Alternatively, provide man:chmod[1] with a leading two (2): [source,shell] .... # chmod 2755 sgidexample.sh .... In the following listing, notice that the `s` is now in the field designated for the group permission settings: [source,shell] .... -rwxr-sr-x 1 trhodes trhodes 44 Aug 31 01:49 sgidexample.sh .... [NOTE] ==== In these examples, even though the shell script in question is an executable file, it will not run with a different EUID or effective user ID. This is because shell scripts may not access the man:setuid[2] system calls. ==== The `setuid` and `setgid` permission bits may lower system security, by allowing for elevated permissions. The third special permission, the `sticky bit`, can strengthen the security of a system. When the `sticky bit` is set on a directory, it allows file deletion only by the file owner. This is useful to prevent file deletion in public directories, such as `/tmp`, by users who do not own the file. To utilize this permission, add the `t` mode to the file: [source,shell] .... # chmod +t /tmp .... Alternatively, prefix the permission set with a one (1): [source,shell] .... # chmod 1777 /tmp .... The `sticky bit` permission will display as a `t` at the very end of the permission set: [source,shell] .... # ls -al / | grep tmp .... [source,shell] .... drwxrwxrwt 10 root wheel 512 Aug 31 01:49 tmp .... [[dirstructure]] == Directory Structure The FreeBSD directory hierarchy is fundamental to obtaining an overall understanding of the system. The most important directory is root or, "/". This directory is the first one mounted at boot time and it contains the base system necessary to prepare the operating system for multi-user operation. The root directory also contains mount points for other file systems that are mounted during the transition to multi-user operation. A mount point is a directory where additional file systems can be grafted onto a parent file system (usually the root file system). This is further described in crossref:basics[disk-organization, Disk Organization]. Standard mount points include `/usr/`, `/var/`, `/tmp/`, `/mnt/`, and `/media/`. These directories are usually referenced to entries in `/etc/fstab`. This file is a table of various file systems and mount points and is read by the system. Most of the file systems in `/etc/fstab` are mounted automatically at boot time from the script man:rc[8] unless their entry includes `noauto`. Details can be found in crossref:basics[disks-fstab, The fstab File]. A complete description of the file system hierarchy is available in man:hier[7]. The following table provides a brief overview of the most common directories. [cols="25h,~"] |=== | Directory | Description |`/` |Root directory of the file system. |`/bin/` |User utilities fundamental to both single-user and multi-user environments. |`/boot/` |Programs and configuration files used during operating system bootstrap. |`/boot/defaults/` |Default boot configuration files. Refer to man:loader.conf[5] for details. |`/dev/` |Device special files managed by man:devfs[5] |`/etc/` |System configuration files and scripts. |`/etc/defaults/` |Default system configuration files. Refer to man:rc[8] for details. |`/etc/periodic/` |Scripts that run daily, weekly, and monthly, via man:cron[8]. Refer to man:periodic[8] for details. |`/lib/` |Critical system libraries needed for binaries in `/bin` and `/sbin` |`/libexec/` |Critical system files |`/media/` |Contains subdirectories to be used as mount points for removable media such as CDs, USB drives, and floppy disks |`/mnt/` |Empty directory commonly used by system administrators as a temporary mount point. |`/net/` |Automounted NFS shares; see man:auto_master[5] |`/proc/` |Process file system. Refer to man:procfs[5] for details. |`/rescue/` |Statically linked programs for emergency recovery as described in man:rescue[8]. |`/root/` |Home directory for the `root` account. |`/sbin/` |System programs and administration utilities fundamental to both single-user and multi-user environments. |`/tmp/` |Temporary files which are usually _not_ preserved across a system reboot. A memory-based file system is often mounted at `/tmp`. This can be automated using the tmpmfs-related variables of man:rc.conf[5] or with an entry in `/etc/fstab`; refer to man:mdmfs[8] for details. |`/usr/` |The majority of user utilities and applications. |`/usr/bin/` |Common utilities, programming tools, and applications. |`/usr/include/` |Standard C include files. |`/usr/lib/` |Archive libraries. |`/usr/libdata/` |Miscellaneous utility data files. |`/usr/libexec/` |System daemons and system utilities executed by other programs. |`/usr/local/` |Local executables and libraries. Also used as the default destination for the FreeBSD ports framework. Within `/usr/local`, the general layout sketched out by man:hier[7] for `/usr` should be used. Exceptions are the man directory, which is directly under `/usr/local` rather than under `/usr/local/share`, and the ports documentation is in `share/doc/port`. |`/usr/ports/` |The FreeBSD Ports Collection (optional). |`/usr/sbin/` |System daemons and system utilities executed by users. |`/usr/share/` |Architecture-independent files. |`/usr/src/` |BSD and/or local source files. |`/var/` |Multi-purpose log, temporary, transient, and spool files. |`/var/log/` |Miscellaneous system log files. |`/var/tmp/` |Temporary files which are usually preserved across a system reboot. |=== [[disk-organization]] == Disk Organization The smallest unit of organization that FreeBSD uses to find files is the filename. Filenames are case-sensitive, which means that `readme.txt` and `README.TXT` are two separate files. FreeBSD does not use the extension of a file to determine whether the file is a program, document, or some other form of data. Files are stored in directories. A directory may contain no files, or it may contain many hundreds of files. A directory can also contain other directories, allowing a hierarchy of directories within one another in order to organize data. Files and directories are referenced by giving the file or directory name, followed by a forward slash, `/`, followed by any other directory names that are necessary. For example, if the directory `foo` contains a directory `bar` which contains the file `readme.txt`, the full name, or _path_, to the file is `foo/bar/readme.txt`. Note that this is different from Windows(R) which uses `\` to separate file and directory names. FreeBSD does not use drive letters, or other drive names in the path. For example, one would not type `c:\foo\bar\readme.txt` on FreeBSD. [[disks-file-systems]] === File systems Directories and files are stored in a file system. Each file system contains exactly one directory at the very top level, called the _root directory_ for that file system. This root directory can contain other directories. One file system is designated the _root file system_ or `/`. Every other file system is _mounted_ under the root file system. No matter how many disks are on the FreeBSD system, every directory appears to be part of the same disk. Consider three file systems, called `A`, `B`, and `C`. Each file system has one root directory, which contains two other directories, called `A1`, `A2` (and likewise `B1`, `B2` and `C1`, `C2`). Call `A` the root file system. If man:ls[1] is used to view the contents of this directory, it will show two subdirectories, `A1` and `A2`. The directory tree looks like this: image::example-dir1.png[Directory tree with the root directory and two subdirectories, A1 and A2] A file system must be mounted on to a directory in another file system. When mounting file system `B` on to the directory `A1`, the root directory of `B` replaces `A1`, and the directories in `B` appear accordingly: image::example-dir2.png[Directory tree with the root directory and two subdirectories, A1 and A2. And more subdirectories, B1 and B2 hanging from A1] Any files that are in the `B1` or `B2` directories can be reached with the path `/A1/B1` or `/A1/B2` as necessary. Any files that were in `/A1` have been temporarily hidden. They will reappear if `B` is _unmounted_ from `A`. If `B` had been mounted on `A2` then the diagram would look like this: image::example-dir3.png[Directory tree with the root directory and two subdirectories, A1 and A2. And more subdirectories, B1 and B2 hanging from A2] and the paths would be `/A2/B1` and `/A2/B2` respectively. File systems can be mounted on top of one another. Continuing the last example, the `C` file system could be mounted on top of the `B1` directory in the `B` file system, leading to this arrangement: image::example-dir4.png[A complex directory tree. With different subdirectories hanging from root.] Or `C` could be mounted directly on to the `A` file system, under the `A1` directory: image::example-dir5.png[A complex directory tree. With different subdirectories hanging from root.] It is entirely possible to have one large root file system, and not need to create any others. There are some drawbacks to this approach, and one advantage. .Benefits of Multiple File Systems * Different file systems can have different _mount options_. For example, the root file system can be mounted read-only, making it impossible for users to inadvertently delete or edit a critical file. Separating user-writable file systems, such as `/home`, from other file systems allows them to be mounted _nosuid_. This option prevents the _suid_/_guid_ bits on executables stored on the file system from taking effect, possibly improving security. * FreeBSD automatically optimizes the layout of files on a file system, depending on how the file system is being used. So a file system that contains many small files that are written frequently will have a different optimization to one that contains fewer, larger files. By having one big file system this optimization breaks down. * FreeBSD's file systems are robust if power is lost. However, a power loss at a critical point could still damage the structure of the file system. By splitting data over multiple file systems it is more likely that the system will still come up, making it easier to restore from backup as necessary. .Benefit of a Single File System * File systems are a fixed size. When creating a file system during the FreeBSD installation and giving it a specific size, it may be that one wants to make the partition bigger. This is not easily accomplished without backing up, recreating the file system with the new size, and then restoring the backed up data. + [IMPORTANT] ==== FreeBSD features the man:growfs[8] command, which makes it possible to increase the size of file system on the fly, removing this limitation. A file system can only be expanded into free space in the partition in which it resides. If there is space after the partition, the partition can be expanded with man:gpart[8]. If the partition is the last one on a virtual disk, and the disk is expanded, the partition can then be expanded. ==== [[disks-partitions]] === Disk partitions File systems are contained in _partitions_. Disks are divided into partitions using one of several partitioning schemes; see crossref:bsdinstall[bsdinstall-part-manual, Manual Partitioning]. The newer scheme is GPT; older BIOS-based computers use MBR. GPT supports division of a disk into partitions with a size, offset, and type. It supports a large number of partitions and partition types, and is recommended whenever its use is possible. GPT partitions use the disk name with a suffix, where the suffix is `p1` for the first partition, `p2` for the second, and so on. MBR, however, supports only a small number of partitions. The MBR partitions are known in FreeBSD as `slices`. Slices may be used for different operating systems. FreeBSD slices are subdivided into partitions using BSD labels (see man:bsdlabel[8]). Slice numbers follow the device name, prefixed with an `s`, starting at 1. So "da0__s1__" is the first slice on the first SCSI drive. There can only be four physical slices on a disk, but there can be logical slices inside physical slices of the appropriate type. These extended slices are numbered starting at 5, so "ada0__s5__" is the first extended slice on the first SATA disk. These devices are used by file systems that expect to occupy a slice. Each GPT or BSD partition can contain only one file system, which means that file systems are often described by either their typical mount point in the file system hierarchy, or the name of the partition they are contained in. FreeBSD also uses disk space for _swap space_ to provide _virtual memory_. This allows the computer to behave as though it has much more memory than it actually does. When FreeBSD runs out of memory, it moves some of the data that is not currently being used to the swap space, and moves it back in (moving something else out) when it needs it. This is called _paging_. Some BSD partitions have certain conventions associated with them. [cols="25h,~"] |=== | Partition | Convention |`a` |Normally contains the root file system. |`b` |Normally contains swap space. |`c` |Normally the same size as the enclosing slice. This allows utilities that need to work on the entire slice, such as a bad block scanner, to work on the `c` partition. A file system would not normally be created on this partition. |`d` |Partition `d` used to have a special meaning associated with it, although that is now gone and `d` may work as any normal partition. |=== Slices and "dangerously dedicated" physical drives contain BSD partitions, which are represented as letters from `a` to `h`. This letter is appended to the device name, so "da0__a__" is the `a` partition on the first `da` drive, which is "dangerously dedicated". "ada1s3__e__" is the fifth partition in the third slice of the second SATA disk drive. Finally, each disk on the system is identified. A disk name starts with a code that indicates the type of disk, and then a number, indicating which disk it is. Unlike partitions and slices, disk numbering starts at 0. Common codes are listed in crossref:basics[disks-naming,Disk Device Names]. When referring to a partition in a slice, include the disk name, `s`, the slice number, and then the partition letter. -Examples are shown in crossref:basics[basics-disk-slice-part,Sample Disk, Slice, and Partition Names]. +Examples are shown in crossref:basics[basics-disk-slice-part,"Sample Disk, Slice, and Partition Names"]. GPT partitions include the disk name, `p`, and then the partition number. crossref:basics[basics-concept-disk-model,Conceptual Model of a Disk] shows a conceptual model of a disk layout using MBR slices. When installing FreeBSD, configure the disk slices if using MBR, and create partitions within the slice to be used for FreeBSD. If using GPT, configure partitions for each file system. In either case, create a file system or swap space in each partition, and decide where each file system will be mounted. See man:gpart[8] for information on manipulating partitions. [[disks-naming]] .Disk Device Names [cols="1,1", frame="none", options="header"] |=== | Drive Type | Drive Device Name |SATA and IDE hard drives |`ada` |SCSI hard drives and USB storage devices |`da` |NVMe storage |`nvd` or `nda` |SATA and IDE CD-ROM drives |`cd` |SCSI CD-ROM drives |`cd` |Floppy drives |`fd` |SCSI tape drives |`sa` |RAID drives |Examples include `aacd` for Adaptec(R) AdvancedRAID, `mlxd` and `mlyd` for Mylex(R), `amrd` for AMI MegaRAID(R), `idad` for Compaq Smart RAID, `twed` for 3ware(R) RAID. |=== [example] ==== [[basics-disk-slice-part]] .Sample Disk, Slice, and Partition Names [.informaltable] [cols="1,1", frame="none", options="header"] |=== | Name | Meaning |`ada0s1a` |The first partition (`a`) on the first slice (`s1`) on the first SATA disk (`ada0`). |`da1s2e` |The fifth partition (`e`) on the second slice (`s2`) on the second SCSI disk (`da1`). |=== ==== [[basics-concept-disk-model]] .Conceptual Model of a Disk [example] ==== This diagram shows FreeBSD's view of the first SATA disk attached to the system. Assume that the disk is 250 GB in size, and contains an 80 GB slice and a 170 GB slice (MS-DOS(R) partitions). The first slice contains a Windows(R) NTFS file system, `C:`, and the second slice contains a FreeBSD installation. This example FreeBSD installation has four data partitions and a swap partition. The four partitions each hold a file system. Partition `a` is used for the root file system, `d` for `/var/`, `e` for `/tmp/`, and `f` for `/usr/`. Partition letter `c` refers to the entire slice, and so is not used for ordinary partitions. image::disk-layout.png[Layout of a shared drive between Windows and FreeBSD] ==== [[mount-unmount]] == Mounting and Unmounting File Systems The file system is best visualized as a tree, rooted, as it were, at `/`. `/dev`, `/usr`, and the other directories in the root directory are branches, which may have their own branches, such as `/usr/local`, and so on. There are various reasons to house some of these directories on separate file systems. `/var` contains the directories `log/`, `spool/`, and various types of temporary files, and as such, may get filled up. Filling up the root file system is not a good idea, so splitting `/var` from `/` is often favorable. Another common reason to contain certain directory trees on other file systems is if they are to be housed on separate physical disks, or are separate virtual disks, such as Network File System mounts, described in crossref:network-servers[network-nfs,“Network File System (NFS)”], or CDROM drives. [[disks-fstab]] === The fstab File During the boot process (crossref:boot[boot,The FreeBSD Booting Process]), file systems listed in `/etc/fstab` are automatically mounted except for the entries containing `noauto`. This file contains entries in the following format: [.programlisting] .... device /mount-point fstype options dumpfreq passno .... `device`:: An existing device name as explained in crossref:basics[disks-naming,Disk Device Names]. `mount-point`:: An existing directory on which to mount the file system. `fstype`:: The file system type to pass to man:mount[8]. The default FreeBSD file system is `ufs`. `options`:: Either `rw` for read-write file systems, or `ro` for read-only file systems, followed by any other options that may be needed. A common option is `noauto` for file systems not normally mounted during the boot sequence. Other options are listed in man:mount[8]. `dumpfreq`:: Used by man:dump[8] to determine which file systems require dumping. If the field is missing, a value of zero is assumed. `passno`:: Determines the order in which UFS file systems should be checked by man:fsck[8] after a reboot. File systems that should be skipped should have their `passno` set to zero. The root file system needs to be checked before everything else and should have its `passno` set to one. The other file systems should be set to values greater than one. If more than one file system has the same `passno`, man:fsck[8] will attempt to check file systems in parallel if possible. Refer to man:fstab[5] for more information on the format of `/etc/fstab` and its options. [[disks-mount]] === Using man:mount[8] File systems are mounted using man:mount[8]. The most basic syntax is as follows: [example] ==== [source,shell] .... # mount device mountpoint .... ==== A file system listed in `/etc/fstab` can also be mounted by providing just the mountpoint. This command provides many options which are described in man:mount[8]. The most commonly used options include: .Mount Options `-a`:: Mount all the file systems listed in `/etc/fstab`, except those marked as "noauto", excluded by the `-t` flag, or those that are already mounted. `-d`:: Do everything except for the actual mount system call. This option is useful in conjunction with the `-v` flag to determine what man:mount[8] is actually trying to do. `-f`:: Force the mount of an unclean file system (dangerous), or the revocation of write access when downgrading a file system's mount status from read-write to read-only. `-r`:: Mount the file system read-only. This is identical to using `-o ro`. ``-t _fstype_``:: Mount the specified file system type or mount only file systems of the given type, if `-a` is included. "ufs" is the default file system type. `-u`:: Update mount options on the file system. `-v`:: Be verbose. `-w`:: Mount the file system read-write. The following options can be passed to `-o` as a comma-separated list: nosuid:: Do not interpret setuid or setgid flags on the file system. This is also a useful security option. [[disks-umount]] === Using man:umount[8] To unmount a file system use man:umount[8]. This command takes one parameter which can be a mountpoint, device name, `-a` or `-A`. All forms take `-f` to force unmounting, and `-v` for verbosity. Be warned that `-f` is not generally a good idea as it might crash the computer or damage data on the file system. To unmount all mounted file systems, or just the file system types listed after `-t`, use `-a` or `-A`. Note that `-A` does not attempt to unmount the root file system. [[basics-processes]] == Processes and Daemons FreeBSD is a multi-tasking operating system. Each program running at any one time is called a _process_. Every running command starts at least one new process and there are a number of system processes that are run by FreeBSD. Each process is uniquely identified by a number called a _process ID_ (PID). Similar to files, each process has one owner and group, and the owner and group permissions are used to determine which files and devices the process can open. Most processes also have a parent process that started them. For example, the shell is a process, and any command started in the shell is a process which has the shell as its parent process. The exception is a special process called man:init[8] which is always the first process to start at boot time and which always has a PID of `1`. Some programs are not designed to be run with continuous user input and disconnect from the terminal at the first opportunity. For example, a web server responds to web requests, rather than user input. Mail servers are another example of this type of application. These types of programs are known as _daemons_. The term daemon comes from Greek mythology and represents an entity that is neither good nor evil, and which invisibly performs useful tasks. This is why the BSD mascot is the cheerful-looking daemon with sneakers and a pitchfork. There is a convention to name programs that normally run as daemons with a trailing "d". For example, BIND is the Berkeley Internet Name Domain, but the actual program that executes is `named`. The Apache web server program is `httpd` and the line printer spooling daemon is `lpd`. This is only a naming convention. For example, the main mail daemon for the Sendmail application is `sendmail`, and not `maild`. === Viewing Processes To see the processes running on the system, use man:ps[1] or man:top[1]. To display a static list of the currently running processes, their PIDs, how much memory they are using, and the command they were started with, use man:ps[1]. To display all the running processes and update the display every few seconds in order to interactively see what the computer is doing, use man:top[1]. By default, man:ps[1] only shows the commands that are running and owned by the user. For example: [source,shell] .... % ps .... The output should be similar to the following: [.programlisting] .... PID TT STAT TIME COMMAND 8203 0 Ss 0:00.59 /bin/csh 8895 0 R+ 0:00.00 ps .... The output from man:ps[1] is organized into a number of columns. The `PID` column displays the process ID. PIDs are assigned starting at 1, go up to 99999, then wrap around back to the beginning. However, a PID is not reassigned if it is already in use. The `TT` column shows the tty the program is running on and `STAT` shows the program's state. `TIME` is the amount of time the program has been running on the CPU. This is usually not the elapsed time since the program was started, as most programs spend a lot of time waiting for things to happen before they need to spend time on the CPU. Finally, `COMMAND` is the command that was used to start the program. A number of different options are available to change the information that is displayed. One of the most useful sets is `auxww`, where `a` displays information about all the running processes of all users, `u` displays the username and memory usage of the process' owner, `x` displays information about daemon processes, and `ww` causes man:ps[1] to display the full command line for each process, rather than truncating it once it gets too long to fit on the screen. The output from man:top[1] is similar: [source,shell] .... % top .... The output should be similar to the following: [.programlisting] .... last pid: 9609; load averages: 0.56, 0.45, 0.36 up 0+00:20:03 10:21:46 107 processes: 2 running, 104 sleeping, 1 zombie CPU: 6.2% user, 0.1% nice, 8.2% system, 0.4% interrupt, 85.1% idle Mem: 541M Active, 450M Inact, 1333M Wired, 4064K Cache, 1498M Free ARC: 992M Total, 377M MFU, 589M MRU, 250K Anon, 5280K Header, 21M Other Swap: 2048M Total, 2048M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 557 root 1 -21 r31 136M 42296K select 0 2:20 9.96% Xorg 8198 dru 2 52 0 449M 82736K select 3 0:08 5.96% kdeinit4 8311 dru 27 30 0 1150M 187M uwait 1 1:37 0.98% firefox 431 root 1 20 0 14268K 1728K select 0 0:06 0.98% moused 9551 dru 1 21 0 16600K 2660K CPU3 3 0:01 0.98% top 2357 dru 4 37 0 718M 141M select 0 0:21 0.00% kdeinit4 8705 dru 4 35 0 480M 98M select 2 0:20 0.00% kdeinit4 8076 dru 6 20 0 552M 113M uwait 0 0:12 0.00% soffice.bin 2623 root 1 30 10 12088K 1636K select 3 0:09 0.00% powerd 2338 dru 1 20 0 440M 84532K select 1 0:06 0.00% kwin 1427 dru 5 22 0 605M 86412K select 1 0:05 0.00% kdeinit4 .... The output is split into two sections. The header (the first five or six lines) shows the PID of the last process to run, the system load averages (which are a measure of how busy the system is), the system uptime (time since the last reboot) and the current time. The other figures in the header relate to how many processes are running, how much memory and swap space has been used, and how much time the system is spending in different CPU states. If the ZFS file system module has been loaded, an `ARC` line indicates how much data was read from the memory cache instead of from disk. Below the header is a series of columns containing similar information to the output from man:ps[1], such as the PID, username, amount of CPU time, and the command that started the process. By default, man:top[1] also displays the amount of memory space taken by the process. This is split into two columns: one for total size and one for resident size. Total size is how much memory the application has needed and the resident size is how much it is actually using now. man:top[1] automatically updates the display every two seconds. A different interval can be specified with `-s`. [[basics-daemons]] === Killing Processes One way to communicate with any running process or daemon is to send a _signal_ using man:kill[1]. There are a number of different signals; some have a specific meaning while others are described in the application's documentation. A user can only send a signal to a process they own and sending a signal to someone else's process will result in a permission denied error. The exception is the `root` user, who can send signals to anyone's processes. The operating system can also send a signal to a process. If an application is badly written and tries to access memory that it is not supposed to, FreeBSD will send the process the "Segmentation Violation" signal (`SIGSEGV`). If an application has been written to use the man:alarm[3] system call to be alerted after a period of time has elapsed, it will be sent the "Alarm" signal (`SIGALRM`). Two signals can be used to stop a process: `SIGTERM` and `SIGKILL`. `SIGTERM` is the polite way to kill a process as the process can read the signal, close any log files it may have open, and attempt to finish what it is doing before shutting down. In some cases, a process may ignore `SIGTERM` if it is in the middle of some task that cannot be interrupted. `SIGKILL` cannot be ignored by a process. Sending a `SIGKILL` to a process will usually stop that process there and then. footnote:[There are a few tasks that cannot be interrupted. For example, if the process is trying to read from a file that is on another computer on the network, and the other computer is unavailable, the process is said to be uninterruptible. Eventually the process will time out, typically after two minutes. As soon as this time out occurs the process will be killed.]. Other commonly used signals are `SIGHUP`, `SIGUSR1`, and `SIGUSR2`. Since these are general purpose signals, different applications will respond differently. For example, after changing a web server's configuration file, the web server needs to be told to re-read its configuration. Restarting `httpd` would result in a brief outage period on the web server. Instead, send the daemon the `SIGHUP` signal. Be aware that different daemons will have different behavior, so refer to the documentation for the daemon to determine if `SIGHUP` will achieve the desired results. [IMPORTANT] ==== Killing a random process on the system is a bad idea. In particular, man:init[8], PID 1, is special. Running `/bin/kill -s KILL 1` is a quick, and unrecommended, way to shutdown the system. _Always_ double check the arguments to man:kill[1] _before_ pressing kbd:[Return]. ==== [[shells]] == Shells A _shell_ provides a command line interface for interacting with the operating system. A shell receives commands from the input channel and executes them. Many shells provide built in functions to help with everyday tasks such as file management, file globbing, command line editing, command macros, and environment variables. FreeBSD comes with several shells, including an extended POSIX(R) shell (man:sh[1]) and the extended C shell (man:tcsh[1]). Other shells are available from the FreeBSD Ports Collection, such as `zsh` and `bash`. The shell that is used is really a matter of taste. A C programmer might feel more comfortable with a C-like shell such as man:tcsh[1]. A Linux(R) user might prefer `bash`. Each shell has unique properties that may or may not work with a user's preferred working environment, which is why there is a choice of which shell to use. One common shell feature is filename completion. After a user types the first few letters of a command or filename and presses kbd:[Tab], the shell completes the rest of the command or filename. Consider two files called `foobar` and `football`. To delete `foobar`, the user might type `rm foo` and press kbd:[Tab] to complete the filename. But the shell only shows `rm foo`. It was unable to complete the filename because both `foobar` and `football` start with `foo`. Some shells sound a beep or show all the choices if more than one name matches. The user must then type more characters to identify the desired filename. Typing a `t` and pressing kbd:[Tab] again is enough to let the shell determine which filename is desired and fill in the rest. Another feature of the shell is the use of environment variables. Environment variables are a variable/key pair stored in the shell's environment. This environment can be read by any program invoked by the shell, and thus contains a lot of program configuration. crossref:basics[shell-env-vars,Common Environment Variables] provides a list of common environment variables and their meanings. Note that the names of environment variables are always in uppercase. [[shell-env-vars]] .Common Environment Variables [cols="25h,~"] |=== | Variable | Description |`USER` |Current logged in user's name. |`PATH` |Colon-separated list of directories to search for binaries. |`DISPLAY` |Network name of the Xorg display to connect to, if available. |`SHELL` |The current shell. |`TERM` |The name of the user's type of terminal. Used to determine the capabilities of the terminal. |`TERMCAP` |Database entry of the terminal escape codes to perform various terminal functions. |`OSTYPE` |Type of operating system. |`MACHTYPE` |The system's CPU architecture. |`EDITOR` |The user's preferred text editor. |`PAGER` |The user's preferred utility for viewing text one page at a time. |`MANPATH` |Colon-separated list of directories to search for manual pages. |=== How to set an environment variable differs between shells. In man:tcsh[1] and man:csh[1], use `setenv` to set environment variables. In man:sh[1] and `bash`, use `export` to set the current environment variables. This example sets the default `EDITOR` to `/usr/local/bin/emacs` for the man:tcsh[1] shell: [source,shell] .... % setenv EDITOR /usr/local/bin/emacs .... The equivalent command for `bash` would be: [source,shell] .... % export EDITOR="/usr/local/bin/emacs" .... To expand an environment variable in order to see its current setting, type a `$` character in front of its name on the command line. For example, `echo $TERM` displays the current `$TERM` setting. Shells treat special characters, known as meta-characters, as special representations of data. The most common meta-character is `\*`, which represents any number of characters in a filename. Meta-characters can be used to perform filename globbing. For example, `echo *` is equivalent to `ls` because the shell takes all the files that match `*` and `echo` lists them on the command line. To prevent the shell from interpreting a special character, escape it from the shell by starting it with a backslash (`\`). For example, `echo $TERM` prints the terminal setting whereas `echo \$TERM` literally prints the string `$TERM`. [[changing-shells]] === Changing the Shell The easiest way to permanently change the default shell is to use `chsh`. Running this command will open the editor that is configured in the `EDITOR` environment variable, which by default is set to man:vi[1]. Change the `Shell:` line to the full path of the new shell. Alternately, use `chsh -s` which will set the specified shell without opening an editor. For example, to change the shell to `bash`: [source,shell] .... % chsh -s /usr/local/bin/bash .... Enter the password at the prompt and press kbd:[Return] to change the shell. Log off and log in again to start using the new shell. [NOTE] ==== The new shell _must_ be present in `/etc/shells`. If the shell was installed from the FreeBSD Ports Collection as described in crossref:ports[ports,Installing Applications: Packages and Ports], it should be automatically added to this file. If it is missing, add it using this command, replacing the path with the path of the shell: [source,shell] .... # echo /usr/local/bin/bash >> /etc/shells .... Then, rerun man:chsh[1]. ==== === Advanced Shell Techniques The UNIX(R) shell is not just a command interpreter, it acts as a powerful tool which allows users to execute commands, redirect their output, redirect their input and chain commands together to improve the final command output. When this functionality is mixed with built in commands, the user is provided with an environment that can maximize efficiency. Shell redirection is the action of sending the output or the input of a command into another command or into a file. To capture the output of the man:ls[1] command, for example, into a file, redirect the output: [source,shell] .... % ls > directory_listing.txt .... The directory contents will now be listed in `directory_listing.txt`. Some commands can be used to read input, such as man:sort[1]. To sort this listing, redirect the input: [source,shell] .... % sort < directory_listing.txt .... The input will be sorted and placed on the screen. To redirect that input into another file, one could redirect the output of man:sort[1] by mixing the direction: [source,shell] .... % sort < directory_listing.txt > sorted.txt .... In all of the previous examples, the commands are performing redirection using file descriptors. Every UNIX(R) system has file descriptors, which include standard input (stdin), standard output (stdout), and standard error (stderr). Each one has a purpose, where input could be a keyboard or a mouse, something that provides input. Output could be a screen or paper in a printer. And error would be anything that is used for diagnostic or error messages. All three are considered I/O based file descriptors and sometimes considered streams. Through the use of these descriptors, the shell allows output and input to be passed around through various commands and redirected to or from a file. Another method of redirection is the pipe operator. The UNIX(R) pipe operator, "|" allows the output of one command to be directly passed or directed to another program. Basically, a pipe allows the standard output of a command to be passed as standard input to another command, for example: [source,shell] .... % cat directory_listing.txt | sort | less .... In that example, the contents of `directory_listing.txt` will be sorted and the output passed to man:less[1]. This allows the user to scroll through the output at their own pace and prevent it from scrolling off the screen. [[editors]] == Text Editors Most FreeBSD configuration is done by editing text files, so it is a good idea to become familiar with a text editor. FreeBSD comes with a few as part of the base system, and many more are available in the Ports Collection. A simple editor to learn is man:ee[1], which stands for easy editor. To start this editor, type `ee _filename_` where _filename_ is the name of the file to be edited. Once inside the editor, all of the commands for manipulating the editor's functions are listed at the top of the display. The caret (`^`) represents kbd:[Ctrl], so `^e` expands to kbd:[Ctrl+e]. To leave man:ee[1], press kbd:[Esc], then choose the "leave editor" option from the main menu. The editor will prompt to save any changes if the file has been modified. FreeBSD also comes with more powerful text editors, such as man:vi[1], as part of the base system. Other editors, like package:editors/emacs[] and package:editors/vim[], are part of the FreeBSD Ports Collection. These editors offer more functionality at the expense of being more complicated to learn. Learning a more powerful editor such as vim or Emacs can save more time in the long run. Many applications which modify files or require typed input will automatically open a text editor. To change the default editor, set the `EDITOR` environment variable as described in crossref:basics[shells, Shells]. [[basics-devices]] == Devices and Device Nodes A device is a term used mostly for hardware-related activities in a system, including disks, printers, graphics cards, and keyboards. When FreeBSD boots, the majority of the boot messages refer to devices being detected. A copy of the boot messages is saved to `/var/run/dmesg.boot`. Each device has a device name and number. For example, `ada0` is the first SATA hard drive, while `kbd0` represents the keyboard. Most devices in FreeBSD must be accessed through special files called device nodes, which are located in `/dev`. [[basics-more-information]] == Manual Pages The most comprehensive documentation on FreeBSD is in the form of manual pages. Nearly every program on the system comes with a short reference manual explaining the basic operation and available arguments. These manuals can be viewed using `man`: [source,shell] .... % man command .... where _command_ is the name of the command to learn about. For example, to learn more about man:ls[1], type: [source,shell] .... % man ls .... Manual pages are divided into sections which represent the type of topic. In FreeBSD, the following sections are available: . User commands. . System calls and error numbers. . Functions in the C libraries. . Device drivers. . File formats. . Games and other diversions. . Miscellaneous information. . System maintenance and operation commands. . System kernel interfaces. In some cases, the same topic may appear in more than one section of the online manual. For example, there is a `chmod` user command and a `chmod()` system call. To tell man:man[1] which section to display, specify the section number: [source,shell] .... % man 1 chmod .... This will display the manual page for the user command man:chmod[1]. References to a particular section of the online manual are traditionally placed in parenthesis in written documentation, so man:chmod[1] refers to the user command and man:chmod[2] refers to the system call. If the name of the manual page is unknown, use `man -k` to search for keywords in the manual page descriptions: [source,shell] .... % man -k mail .... This command displays a list of commands that have the keyword "mail" in their descriptions. This is equivalent to using man:apropos[1]. To read the descriptions for all of the commands in `/usr/sbin`, type: [source,shell] .... % cd /usr/sbin % man -f * | more .... or [source,shell] .... % cd /usr/sbin % whatis * |more .... [[basics-info]] === GNU Info Files FreeBSD includes several applications and utilities produced by the Free Software Foundation (FSF). In addition to manual pages, these programs may include hypertext documents called `info` files. These can be viewed using man:info[1] or, if package:editors/emacs[] is installed, the info mode of emacs. To use man:info[1], type: [source,shell] .... % info .... For a brief introduction, type `h`. For a quick command reference, type `?`. diff --git a/documentation/content/en/books/handbook/serialcomms/_index.adoc b/documentation/content/en/books/handbook/serialcomms/_index.adoc index af9c67f81d..b31793384c 100644 --- a/documentation/content/en/books/handbook/serialcomms/_index.adoc +++ b/documentation/content/en/books/handbook/serialcomms/_index.adoc @@ -1,1219 +1,1219 @@ --- title: Chapter 30. Serial Communications part: Part IV. Network Communication prev: books/handbook/partiv next: books/handbook/ppp-and-slip description: This chapter covers some of the ways serial communications can be used on FreeBSD tags: ["serial", "communications", "terminal", "modem", "console"] showBookMenu: true weight: 34 params: path: "/books/handbook/serialcomms/" --- [[serialcomms]] = Serial Communications :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumoffset: 30 :partnums: :source-highlighter: rouge :experimental: :images-path: books/handbook/serialcomms/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[serial-synopsis]] == Synopsis UNIX(R) has always had support for serial communications as the very first UNIX(R) machines relied on serial lines for user input and output. Things have changed a lot from the days when the average terminal consisted of a 10-character-per-second serial printer and a keyboard. This chapter covers some of the ways serial communications can be used on FreeBSD. Read this chapter to learn: * How to connect terminals to a FreeBSD system. * How to use a modem to dial out to remote hosts. * How to allow remote users to login to a FreeBSD system with a modem. * How to boot a FreeBSD system from a serial console. Before reading this chapter: * Know how to crossref:kernelconfig[kernelconfig, configure and install a custom kernel]. * Understand crossref:basics[basics, FreeBSD permissions and processes]. * Have access to the technical manual for the serial hardware to be used with FreeBSD. [[serial]] == Serial Terminology and Hardware The following terms are often used in serial communications: bps:: Bits per Second (bps) is the rate at which data is transmitted. DTE:: Data Terminal Equipment (DTE) is one of two endpoints in a serial communication. An example would be a computer. DCE:: Data Communications Equipment (DCE) is the other endpoint in a serial communication. Typically, it is a modem or serial terminal. RS-232:: The original standard which defined hardware serial communications. It has since been renamed to TIA-232. When referring to communication data rates, this section does not use the term _baud_. Baud refers to the number of electrical state transitions made in a period of time, while bps is the correct term to use. To connect a serial terminal to a FreeBSD system, a serial port on the computer and the proper cable to connect to the serial device are needed. Users who are already familiar with serial hardware and cabling can safely skip this section. [[term-cables-null]] === Serial Cables and Ports There are several different kinds of serial cables. The two most common types are null-modem cables and standard RS-232 cables. The documentation for the hardware should describe the type of cable required. These two types of cables differ in how the wires are connected to the connector. Each wire represents a signal, with the defined signals summarized in crossref:serialcomms[serialcomms-signal-names,RS-232C Signal Names]. A standard serial cable passes all of the RS-232C signals straight through. For example, the "Transmitted Data" pin on one end of the cable goes to the "Transmitted Data" pin on the other end. This is the type of cable used to connect a modem to the FreeBSD system, and is also appropriate for some terminals. A null-modem cable switches the "Transmitted Data" pin of the connector on one end with the "Received Data" pin on the other end. The connector can be either a DB-25 or a DB-9. A null-modem cable can be constructed using the pin connections summarized in crossref:serialcomms[nullmodem-db25, DB-25 to DB-25 Null-Modem Cable], crossref:serialcomms[nullmodem-db9,DB-9 to DB-9 Null-Modem Cable], and -crossref:serialcomms[nullmodem-db9-25,DB-9 to DB-25 Null-Modem Cable, DB-9 to DB-25 Null-Modem Cable]. +crossref:serialcomms[nullmodem-db9-25,"DB-9 to DB-25 Null-Modem Cable, DB-9 to DB-25 Null-Modem Cable"]. While the standard calls for a straight-through pin 1 to pin 1 "Protective Ground" line, it is often omitted. Some terminals work using only pins 2, 3, and 7, while others require different configurations. When in doubt, refer to the documentation for the hardware. [[serialcomms-signal-names]] .RS-232C Signal Names [cols="1,1", frame="none", options="header"] |=== <| Acronyms <| Names |RD |Received Data |TD |Transmitted Data |DTR |Data Terminal Ready |DSR |Data Set Ready |DCD |Data Carrier Detect |SG |Signal Ground |RTS |Request to Send |CTS |Clear to Send |=== [[nullmodem-db25]] .DB-25 to DB-25 Null-Modem Cable [cols="1,1,1,1,1", frame="none", options="header"] |=== <| Signal <| Pin # | <| Pin # <| Signal |SG |7 |connects to |7 |SG |TD |2 |connects to |3 |RD |RD |3 |connects to |2 |TD |RTS |4 |connects to |5 |CTS |CTS |5 |connects to |4 |RTS |DTR |20 |connects to |6 |DSR |DTR |20 |connects to |8 |DCD |DSR |6 |connects to |20 |DTR |DCD |8 |connects to |20 |DTR |=== [[nullmodem-db9]] .DB-9 to DB-9 Null-Modem Cable [cols="1,1,1,1,1", frame="none", options="header"] |=== <| Signal <| Pin # | <| Pin # <| Signal |RD |2 |connects to |3 |TD |TD |3 |connects to |2 |RD |DTR |4 |connects to |6 |DSR |DTR |4 |connects to |1 |DCD |SG |5 |connects to |5 |SG |DSR |6 |connects to |4 |DTR |DCD |1 |connects to |4 |DTR |RTS |7 |connects to |8 |CTS |CTS |8 |connects to |7 |RTS |=== [[nullmodem-db9-25]] .DB-9 to DB-25 Null-Modem Cable [cols="1,1,1,1,1", frame="none", options="header"] |=== <| Signal <| Pin # | <| Pin # <| Signal |RD |2 |connects to |2 |TD |TD |3 |connects to |3 |RD |DTR |4 |connects to |6 |DSR |DTR |4 |connects to |8 |DCD |SG |5 |connects to |7 |SG |DSR |6 |connects to |20 |DTR |DCD |1 |connects to |20 |DTR |RTS |7 |connects to |5 |CTS |CTS |8 |connects to |4 |RTS |=== [NOTE] ==== When one pin at one end connects to a pair of pins at the other end, it is usually implemented with one short wire between the pair of pins in their connector and a long wire to the other single pin. ==== Serial ports are the devices through which data is transferred between the FreeBSD host computer and the terminal. Several kinds of serial ports exist. Before purchasing or constructing a cable, make sure it will fit the ports on the terminal and on the FreeBSD system. Most terminals have DB-25 ports. Personal computers may have DB-25 or DB-9 ports. A multiport serial card may have RJ-12 or RJ-45/ ports. See the documentation that accompanied the hardware for specifications on the kind of port or visually verify the type of port. In FreeBSD, each serial port is accessed through an entry in [.filename]#/dev#. There are two different kinds of entries: * Call-in ports are named [.filename]#/dev/ttyuN# where _N_ is the port number, starting from zero. If a terminal is connected to the first serial port ([.filename]#COM1#), use [.filename]#/dev/ttyu0# to refer to the terminal. If the terminal is on the second serial port ([.filename]#COM2#), use [.filename]#/dev/ttyu1#, and so forth. Generally, the call-in port is used for terminals. Call-in ports require that the serial line assert the "Data Carrier Detect" signal to work correctly. * Call-out ports are named [.filename]#/dev/cuauN# on FreeBSD versions 8.X and higher and [.filename]#/dev/cuadN# on FreeBSD versions 7.X and lower. Call-out ports are usually not used for terminals, but are used for modems. The call-out port can be used if the serial cable or the terminal does not support the "Data Carrier Detect" signal. FreeBSD also provides initialization devices ([.filename]#/dev/ttyuN.init# and [.filename]#/dev/cuauN.init# or [.filename]#/dev/cuadN.init#) and locking devices ([.filename]#/dev/ttyuN.lock# and [.filename]#/dev/cuauN.lock# or [.filename]#/dev/cuadN.lock#). The initialization devices are used to initialize communications port parameters each time a port is opened, such as `crtscts` for modems which use `RTS/CTS` signaling for flow control. The locking devices are used to lock flags on ports to prevent users or programs changing certain parameters. Refer to man:termios[4], man:uart[4], and man:stty[1] for information on terminal settings, locking and initializing devices, and setting terminal options, respectively. [[serial-hw-config]] === Serial Port Configuration By default, FreeBSD supports four serial ports which are commonly known as [.filename]#COM1#, [.filename]#COM2#, [.filename]#COM3#, and [.filename]#COM4#. FreeBSD also supports dumb multi-port serial interface cards, such as the BocaBoard 1008 and 2016, as well as more intelligent multi-port cards such as those made by Digiboard. However, the default kernel only looks for the standard [.filename]#COM# ports. To see if the system recognizes the serial ports, look for system boot messages that start with `uart`: [source,shell] .... # grep uart /var/run/dmesg.boot .... If the system does not recognize all of the needed serial ports, additional entries can be added to [.filename]#/boot/device.hints#. This file already contains `hint.uart.0.\*` entries for [.filename]#COM1# and `hint.uart.1.*` entries for [.filename]#COM2#. When adding a port entry for [.filename]#COM3# use `0x3E8`, and for [.filename]#COM4# use `0x2E8`. Common IRQ addresses are `5` for [.filename]#COM3# and `9` for [.filename]#COM4#. To determine the default set of terminal I/O settings used by the port, specify its device name. This example determines the settings for the call-in port on [.filename]#COM2#: [source,shell] .... # stty -a -f /dev/ttyu1 .... System-wide initialization of serial devices is controlled by [.filename]#/etc/rc.d/serial#. This file affects the default settings of serial devices. To change the settings for a device, use `stty`. By default, the changed settings are in effect until the device is closed and when the device is reopened, it goes back to the default set. To permanently change the default set, open and adjust the settings of the initialization device. For example, to turn on `CLOCAL` mode, 8 bit communication, and `XON/XOFF` flow control for [.filename]#ttyu5#, type: [source,shell] .... # stty -f /dev/ttyu5.init clocal cs8 ixon ixoff .... To prevent certain settings from being changed by an application, make adjustments to the locking device. For example, to lock the speed of [.filename]#ttyu5# to 57600 bps, type: [source,shell] .... # stty -f /dev/ttyu5.lock 57600 .... Now, any application that opens [.filename]#ttyu5# and tries to change the speed of the port will be stuck with 57600 bps. [[term]] == Terminals Terminals provide a convenient and low-cost way to access a FreeBSD system when not at the computer's console or on a connected network. This section describes how to use terminals with FreeBSD. The original UNIX(R) systems did not have consoles. Instead, users logged in and ran programs through terminals that were connected to the computer's serial ports. The ability to establish a login session on a serial port still exists in nearly every UNIX(R)-like operating system today, including FreeBSD. By using a terminal attached to an unused serial port, a user can log in and run any text program that can normally be run on the console or in an `xterm` window. Many terminals can be attached to a FreeBSD system. An older spare computer can be used as a terminal wired into a more powerful computer running FreeBSD. This can turn what might otherwise be a single-user computer into a powerful multiple-user system. FreeBSD supports three types of terminals: Dumb terminals:: Dumb terminals are specialized hardware that connect to computers over serial lines. They are called "dumb" because they have only enough computational power to display, send, and receive text. No programs can be run on these devices. Instead, dumb terminals connect to a computer that runs the needed programs. + There are hundreds of kinds of dumb terminals made by many manufacturers, and just about any kind will work with FreeBSD. Some high-end terminals can even display graphics, but only certain software packages can take advantage of these advanced features. + Dumb terminals are popular in work environments where workers do not need access to graphical applications. Computers Acting as Terminals:: Since a dumb terminal has just enough ability to display, send, and receive text, any spare computer can be a dumb terminal. All that is needed is the proper cable and some _terminal emulation_ software to run on the computer. + This configuration can be useful. For example, if one user is busy working at the FreeBSD system's console, another user can do some text-only work at the same time from a less powerful personal computer hooked up as a terminal to the FreeBSD system. + There are at least two utilities in the base-system of FreeBSD that can be used to work through a serial connection: man:cu[1] and man:tip[1]. + For example, to connect from a client system that runs FreeBSD to the serial connection of another system: + [source,shell] .... # cu -l /dev/cuauN .... + Ports are numbered starting from zero. This means that [.filename]#COM1# is [.filename]#/dev/cuau0#. + Additional programs are available through the Ports Collection, such as package:comms/minicom[]. X Terminals:: X terminals are the most sophisticated kind of terminal available. Instead of connecting to a serial port, they usually connect to a network like Ethernet. Instead of being relegated to text-only applications, they can display any Xorg application. + This chapter does not cover the setup, configuration, or use of X terminals. [[term-config]] === Terminal Configuration This section describes how to configure a FreeBSD system to enable a login session on a serial terminal. It assumes that the system recognizes the serial port to which the terminal is connected and that the terminal is connected with the correct cable. In FreeBSD, `init` reads [.filename]#/etc/ttys# and starts a `getty` process on the available terminals. The `getty` process is responsible for reading a login name and starting the `login` program. The ports on the FreeBSD system which allow logins are listed in [.filename]#/etc/ttys#. For example, the first virtual console, [.filename]#ttyv0#, has an entry in this file, allowing logins on the console. This file also contains entries for the other virtual consoles, serial ports, and pseudo-ttys. For a hardwired terminal, the serial port's [.filename]#/dev# entry is listed without the `/dev` part. For example, [.filename]#/dev/ttyv0# is listed as `ttyv0`. The default [.filename]#/etc/ttys# configures support for the first four serial ports, [.filename]#ttyu0# through [.filename]#ttyu3#: [.programlisting] .... ttyu0 "/usr/libexec/getty std.115200" dialup off secure ttyu1 "/usr/libexec/getty std.115200" dialup off secure ttyu2 "/usr/libexec/getty std.115200" dialup off secure ttyu3 "/usr/libexec/getty std.115200" dialup off secure .... When attaching a terminal to one of those ports, modify the default entry to set the required speed and terminal type, to turn the device `on` and, if needed, to change the port's `secure` setting. If the terminal is connected to another port, add an entry for the port. crossref:serialcomms[ex-etc-ttys,Configuring Terminal Entries] configures two terminals in [.filename]#/etc/ttys#. The first entry configures a Wyse-50 connected to [.filename]#COM2#. The second entry configures an old computer running Procomm terminal software emulating a VT-100 terminal. The computer is connected to the sixth serial port on a multi-port serial card. [example] [[ex-etc-ttys]] .Configuring Terminal Entries ==== [.programlisting] .... ttyu1 "/usr/libexec/getty std.38400" wy50 on insecure ttyu5 "/usr/libexec/getty std.19200" vt100 on insecure .... The first field specifies the device name of the serial terminal. The second field tells `getty` to initialize and open the line, set the line speed, prompt for a user name, and then execute the `login` program. The optional _getty type_ configures characteristics on the terminal line, like bps rate and parity. The available getty types are listed in [.filename]#/etc/gettytab#. In almost all cases, the getty types that start with `std` will work for hardwired terminals as these entries ignore parity. There is a `std` entry for each bps rate from 110 to 115200. Refer to man:gettytab[5] for more information. When setting the getty type, make sure to match the communications settings used by the terminal. For this example, the Wyse-50 uses no parity and connects at 38400 bps. The computer uses no parity and connects at 19200 bps. The third field is the type of terminal. For dial-up ports, `unknown` or `dialup` is typically used since users may dial up with practically any type of terminal or software. Since the terminal type does not change for hardwired terminals, a real terminal type from [.filename]#/etc/termcap# can be specified. For this example, the Wyse-50 uses the real terminal type while the computer running Procomm is set to emulate a VT-100. The fourth field specifies if the port should be enabled. To enable logins on this port, this field must be set to `on`. The final field is used to specify whether the port is secure. Marking a port as `secure` means that it is trusted enough to allow `root` to login from that port. Insecure ports do not allow `root` logins. On an insecure port, users must login from unprivileged accounts and then use `su` or a similar mechanism to gain superuser privileges, as described in crossref:basics[users-superuser,“The Superuser Account”]. For security reasons, it is recommended to change this setting to `insecure`. ==== After making any changes to [.filename]#/etc/ttys#, send a SIGHUP (hangup) signal to the `init` process to force it to re-read its configuration file: [source,shell] .... # kill -HUP 1 .... Since `init` is always the first process run on a system, it always has a process ID of `1`. If everything is set up correctly, all cables are in place, and the terminals are powered up, a `getty` process should now be running on each terminal and login prompts should be available on each terminal. [[term-debug]] === Troubleshooting the Connection Even with the most meticulous attention to detail, something could still go wrong while setting up a terminal. Here is a list of common symptoms and some suggested fixes. If no login prompt appears, make sure the terminal is plugged in and powered up. If it is a personal computer acting as a terminal, make sure it is running terminal emulation software on the correct serial port. Make sure the cable is connected firmly to both the terminal and the FreeBSD computer. Make sure it is the right kind of cable. Make sure the terminal and FreeBSD agree on the bps rate and parity settings. For a video display terminal, make sure the contrast and brightness controls are turned up. If it is a printing terminal, make sure paper and ink are in good supply. Use `ps` to make sure that a `getty` process is running and serving the terminal. For example, the following listing shows that a `getty` is running on the second serial port, [.filename]#ttyu1#, and is using the `std.38400` entry in [.filename]#/etc/gettytab#: [source,shell] .... # ps -axww|grep ttyu 22189 d1 Is+ 0:00.03 /usr/libexec/getty std.38400 ttyu1 .... If no `getty` process is running, make sure the port is enabled in [.filename]#/etc/ttys#. Remember to run `kill -HUP 1` after modifying [.filename]#/etc/ttys#. If the `getty` process is running but the terminal still does not display a login prompt, or if it displays a prompt but will not accept typed input, the terminal or cable may not support hardware handshaking. Try changing the entry in [.filename]#/etc/ttys# from `std.38400` to `3wire.38400`, then run `kill -HUP 1` after modifying [.filename]#/etc/ttys#. The `3wire` entry is similar to `std`, but ignores hardware handshaking. The bps may also need to be reduced or software flow control enabled when using `3wire` to prevent buffer overflows. If garbage appears instead of a login prompt, make sure the terminal and FreeBSD agree on the bps rate and parity settings. Check the `getty` processes to make sure the correct _getty_ type is in use. If not, edit [.filename]#/etc/ttys# and run `kill -HUP 1`. If characters appear doubled and the password appears when typed, switch the terminal, or the terminal emulation software, from "half duplex" or "local echo" to "full duplex." [[dialup]] == Dial-in Service Configuring a FreeBSD system for dial-in service is similar to configuring terminals, except that modems are used instead of terminal devices. FreeBSD supports both external and internal modems. External modems are more convenient because they often can be configured via parameters stored in non-volatile RAM and they usually provide lighted indicators that display the state of important RS-232 signals, indicating whether the modem is operating properly. Internal modems usually lack non-volatile RAM, so their configuration may be limited to setting DIP switches. If the internal modem has any signal indicator lights, they are difficult to view when the system's cover is in place. When using an external modem, a proper cable is needed. A standard RS-232C serial cable should suffice. FreeBSD needs the RTS and CTS signals for flow control at speeds above 2400 bps, the CD signal to detect when a call has been answered or the line has been hung up, and the DTR signal to reset the modem after a session is complete. Some cables are wired without all of the needed signals, so if a login session does not go away when the line hangs up, there may be a problem with the cable. Refer to crossref:serialcomms[term-cables-null, Serial Cables and Ports] for more information about these signals. Like other UNIX(R)-like operating systems, FreeBSD uses the hardware signals to find out when a call has been answered or a line has been hung up and to hangup and reset the modem after a call. FreeBSD avoids sending commands to the modem or watching for status reports from the modem. FreeBSD supports the NS8250, NS16450, NS16550, and NS16550A-based RS-232C (CCITT V.24) communications interfaces. The 8250 and 16450 devices have single-character buffers. The 16550 device provides a 16-character buffer, which allows for better system performance. Bugs in plain 16550 devices prevent the use of the 16-character buffer, so use 16550A devices if possible. As single-character-buffer devices require more work by the operating system than the 16-character-buffer devices, 16550A-based serial interface cards are preferred. If the system has many active serial ports or will have a heavy load, 16550A-based cards are better for low-error-rate communications. The rest of this section demonstrates how to configure a modem to receive incoming connections, how to communicate with the modem, and offers some troubleshooting tips. [[dialup-ttys]] === Modem Configuration As with terminals, `init` spawns a `getty` process for each configured serial port used for dial-in connections. When a user dials the modem's line and the modems connect, the "Carrier Detect" signal is reported by the modem. The kernel notices that the carrier has been detected and instructs `getty` to open the port and display a `login:` prompt at the specified initial line speed. In a typical configuration, if garbage characters are received, usually due to the modem's connection speed being different than the configured speed, `getty` tries adjusting the line speeds until it receives reasonable characters. After the user enters their login name, `getty` executes `login`, which completes the login process by asking for the user's password and then starting the user's shell. There are two schools of thought regarding dial-up modems. One configuration method is to set the modems and systems so that no matter at what speed a remote user dials in, the dial-in RS-232 interface runs at a locked speed. The benefit of this configuration is that the remote user always sees a system login prompt immediately. The downside is that the system does not know what a user's true data rate is, so full-screen programs like Emacs will not adjust their screen-painting methods to make their response better for slower connections. The second method is to configure the RS-232 interface to vary its speed based on the remote user's connection speed. As `getty` does not understand any particular modem's connection speed reporting, it gives a `login:` message at an initial speed and watches the characters that come back in response. If the user sees junk, they should press kbd:[Enter] until they see a recognizable prompt. If the data rates do not match, `getty` sees anything the user types as junk, tries the next speed, and gives the `login:` prompt again. This procedure normally only takes a keystroke or two before the user sees a good prompt. This login sequence does not look as clean as the locked-speed method, but a user on a low-speed connection should receive better interactive response from full-screen programs. When locking a modem's data communications rate at a particular speed, no changes to [.filename]#/etc/gettytab# should be needed. However, for a matching-speed configuration, additional entries may be required in order to define the speeds to use for the modem. This example configures a 14.4 Kbps modem with a top interface speed of 19.2 Kbps using 8-bit, no parity connections. It configures `getty` to start the communications rate for a V.32bis connection at 19.2 Kbps, then cycles through 9600 bps, 2400 bps, 1200 bps, 300 bps, and back to 19.2 Kbps. Communications rate cycling is implemented with the `nx=` (next table) capability. Each line uses a `tc=` (table continuation) entry to pick up the rest of the settings for a particular data rate. [.programlisting] .... # # Additions for a V.32bis Modem # um|V300|High Speed Modem at 300,8-bit:\ :nx=V19200:tc=std.300: un|V1200|High Speed Modem at 1200,8-bit:\ :nx=V300:tc=std.1200: uo|V2400|High Speed Modem at 2400,8-bit:\ :nx=V1200:tc=std.2400: up|V9600|High Speed Modem at 9600,8-bit:\ :nx=V2400:tc=std.9600: uq|V19200|High Speed Modem at 19200,8-bit:\ :nx=V9600:tc=std.19200: .... For a 28.8 Kbps modem, or to take advantage of compression on a 14.4 Kbps modem, use a higher communications rate, as seen in this example: [.programlisting] .... # # Additions for a V.32bis or V.34 Modem # Starting at 57.6 Kbps # vm|VH300|Very High Speed Modem at 300,8-bit:\ :nx=VH57600:tc=std.300: vn|VH1200|Very High Speed Modem at 1200,8-bit:\ :nx=VH300:tc=std.1200: vo|VH2400|Very High Speed Modem at 2400,8-bit:\ :nx=VH1200:tc=std.2400: vp|VH9600|Very High Speed Modem at 9600,8-bit:\ :nx=VH2400:tc=std.9600: vq|VH57600|Very High Speed Modem at 57600,8-bit:\ :nx=VH9600:tc=std.57600: .... For a slow CPU or a heavily loaded system without 16550A-based serial ports, this configuration may produce `uart` "silo" errors at 57.6 Kbps. The configuration of [.filename]#/etc/ttys# is similar to crossref:serialcomms[ex-etc-ttys,Configuring Terminal Entries], but a different argument is passed to `getty` and `dialup` is used for the terminal type. Replace _xxx_ with the process `init` will run on the device: [.programlisting] .... ttyu0 "/usr/libexec/getty xxx" dialup on .... The `dialup` terminal type can be changed. For example, setting `vt102` as the default terminal type allows users to use VT102 emulation on their remote systems. For a locked-speed configuration, specify the speed with a valid type listed in [.filename]#/etc/gettytab#. This example is for a modem whose port speed is locked at 19.2 Kbps: [.programlisting] .... ttyu0 "/usr/libexec/getty std.19200" dialup on .... In a matching-speed configuration, the entry needs to reference the appropriate beginning "auto-baud" entry in [.filename]#/etc/gettytab#. To continue the example for a matching-speed modem that starts at 19.2 Kbps, use this entry: [.programlisting] .... ttyu0 "/usr/libexec/getty V19200" dialup on .... After editing [.filename]#/etc/ttys#, wait until the modem is properly configured and connected before signaling `init`: [source,shell] .... # kill -HUP 1 .... High-speed modems, like V.32, V.32bis, and V.34 modems, use hardware (`RTS/CTS`) flow control. Use `stty` to set the hardware flow control flag for the modem port. This example sets the `crtscts` flag on [.filename]#COM2#'s dial-in and dial-out initialization devices: [source,shell] .... # stty -f /dev/ttyu1.init crtscts # stty -f /dev/cuau1.init crtscts .... === Troubleshooting This section provides a few tips for troubleshooting a dial-up modem that will not connect to a FreeBSD system. Hook up the modem to the FreeBSD system and boot the system. If the modem has status indication lights, watch to see whether the modem's DTR indicator lights when the `login:` prompt appears on the system's console. If it lights up, that should mean that FreeBSD has started a `getty` process on the appropriate communications port and is waiting for the modem to accept a call. If the DTR indicator does not light, login to the FreeBSD system through the console and type `ps ax` to see if FreeBSD is running a `getty` process on the correct port: [source,shell] .... 114 ?? I 0:00.10 /usr/libexec/getty V19200 ttyu0 .... If the second column contains a `d0` instead of a `??` and the modem has not accepted a call yet, this means that `getty` has completed its open on the communications port. This could indicate a problem with the cabling or a misconfigured modem because `getty` should not be able to open the communications port until the carrier detect signal has been asserted by the modem. If no `getty` processes are waiting to open the port, double-check that the entry for the port is correct in [.filename]#/etc/ttys#. Also, check [.filename]#/var/log/messages# to see if there are any log messages from `init` or `getty`. Next, try dialing into the system. Be sure to use 8 bits, no parity, and 1 stop bit on the remote system. If a prompt does not appear right away, or the prompt shows garbage, try pressing kbd:[Enter] about once per second. If there is still no `login:` prompt, try sending a `BREAK`. When using a high-speed modem, try dialing again after locking the dialing modem's interface speed. If there is still no `login:` prompt, check [.filename]#/etc/gettytab# again and double-check that: * The initial capability name specified in the entry in [.filename]#/etc/ttys# matches the name of a capability in [.filename]#/etc/gettytab#. * Each `nx=` entry matches another [.filename]#gettytab# capability name. * Each `tc=` entry matches another [.filename]#gettytab# capability name. If the modem on the FreeBSD system will not answer, make sure that the modem is configured to answer the phone when DTR is asserted. If the modem seems to be configured correctly, verify that the DTR line is asserted by checking the modem's indicator lights. If it still does not work, try sending an email to the {freebsd-questions} describing the modem and the problem. [[dialout]] == Dial-out Service The following are tips for getting the host to connect over the modem to another computer. This is appropriate for establishing a terminal session with a remote host. This kind of connection can be helpful to get a file on the Internet if there are problems using PPP. If PPP is not working, use the terminal session to FTP the needed file. Then use zmodem to transfer it to the machine. [[hayes-unsupported]] === Using a Stock Hayes Modem A generic Hayes dialer is built into `tip`. Use `at=hayes` in [.filename]#/etc/remote#. The Hayes driver is not smart enough to recognize some of the advanced features of newer modems messages like `BUSY`, `NO DIALTONE`, or `CONNECT 115200`. Turn those messages off when using `tip` with `ATX0&W`. The dial timeout for `tip` is 60 seconds. The modem should use something less, or else `tip` will think there is a communication problem. Try `ATS7=45&W`. [[direct-at]] === Using `AT` Commands Create a "direct" entry in [.filename]#/etc/remote#. For example, if the modem is hooked up to the first serial port, [.filename]#/dev/cuau0#, use the following line: [.programlisting] .... cuau0:dv=/dev/cuau0:br#19200:pa=none .... Use the highest bps rate the modem supports in the `br` capability. Then, type `tip cuau0` to connect to the modem. Or, use `cu` as `root` with the following command: [source,shell] .... # cu -lline -sspeed .... _line_ is the serial port, such as [.filename]#/dev/cuau0#, and _speed_ is the speed, such as `57600`. When finished entering the AT commands, type `~.` to exit. [[gt-failure]] === The `@` Sign Does Not Work The `@` sign in the phone number capability tells `tip` to look in [.filename]#/etc/phones# for a phone number. But, the `@` sign is also a special character in capability files like [.filename]#/etc/remote#, so it needs to be escaped with a backslash: [.programlisting] .... pn=\@ .... [[dial-command-line]] === Dialing from the Command Line Put a "generic" entry in [.filename]#/etc/remote#. For example: [.programlisting] .... tip115200|Dial any phone number at 115200 bps:\ :dv=/dev/cuau0:br#115200:at=hayes:pa=none:du: tip57600|Dial any phone number at 57600 bps:\ :dv=/dev/cuau0:br#57600:at=hayes:pa=none:du: .... This should now work: [source,shell] .... # tip -115200 5551234 .... Users who prefer `cu` over `tip`, can use a generic `cu` entry: [.programlisting] .... cu115200|Use cu to dial any number at 115200bps:\ :dv=/dev/cuau1:br#57600:at=hayes:pa=none:du: .... and type: [source,shell] .... # cu 5551234 -s 115200 .... [[set-bps]] === Setting the bps Rate Put in an entry for `tip1200` or `cu1200`, but go ahead and use whatever bps rate is appropriate with the `br` capability. `tip` thinks a good default is 1200 bps which is why it looks for a `tip1200` entry. 1200 bps does not have to be used, though. [[terminal-server]] === Accessing a Number of Hosts Through a Terminal Server Rather than waiting until connected and typing `CONNECT _host_` each time, use ``tip``'s `cm` capability. For example, these entries in [.filename]#/etc/remote# allow typing `tip pain` or `tip muffin` to connect to the hosts `pain` or `muffin`, and `tip deep13` to connect to the terminal server. [.programlisting] .... pain|pain.deep13.com|Forrester's machine:\ :cm=CONNECT pain\n:tc=deep13: muffin|muffin.deep13.com|Frank's machine:\ :cm=CONNECT muffin\n:tc=deep13: deep13:Gizmonics Institute terminal server:\ :dv=/dev/cuau2:br#38400:at=hayes:du:pa=none:pn=5551234: .... [[tip-multiline]] === Using More Than One Line with `tip` This is often a problem where a university has several modem lines and several thousand students trying to use them. Make an entry in [.filename]#/etc/remote# and use `@` for the `pn` capability: [.programlisting] .... big-university:\ :pn=\@:tc=dialout dialout:\ :dv=/dev/cuau3:br#9600:at=courier:du:pa=none: .... Then, list the phone numbers in [.filename]#/etc/phones#: [.programlisting] .... big-university 5551111 big-university 5551112 big-university 5551113 big-university 5551114 .... `tip` will try each number in the listed order, then give up. To keep retrying, run `tip` in a `while` loop. [[multi-controlp]] === Using the Force Character kbd:[Ctrl+P] is the default "force" character, used to tell `tip` that the next character is literal data. The force character can be set to any other character with the `~s` escape, which means "set a variable." Type `~sforce=_single-char_` followed by a newline. _single-char_ is any single character. If _single-char_ is left out, then the force character is the null character, which is accessed by typing kbd:[Ctrl+2] or kbd:[Ctrl+Space]. A pretty good value for _single-char_ is kbd:[Shift+Ctrl+6], which is only used on some terminal servers. To change the force character, specify the following in [.filename]#~/.tiprc#: [.programlisting] .... force=single-char .... [[uppercase]] === Upper Case Characters This happens when kbd:[Ctrl+A] is pressed, which is ``tip``'s "raise character", specially designed for people with broken caps-lock keys. Use `~s` to set `raisechar` to something reasonable. It can be set to be the same as the force character, if neither feature is used. Here is a sample [.filename]#~/.tiprc# for Emacs users who need to type kbd:[Ctrl+2] and kbd:[Ctrl+A]: [.programlisting] .... force=^^ raisechar=^^ .... The `^^` is kbd:[Shift+Ctrl+6]. [[tip-filetransfer]] === File Transfers with `tip` When talking to another UNIX(R)-like operating system, files can be sent and received using `~p` (put) and `~t` (take). These commands run `cat` and `echo` on the remote system to accept and send files. The syntax is: `~p` local-file [ remote-file ] `~t` remote-file [ local-file ] There is no error checking, so another protocol, like zmodem, should probably be used. [[zmodem-tip]] === Using zmodem with `tip`? To receive files, start the sending program on the remote end. Then, type `~C rz` to begin receiving them locally. To send files, start the receiving program on the remote end. Then, type `~C sz _files_` to send them to the remote system. [[serialconsole-setup]] == Setting Up the Serial Console FreeBSD has the ability to boot a system with a dumb terminal on a serial port as a console. This configuration is useful for system administrators who wish to install FreeBSD on machines that have no keyboard or monitor attached, and developers who want to debug the kernel or device drivers. As described in crossref:boot[boot,The FreeBSD Booting Process], FreeBSD employs a three stage bootstrap. The first two stages are in the boot block code which is stored at the beginning of the FreeBSD slice on the boot disk. The boot block then loads and runs the boot loader as the third stage code. In order to set up booting from a serial console, the boot block code, the boot loader code, and the kernel need to be configured. [[serialconsole-howto-fast]] === Quick Serial Console Configuration This section provides a fast overview of setting up the serial console. This procedure can be used when the dumb terminal is connected to [.filename]#COM1#. [.procedure] .Procedure: Configuring a Serial Console on [.filename]#COM1# . Connect the serial cable to [.filename]#COM1# and the controlling terminal. . To configure boot messages to display on the serial console, issue the following command as the superuser: + [source,shell] .... # echo 'console="comconsole"' >> /boot/loader.conf .... . Edit [.filename]#/etc/ttys# and change `off` to `on` and `dialup` to `vt100` for the [.filename]#ttyu0# entry. Otherwise, a password will not be required to connect via the serial console, resulting in a potential security hole. . Reboot the system to see if the changes took effect. If a different configuration is required, see the next section for a more in-depth configuration explanation. [[serialconsole-howto]] === In-Depth Serial Console Configuration This section provides a more detailed explanation of the steps needed to setup a serial console in FreeBSD. [.procedure] .Procedure: Configuring a Serial Console . Prepare a serial cable. + Use either a null-modem cable or a standard serial cable and a null-modem adapter. See crossref:serialcomms[term-cables-null, Serial Cables and Ports] for a discussion on serial cables. . Unplug the keyboard. + Many systems probe for the keyboard during the Power-On Self-Test (POST) and will generate an error if the keyboard is not detected. Some machines will refuse to boot until the keyboard is plugged in. + If the computer complains about the error, but boots anyway, no further configuration is needed. + If the computer refuses to boot without a keyboard attached, configure the BIOS so that it ignores this error. Consult the motherboard's manual for details on how to do this. + [TIP] ==== Try setting the keyboard to "Not installed" in the BIOS. This setting tells the BIOS not to probe for a keyboard at power-on so it should not complain if the keyboard is absent. If that option is not present in the BIOS, look for an "Halt on Error" option instead. Setting this to "All but Keyboard" or to "No Errors" will have the same effect. ==== + If the system has a PS/2(R) mouse, unplug it as well. PS/2(R) mice share some hardware with the keyboard and leaving the mouse plugged in can fool the keyboard probe into thinking the keyboard is still there. + [NOTE] ==== While most systems will boot without a keyboard, quite a few will not boot without a graphics adapter. Some systems can be configured to boot with no graphics adapter by changing the "graphics adapter" setting in the BIOS configuration to "Not installed". Other systems do not support this option and will refuse to boot if there is no display hardware in the system. With these machines, leave some kind of graphics card plugged in, even if it is just a junky mono board. A monitor does not need to be attached. ==== . Plug a dumb terminal, an old computer with a modem program, or the serial port on another UNIX(R) box into the serial port. . Add the appropriate `hint.uart.*` entries to [.filename]#/boot/device.hints# for the serial port. Some multi-port cards also require kernel configuration options. Refer to man:uart[4] for the required options and device hints for each supported serial port. . Create [.filename]#boot.config# in the root directory of the `a` partition on the boot drive. + This file instructs the boot block code how to boot the system. In order to activate the serial console, one or more of the following options are needed. When using multiple options, include them all on the same line: + `-h`::: Toggles between the internal and serial consoles. Use this to switch console devices. For instance, to boot from the internal (video) console, use `-h` to direct the boot loader and the kernel to use the serial port as its console device. Alternatively, to boot from the serial port, use `-h` to tell the boot loader and the kernel to use the video display as the console instead. `-D`::: Toggles between the single and dual console configurations. In the single configuration, the console will be either the internal console (video display) or the serial port, depending on the state of `-h`. In the dual console configuration, both the video display and the serial port will become the console at the same time, regardless of the state of `-h`. However, the dual console configuration takes effect only while the boot block is running. Once the boot loader gets control, the console specified by `-h` becomes the only console. `-P`::: Makes the boot block probe the keyboard. If no keyboard is found, the `-D` and `-h` options are automatically set. + [NOTE] ==== Due to space constraints in the current version of the boot blocks, `-P` is capable of detecting extended keyboards only. Keyboards with less than 101 keys and without F11 and F12 keys may not be detected. Keyboards on some laptops may not be properly found because of this limitation. If this is the case, do not use `-P`. ==== + Use either `-P` to select the console automatically or `-h` to activate the serial console. Refer to man:boot[8] and man:boot.config[5] for more details. + The options, except for `-P`, are passed to the boot loader. The boot loader will determine whether the internal video or the serial port should become the console by examining the state of `-h`. This means that if `-D` is specified but `-h` is not specified in [.filename]#/boot.config#, the serial port can be used as the console only during the boot block as the boot loader will use the internal video display as the console. . Boot the machine. + When FreeBSD starts, the boot blocks echo the contents of [.filename]#/boot.config# to the console. For example: + [source,shell] .... /boot.config: -P Keyboard: no .... + The second line appears only if `-P` is in [.filename]#/boot.config# and indicates the presence or absence of the keyboard. These messages go to either the serial or internal console, or both, depending on the option in [.filename]#/boot.config#: + [.informaltable] [cols="1,1", frame="none", options="header"] |=== <| Options <| Message goes to |none |internal console |`-h` |serial console |`-D` |serial and internal consoles |`-Dh` |serial and internal consoles |`-P`, keyboard present |internal console |`-P`, keyboard absent |serial console |=== + After the message, there will be a small pause before the boot blocks continue loading the boot loader and before any further messages are printed to the console. Under normal circumstances, there is no need to interrupt the boot blocks, but one can do so in order to make sure things are set up correctly. + Press any key, other than kbd:[Enter], at the console to interrupt the boot process. The boot blocks will then prompt for further action: + [source,shell] .... >> FreeBSD/i386 BOOT Default: 0:ad(0,a)/boot/loader boot: .... + Verify that the above message appears on either the serial or internal console, or both, according to the options in [.filename]#/boot.config#. If the message appears in the correct console, press kbd:[Enter] to continue the boot process. + If there is no prompt on the serial terminal, something is wrong with the settings. Enter `-h` then kbd:[Enter] or kbd:[Return] to tell the boot block (and then the boot loader and the kernel) to choose the serial port for the console. Once the system is up, go back and check what went wrong. During the third stage of the boot process, one can still switch between the internal console and the serial console by setting appropriate environment variables in the boot loader. See man:loader[8] for more information. [NOTE] ==== This line in [.filename]#/boot/loader.conf# or [.filename]#/boot/loader.conf.local# configures the boot loader and the kernel to send their boot messages to the serial console, regardless of the options in [.filename]#/boot.config#: [.programlisting] .... console="comconsole" .... That line should be the first line of [.filename]#/boot/loader.conf# so that boot messages are displayed on the serial console as early as possible. If that line does not exist, or if it is set to `console="vidconsole"`, the boot loader and the kernel will use whichever console is indicated by `-h` in the boot block. See man:loader.conf[5] for more information. At the moment, the boot loader has no option equivalent to `-P` in the boot block, and there is no provision to automatically select the internal console and the serial console based on the presence of the keyboard. ==== [TIP] ==== While it is not required, it is possible to provide a `login` prompt over the serial line. To configure this, edit the entry for the serial port in [.filename]#/etc/ttys# using the instructions in crossref:serialcomms[term-config, Terminal Configuration]. If the speed of the serial port has been changed, change `std.115200` to match the new setting. ==== === Setting a Faster Serial Port Speed By default, the serial port settings are 115200 baud, 8 bits, no parity, and 1 stop bit. To change the default console speed, use one of the following options: * Edit [.filename]#/etc/make.conf# and set `BOOT_COMCONSOLE_SPEED` to the new console speed. Then, recompile and install the boot blocks and the boot loader: + [source,shell] .... # cd /sys/boot # make clean # make # make install .... + If the serial console is configured in some other way than by booting with `-h`, or if the serial console used by the kernel is different from the one used by the boot blocks, add the following option, with the desired speed, to a custom kernel configuration file and compile a new kernel: + [.programlisting] .... options CONSPEED=19200 .... * Add the `-S__19200__` boot option to [.filename]#/boot.config#, replacing `_19200_` with the speed to use. * Add the following options to [.filename]#/boot/loader.conf#. Replace `_115200_` with the speed to use. + [.programlisting] .... boot_multicons="YES" boot_serial="YES" comconsole_speed="115200" console="comconsole,vidconsole" .... [[serialconsole-ddb]] === Entering the DDB Debugger from the Serial Line To configure the ability to drop into the kernel debugger from the serial console, add the following options to a custom kernel configuration file and compile the kernel using the instructions in crossref:kernelconfig[kernelconfig,Configuring the FreeBSD Kernel]. Note that while this is useful for remote diagnostics, it is also dangerous if a spurious BREAK is generated on the serial port. Refer to man:ddb[4] and man:ddb[8] for more information about the kernel debugger. [.programlisting] .... options BREAK_TO_DEBUGGER options DDB .... diff --git a/documentation/content/en/books/porters-handbook/makefiles/_index.adoc b/documentation/content/en/books/porters-handbook/makefiles/_index.adoc index a9ffe801b9..72aa3c5f0a 100644 --- a/documentation/content/en/books/porters-handbook/makefiles/_index.adoc +++ b/documentation/content/en/books/porters-handbook/makefiles/_index.adoc @@ -1,5450 +1,5450 @@ --- title: Chapter 5. Configuring the Makefile prev: books/porters-handbook/slow-porting next: books/porters-handbook/special description: Configuring the Makefile for FreeBSD Ports tags: ["makefiles", "configuring", "naming", "versions"] showBookMenu: true weight: 5 params: path: "/books/porters-handbook/makefiles/" --- [[makefiles]] = Configuring the Makefile :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumoffset: 5 :partnums: :source-highlighter: rouge :experimental: :g-plus-plus: g++ :images-path: books/porters-handbook/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] Configuring the [.filename]#Makefile# is pretty simple, and again we suggest looking at existing examples before starting. Also, there is a crossref:porting-samplem[porting-samplem,sample Makefile] in this handbook, so take a look and please follow the ordering of variables and sections in that template to make the port easier for others to read. Consider these problems in sequence during the design of the new [.filename]#Makefile#: [[makefile-source]] == The Original Source Does it live in `DISTDIR` as a standard ``gzip``ped tarball named something like [.filename]#foozolix-1.2.tar.gz#? If so, go on to the next step. If not, the distribution file format might require overriding one or more of `DISTVERSION`, `DISTNAME`, `EXTRACT_CMD`, `EXTRACT_BEFORE_ARGS`, `EXTRACT_AFTER_ARGS`, `EXTRACT_SUFX`, or `DISTFILES`. In the worst case, create a custom `do-extract` target to override the default. This is rarely, if ever, necessary. [[makefile-naming]] == Naming The first part of the port's [.filename]#Makefile# names the port, describes its version number, and lists it in the correct category. [[makefile-portname]] === `PORTNAME` Set `PORTNAME` to the base name of the software. It is used as the base for the FreeBSD package, and for crossref:makefiles[makefile-distname,`DISTNAME`]. [IMPORTANT] ==== The package name must be unique across the entire ports tree. Make sure that the `PORTNAME` is not already in use by an existing port, and that no other port already has the same `PKGBASE`. If the name has already been used, add either crossref:makefiles[porting-pkgnameprefix-suffix,`PKGNAMEPREFIX` or `PKGNAMESUFFIX`]. ==== [[makefile-versions]] === Versions, `DISTVERSION` _or_ `PORTVERSION` Set `DISTVERSION` to the version number of the software. `PORTVERSION` is the version used for the FreeBSD package. It will be automatically derived from `DISTVERSION` to be compatible with FreeBSD's package versioning scheme. If the version contains _letters_, it might be needed to set `PORTVERSION` and not `DISTVERSION`. [IMPORTANT] ==== Only one of `PORTVERSION` and `DISTVERSION` can be set at a time. ==== From time to time, some software will use a version scheme that is not compatible with how `DISTVERSION` translates in `PORTVERSION`. [TIP] ==== When updating a port, it is possible to use the `-t` argument of man:pkg-version[8] to check if the new version is greater or lesser than before. See below on how to use man:pkg-version[8] to compare versions. ==== [[makefile-versions-ex-pkg-version]] .Using man:pkg-version[8] to Compare Versions [example] ==== `pkg version -t` takes two versions as arguments, it will respond with `<`, `=` or `>` if the first version is less, equal, or more than the second version, respectively. [source,shell] .... % pkg version -t 1.2 1.3 < <.> % pkg version -t 1.2 1.2 = <.> % pkg version -t 1.2 1.2.0 = <.> % pkg version -t 1.2 1.2.p1 > <.> % pkg version -t 1.2.a1 1.2.b1 < <.> % pkg version -t 1.2 1.2p1 < <.> .... <.> `1.2` is before `1.3`. <.> `1.2` and `1.2` are equal as they have the same version. <.> `1.2` and `1.2.0` are equal as nothing equals zero. <.> `1.2` is after `1.2.p1` as `.p1`, think "pre-release 1". <.> `1.2.a1` is before `1.2.b1`, think "alpha" and "beta", and `a` is before `b`. <.> `1.2` is before `1.2p1` as `2p1`, think "2, patch level 1" which is a version after any `2.X` but before `3`. [NOTE] ===== In here, the `a`, `b`, and `p` are used as if meaning "alpha", "beta" or "pre-release" and "patch level", but they are only letters and are sorted alphabetically, so any letter can be used, and they will be sorted appropriately. ===== ==== .Examples of `DISTVERSION` and the Derived `PORTVERSION` [cols="10%,90%", frame="none", options="header"] |=== | DISTVERSION | PORTVERSION |0.7.1d |0.7.1.d |10Alpha3 |10.a3 |3Beta7-pre2 |3.b7.p2 |8:f_17 |8f.17 |=== [[makefile-versions-ex1]] .Using `DISTVERSION` [example] ==== When the version only contains numbers separated by dots, dashes or underscores, use `DISTVERSION`. [.programlisting] .... PORTNAME= nekoto DISTVERSION= 1.2-4 .... It will generate a `PORTVERSION` of `1.2.4`. ==== [[makefile-versions-ex2]] .Using `DISTVERSION` When the Version Starts with a Letter or a Prefix [example] ==== When the version starts or ends with a letter, or a prefix or a suffix that is not part of the version, use `DISTVERSIONPREFIX`, `DISTVERSION`, and `DISTVERSIONSUFFIX`. If the version is `v1.2-4`: [.programlisting] .... PORTNAME= nekoto DISTVERSIONPREFIX= v DISTVERSION= 1_2_4 .... Some of the time, projects using GitHub will use their name in their versions. For example, the version could be `nekoto-1.2-4`: [.programlisting] .... PORTNAME= nekoto DISTVERSIONPREFIX= nekoto- DISTVERSION= 1.2_4 .... Those projects also sometimes use some string at the end of the version, for example, `1.2-4_RELEASE`: [.programlisting] .... PORTNAME= nekoto DISTVERSION= 1.2-4 DISTVERSIONSUFFIX= _RELEASE .... Or they do both, for example, `nekoto-1.2-4_RELEASE`: [.programlisting] .... PORTNAME= nekoto DISTVERSIONPREFIX= nekoto- DISTVERSION= 1.2-4 DISTVERSIONSUFFIX= _RELEASE .... `DISTVERSIONPREFIX` and `DISTVERSIONSUFFIX` will not be used while constructing `PORTVERSION`, but only used in `DISTNAME`. All will generate a `PORTVERSION` of `1.2.4`. ==== [[makefile-versions-ex3]] .Using `DISTVERSION` When the Version Contains Letters Meaning "alpha", "beta", or "pre-release" [example] ==== When the version contains numbers separated by dots, dashes or underscores, and letters are used to mean "alpha", "beta" or "pre-release", which is, before the version without the letters, use `DISTVERSION`. [.programlisting] .... PORTNAME= nekoto DISTVERSION= 1.2-pre4 .... [.programlisting] .... PORTNAME= nekoto DISTVERSION= 1.2p4 .... Both will generate a `PORTVERSION` of `1.2.p4` which is before than 1.2. man:pkg-version[8] can be used to check that fact: [source,shell] .... % pkg version -t 1.2.p4 1.2 < .... ==== [[makefile-versions-ex4]] .Not Using `DISTVERSION` When the Version Contains Letters Meaning "Patch Level" [example] ==== When the version contains letters that are not meant as "alpha", "beta", or "pre", but more in a "patch level", and meaning after the version without the letters, use `PORTVERSION`. [.programlisting] .... PORTNAME= nekoto PORTVERSION= 1.2p4 .... In this case, using `DISTVERSION` is not possible because it would generate a version of `1.2.p4` which would be before `1.2` and not after. man:pkg-version[8] will verify this: [source,shell] .... % pkg version -t 1.2 1.2.p4 > <.> % pkg version -t 1.2 1.2p4 < <.> .... <.> `1.2` is after `1.2.p4`, which is _wrong_ in this case. <.> `1.2` is before `1.2p4`, which is what was needed. ==== For some more advanced examples of setting `PORTVERSION`, when the software's versioning is really not compatible with FreeBSD's, or `DISTNAME` when the distribution file does not contain the version itself, see crossref:makefiles[makefile-distname, `DISTNAME`]. [[makefile-naming-revepoch]] === `PORTREVISION` and `PORTEPOCH` [[makefile-portrevision]] ==== `PORTREVISION` `PORTREVISION` is a monotonically increasing value which is reset to 0 with every increase of `DISTVERSION`, typically every time there is a new official vendor release. If `PORTREVISION` is non-zero, the value is appended to the package name. Changes to `PORTREVISION` are used by automated tools like man:pkg-version[8] to determine that a new package is available. `PORTREVISION` must be increased each time a change is made to the port that changes the generated package in any way. That includes changes that only affect a package built with non-default crossref:makefiles[makefile-options,options]. Examples of when `PORTREVISION` must be bumped: * Addition of patches to correct security vulnerabilities, bugs, or to add new functionality to the port. * Changes to the port [.filename]#Makefile# to enable or disable compile-time options in the package. * Changes in the packing list or the install-time behavior of the package. For example, a change to a script which generates initial data for the package, like man:ssh[1] host keys. * Version bump of a port's shared library dependency (in this case, someone trying to install the old package after installing a newer version of the dependency will fail since it will look for the old libfoo.x instead of libfoo.(x+1)). * Silent changes to the port distfile which have significant functional differences. For example, changes to the distfile requiring a correction to [.filename]#distinfo# with no corresponding change to `DISTVERSION`, where a `diff -ru` of the old and new versions shows non-trivial changes to the code. * Changes to `MAINTAINER`. Examples of changes which do not require a `PORTREVISION` bump: * Style changes to the port skeleton with no functional change to what appears in the resulting package. * Changes to `MASTER_SITES` or other functional changes to the port which do not affect the resulting package. * Trivial patches to the distfile such as correction of typos, which are not important enough that users of the package have to go to the trouble of upgrading. * Build fixes which cause a package to become compilable where it was previously failing. As long as the changes do not introduce any functional change on any other platforms on which the port did previously build. Since `PORTREVISION` reflects the content of the package, if the package was not previously buildable then there is no need to increase `PORTREVISION` to mark a change. A rule of thumb is to decide whether a change committed to a port is something which _some_ people would benefit from having. Either because of an enhancement, fix, or by virtue that the new package will actually work at all. Then weigh that against that fact that it will cause everyone who regularly updates their ports tree to be compelled to update. If yes, `PORTREVISION` must be bumped. [NOTE] ==== People using binary packages will _never_ see the update if `PORTREVISION` is not bumped. Without increasing `PORTREVISION`, the package builders have no way to detect the change and thus, will not rebuild the package. ==== [[makefile-portepoch]] ==== `PORTEPOCH` From time to time a software vendor or FreeBSD porter will do something silly and release a version of their software which is actually numerically less than the previous version. An example of this is a port which goes from foo-20000801 to foo-1.0 (the former will be incorrectly treated as a newer version since 20000801 is a numerically greater value than 1). [TIP] ==== The results of version number comparisons are not always obvious. `pkg version` (see man:pkg-version[8]) can be used to test the comparison of two version number strings. For example: [source,shell] .... % pkg version -t 0.031 0.29 > .... The `>` output indicates that version 0.031 is considered greater than version 0.29, which may not have been obvious to the porter. ==== In situations such as this, `PORTEPOCH` must be increased. If `PORTEPOCH` is nonzero it is appended to the package name as described in section 0 above. `PORTEPOCH` must never be decreased or reset to zero, because that would cause comparison to a package from an earlier epoch to fail. For example, the package would not be detected as out of date. The new version number, `1.0,1` in the above example, is still numerically less than the previous version, 20000801, but the `,1` suffix is treated specially by automated tools and found to be greater than the implied suffix `,0` on the earlier package. Dropping or resetting `PORTEPOCH` incorrectly leads to no end of grief. If the discussion above was not clear enough, please consult the {freebsd-ports}. It is expected that `PORTEPOCH` will not be used for the majority of ports, and that sensible use of `DISTVERSION`, or that use `PORTVERSION` carefully, can often preempt it becoming necessary if a future release of the software changes the version structure. However, care is needed by FreeBSD porters when a vendor release is made without an official version number - such as a code "snapshot" release. The temptation is to label the release with the release date, which will cause problems as in the example above when a new "official" release is made. For example, if a snapshot release is made on the date `20000917`, and the previous version of the software was version `1.2`, do not use `20000917` for `DISTVERSION`. The correct way is a `DISTVERSION` of `1.2.20000917`, or similar, so that the succeeding release, say `1.3`, is still a numerically greater value. [[makefile-portrevision-example]] ==== Example of `PORTREVISION` and `PORTEPOCH` Usage The `gtkmumble` port, version `0.10`, is committed to the ports collection: [.programlisting] .... PORTNAME= gtkmumble DISTVERSION= 0.10 .... `PKGNAME` becomes `gtkmumble-0.10`. A security hole is discovered which requires a local FreeBSD patch. `PORTREVISION` is bumped accordingly. [.programlisting] .... PORTNAME= gtkmumble DISTVERSION= 0.10 PORTREVISION= 1 .... `PKGNAME` becomes `gtkmumble-0.10_1` A new version is released by the vendor, numbered `0.2` (it turns out the author actually intended `0.10` to actually mean `0.1.0`, not "what comes after 0.9" - oops, too late now). Since the new minor version `2` is numerically less than the previous version `10`, `PORTEPOCH` must be bumped to manually force the new package to be detected as "newer". Since it is a new vendor release of the code, `PORTREVISION` is reset to 0 (or removed from the [.filename]#Makefile#). [.programlisting] .... PORTNAME= gtkmumble DISTVERSION= 0.2 PORTEPOCH= 1 .... `PKGNAME` becomes `gtkmumble-0.2,1` The next release is 0.3. Since `PORTEPOCH` never decreases, the version variables are now: [.programlisting] .... PORTNAME= gtkmumble DISTVERSION= 0.3 PORTEPOCH= 1 .... `PKGNAME` becomes `gtkmumble-0.3,1` [NOTE] ==== If `PORTEPOCH` were reset to `0` with this upgrade, someone who had installed the `gtkmumble-0.10_1` package would not detect the `gtkmumble-0.3` package as newer, since `3` is still numerically less than `10`. Remember, this is the whole point of `PORTEPOCH` in the first place. ==== [[porting-pkgnameprefix-suffix]] === `PKGNAMEPREFIX` and `PKGNAMESUFFIX` Two optional variables, `PKGNAMEPREFIX` and `PKGNAMESUFFIX`, are combined with `PORTNAME` and `PORTVERSION` to form `PKGNAME` as `${PKGNAMEPREFIX}${PORTNAME}${PKGNAMESUFFIX}-${PORTVERSION}`. Make sure this conforms to our crossref:makefiles[porting-pkgname,guidelines for a good package name]. In particular, the use of a hyphen (`-`) in `PORTVERSION` is _not_ allowed. Also, if the package name has the _language-_ or the _-compiled.specifics_ part (see below), use `PKGNAMEPREFIX` and `PKGNAMESUFFIX`, respectively. Do not make them part of `PORTNAME`. [[porting-pkgname]] === Package Naming Conventions These are the conventions to follow when naming packages. This is to make the package directory easy to scan, as there are already thousands of packages and users are going to turn away if they hurt their eyes! Package names take the form of [.filename]#language_region-name-compiled.specifics-version.numbers#. The package name is defined as `${PKGNAMEPREFIX}${PORTNAME}${PKGNAMESUFFIX}-${PORTVERSION}`. Make sure to set the variables to conform to that format. [[porting-pkgname-language]] [.filename]#language_region-#:: FreeBSD strives to support the native language of its users. The _language-_ part is a two letter abbreviation of the natural language defined by ISO-639 when the port is specific to a certain language. Examples are `ja` for Japanese, `ru` for Russian, `vi` for Vietnamese, `zh` for Chinese, `ko` for Korean and `de` for German. + If the port is specific to a certain region within the language area, add the two letter country code as well. Examples are `en_US` for US English and `fr_CH` for Swiss French. + The _language-_ part is set in `PKGNAMEPREFIX`. [[porting-pkgname-name]] [.filename]#name#:: Make sure that the port's name and version are clearly separated and placed into `PORTNAME` and `DISTVERSION`. The only reason for `PORTNAME` to contain a version part is if the upstream distribution is really named that way, as in the package:textproc/libxml2[] or package:japanese/kinput2-freewnn[] ports. Otherwise, `PORTNAME` cannot contain any version-specific information. It is quite normal for several ports to have the same `PORTNAME`, as the package:www/apache*[] ports do; in that case, different versions (and different index entries) are distinguished by `PKGNAMEPREFIX` and `PKGNAMESUFFIX` values. + There is a tradition of naming `Perl 5` modules by prepending `p5-` and converting the double-colon separator to a hyphen. For example, the `Data::Dumper` module becomes `p5-Data-Dumper`. [[porting-pkgname-compiled-specifics]] [.filename]#-compiled.specifics#:: If the port can be built with different crossref:makefiles[makefile-masterdir,hardcoded defaults] (usually part of the directory name in a family of ports), the _-compiled.specifics_ part states the compiled-in defaults. The hyphen is optional. Examples are paper size and font units. + The _-compiled.specifics_ part is set in `PKGNAMESUFFIX`. [[porting-pkgname-version-numbers]] [.filename]#-version.numbers#:: The version string follows a dash (`-`) and is a period-separated list of integers and single lowercase alphabetics. In particular, it is not permissible to have another dash inside the version string. The only exception is the string `pl` (meaning "patchlevel"), which can be used _only_ when there are no major and minor version numbers in the software. If the software version has strings like "alpha", "beta", "rc", or "pre", take the first letter and put it immediately after a period. If the version string continues after those names, the numbers follow the single alphabet without an extra period between them (for example, `1.0b2`). + The idea is to make it easier to sort ports by looking at the version string. In particular, make sure version number components are always delimited by a period, and if the date is part of the string, use the `d__yyyy.mm.dd__` format, not `_dd.mm.yyyy_` or the non-Y2K compliant `_yy.mm.dd_` format. It is important to prefix the version with a letter, here `d` (for date), in case a release with an actual version number is made, which would be numerically less than `_yyyy_`. [IMPORTANT] ==== Package name must be unique among all of the ports tree, check that there is not already a port with the same `PORTNAME` and if there is add one of crossref:makefiles[porting-pkgnameprefix-suffix,`PKGNAMEPREFIX` or `PKGNAMESUFFIX`]. ==== Here are some (real) examples on how to convert the name as called by the software authors to a suitable package name, for each line, only one of `DISTVERSION` or `PORTVERSION` is set in, depending on which would be used in the port's [.filename]#Makefile#: .Package Naming Examples [cols="1,1,1,1,1,1,1", frame="none", options="header"] |=== | Distribution Name | PKGNAMEPREFIX | PORTNAME | PKGNAMESUFFIX | DISTVERSION | PORTVERSION | Reason or comment |mule-2.2.2 |(empty) |mule |(empty) |2.2.2 | |No changes required |mule-1.0.1 |(empty) |mule |1 |1.0.1 | |This is version 1 of mule, and version 2 already exists |EmiClock-1.0.2 |(empty) |emiclock |(empty) |1.0.2 | |No uppercase names for single programs |rdist-1.3alpha |(empty) |rdist |(empty) |1.3alpha | |Version will be `1.3.a` |es-0.9-beta1 |(empty) |es |(empty) |0.9-beta1 | |Version will be `0.9.b1` |mailman-2.0rc3 |(empty) |mailman |(empty) |2.0rc3 | |Version will be `2.0.r3` |v3.3beta021.src |(empty) |tiff |(empty) | |3.3 |What the heck was that anyway? |tvtwm |(empty) |tvtwm |(empty) | |p11 |No version in the filename, use what upstream says it is |piewm |(empty) |piewm |(empty) |1.0 | |No version in the filename, use what upstream says it is |xvgr-2.10pl1 |(empty) |xvgr |(empty) | |2.10.pl1 |In that case, `pl1` means patch level, so using DISTVERSION is not possible. |gawk-2.15.6 |ja- |gawk |(empty) |2.15.6 | |Japanese language version |psutils-1.13 |(empty) |psutils |-letter |1.13 | |Paper size hardcoded at package build time |pkfonts |(empty) |pkfonts |300 |1.0 | |Package for 300dpi fonts |=== If there is absolutely no trace of version information in the original source and it is unlikely that the original author will ever release another version, just set the version string to `1.0` (like the `piewm` example above). Otherwise, ask the original author or use the date string the source file was released on (`d__yyyy.mm.dd__`, or `d__yyyymmdd__`) as the version. [TIP] ==== Use any letter. Here, `d` here stands for date, if the source is a Git repository, `g` followed by the commit date is commonly used, using `s` for snapshot is also common. ==== [[makefile-categories]] == Categorization [[makefile-categories-definition]] === `CATEGORIES` When a package is created, it is put under [.filename]#/usr/ports/packages/All# and links are made from one or more subdirectories of [.filename]#/usr/ports/packages#. The names of these subdirectories are specified by the variable `CATEGORIES`. It is intended to make life easier for the user when he is wading through the pile of packages on the FTP site or the CDROM. Please take a look at the crossref:makefiles[porting-categories,current list of categories] and pick the ones that are suitable for the port. This list also determines where in the ports tree the port is imported. If there is more than one category here, the port files must be put in the subdirectory with the name of the first category. See crossref:makefiles[choosing-categories,below] for more discussion about how to pick the right categories. [[porting-categories]] === Current List of Categories Here is the current list of port categories. Those marked with an asterisk (`*`) are _virtual_ categories-those that do not have a corresponding subdirectory in the ports tree. They are only used as secondary categories, and only for search purposes. [NOTE] ==== For non-virtual categories, there is a one-line description in `COMMENT` in that subdirectory's [.filename]#Makefile#. ==== [.informaltable] [cols="1,1,1", frame="none", options="header"] |=== | Category | Description | Notes |[.filename]#accessibility# |Ports to help disabled users. | |[.filename]#afterstep#`*` |Ports to support the http://www.afterstep.org/[AfterStep] window manager. | |[.filename]#arabic# |Arabic language support. | |[.filename]#archivers# |Archiving tools. | |[.filename]#astro# |Astronomical ports. | |[.filename]#audio# |Sound support. | |[.filename]#benchmarks# |Benchmarking utilities. | |[.filename]#biology# |Biology-related software. | |[.filename]#budgie#`*` |Software related to the Budgie desktop environment. | |[.filename]#cad# |Computer aided design tools. | |[.filename]#chinese# |Chinese language support. | |[.filename]#comms# |Communication software. |Mostly software to talk to the serial port. |[.filename]#converters# |Character code converters. | |[.filename]#databases# |Databases. | |[.filename]#deskutils# |Things that used to be on the desktop before computers were invented. | |[.filename]#devel# |Development utilities. |Do not put libraries here just because they are libraries. They should _not_ be in this category unless they truly do not belong anywhere else. |[.filename]#dns# |DNS-related software. | |[.filename]#docs#`*` |Meta-ports for FreeBSD documentation. | |[.filename]#editors# |General editors. |Specialized editors go in the section for those tools. For example, a mathematical-formula editor will go in [.filename]#math#, and have [.filename]#editors# as a second category. |[.filename]#education#`*` |Education-related software. |This includes applications, utilities, or games primarily or substantially designed to help the user learn a specific topic or study in general. It also includes course-writing applications, course-delivery applications, and classroom or school management applications |[.filename]#elisp#`*` |Emacs-lisp ports. | |[.filename]#emulators# |Emulators for other operating systems. |Terminal emulators do _not_ belong here. X-based ones go to [.filename]#x11# and text-based ones to either [.filename]#comms# or [.filename]#misc#, depending on the exact functionality. |[.filename]#enlightenment#`*` |Ports related to the Enlightenment window manager. | |[.filename]#filesystems# |File systems and related utilities. | |[.filename]#finance# |Monetary, financial and related applications. | |[.filename]#french# |French language support. | |[.filename]#ftp# |FTP client and server utilities. |If the port speaks both FTP and HTTP, put it in [.filename]#ftp# with a secondary category of [.filename]#www#. |[.filename]#games# |Games. | |[.filename]#geography#`*` |Geography-related software. | |[.filename]#german# |German language support. | |[.filename]#gnome#`*` |Ports from the https://www.gnome.org/[GNOME] Project. | |[.filename]#gnustep#`*` |Software related to the GNUstep desktop environment. | |[.filename]#graphics# |Graphics utilities. | |[.filename]#hamradio#`*` |Software for amateur radio. | |[.filename]#haskell#`*` |Software related to the Haskell language. | |[.filename]#hebrew# |Hebrew language support. | |[.filename]#hungarian# |Hungarian language support. | |[.filename]#irc# |Internet Relay Chat utilities. | |[.filename]#japanese# |Japanese language support. | |[.filename]#java# |Software related to the Java(TM) language. |The [.filename]#java# category must not be the only one for a port. Save for ports directly related to the Java language, porters are also encouraged not to use [.filename]#java# as the main category of a port. |[.filename]#kde#`*` |Ports from the https://www.kde.org/[KDE] Project (generic). | |[.filename]#kde-applications#`*` |Applications from the https://www.kde.org/[KDE] Project. | |[.filename]#kde-frameworks#`*` |Add-on libraries from the https://www.kde.org/[KDE] Project for programming with Qt. | |[.filename]#kde-plasma#`*` |Desktop from the https://www.kde.org/[KDE] Project. | |[.filename]#kld#`*` |Kernel loadable modules. | |[.filename]#korean# |Korean language support. | |[.filename]#lang# |Programming languages. | |[.filename]#linux#`*` |Linux applications and support utilities. | |[.filename]#lisp#`*` |Software related to the Lisp language. | |[.filename]#mail# |Mail software. | |[.filename]#mate#`*` |Ports related to the MATE desktop environment, a fork of GNOME 2. | |[.filename]#math# |Numerical computation software and other utilities for mathematics. | |[.filename]#mbone#`*` |MBone applications. | |[.filename]#misc# |Miscellaneous utilities |Things that do not belong anywhere else. If at all possible, try to find a better category for the port than `misc`, as ports tend to be overlooked in here. |[.filename]#multimedia# |Multimedia software. | |[.filename]#net# |Miscellaneous networking software. | |[.filename]#net-im# |Instant messaging software. | |[.filename]#net-mgmt# |Networking management software. | |[.filename]#net-p2p# |Peer to peer network applications. | |[.filename]#net-vpn#`*` |Virtual Private Network applications. | |[.filename]#news# |USENET news software. | |[.filename]#parallel#`*` |Applications dealing with parallelism in computing. | |[.filename]#pear#`*` |Ports related to the Pear PHP framework. | |[.filename]#perl5#`*` |Ports that require Perl version 5 to run. | |[.filename]#plan9#`*` |Various programs from https://9p.io/wiki/plan9/Download/index.html[Plan9]. | |[.filename]#polish# |Polish language support. | |[.filename]#ports-mgmt# |Ports for managing, installing and developing FreeBSD ports and packages. | |[.filename]#portuguese# |Portuguese language support. | |[.filename]#print# |Printing software. |Desktop publishing tools (previewers, etc.) belong here too. |[.filename]#python#`*` |Software related to the https://www.python.org/[Python] language. | |[.filename]#ruby#`*` |Software related to the https://www.ruby-lang.org/[Ruby] language. | |[.filename]#rubygems#`*` |Ports of https://www.rubygems.org/[RubyGems] packages. | |[.filename]#russian# |Russian language support. | |[.filename]#scheme#`*` |Software related to the Scheme language. | |[.filename]#science# |Scientific ports that do not fit into other categories such as [.filename]#astro#, [.filename]#biology# and [.filename]#math#. | |[.filename]#security# |Security utilities. | |[.filename]#shells# |Command line shells. | |[.filename]#spanish#`*` |Spanish language support. | |[.filename]#sysutils# |System utilities. | |[.filename]#tcl#`*` |Ports that use Tcl to run. | |[.filename]#textproc# |Text processing utilities. |It does not include desktop publishing tools, which go to [.filename]#print#. |[.filename]#tk#`*` |Ports that use Tk to run. | |[.filename]#ukrainian# |Ukrainian language support. | |[.filename]#vietnamese# |Vietnamese language support. | |[.filename]#wayland#`*` |Ports to support the Wayland display server. | |[.filename]#windowmaker#`*` |Ports to support the Window Maker window manager. | |[.filename]#www# |Software related to the World Wide Web. |HTML language support belongs here too. |[.filename]#x11# |The X Window System and friends. |This category is only for software that directly supports the window system. Do not put regular X applications here. Most of them go into other [.filename]#x11-*# categories (see below). |[.filename]#x11-clocks# |X11 clocks. | |[.filename]#x11-drivers# |X11 drivers. | |[.filename]#x11-fm# |X11 file managers. | |[.filename]#x11-fonts# |X11 fonts and font utilities. | |[.filename]#x11-servers# |X11 servers. | |[.filename]#x11-themes# |X11 themes. | |[.filename]#x11-toolkits# |X11 toolkits. | |[.filename]#x11-wm# |X11 window managers. | |[.filename]#xfce#`*` |Ports related to the https://www.xfce.org/[Xfce] desktop environment. | |[.filename]#zope#`*` |https://www.zope.org/[Zope] support. | |=== [[choosing-categories]] === Choosing the Right Category As many of the categories overlap, choosing which of the categories will be the primary category of the port can be tedious. There are several rules that govern this issue. Here is the list of priorities, in decreasing order of precedence: * The first category must be a physical category (see crossref:makefiles[porting-categories,above]). This is necessary to make the packaging work. Virtual categories and physical categories may be intermixed after that. * Language specific categories always come first. For example, if the port installs Japanese X11 fonts, then the `CATEGORIES` line would read [.filename]#japanese x11-fonts#. * Specific categories are listed before less-specific ones. For instance, an HTML editor is listed as [.filename]#www editors#, not the other way around. Also, do not list [.filename]#net# when the port belongs to any of [.filename]#irc#, [.filename]#mail#, [.filename]#news#, [.filename]#security#, or [.filename]#www#, as [.filename]#net# is included implicitly. * [.filename]#x11# is used as a secondary category only when the primary category is a natural language. In particular, do not put [.filename]#x11# in the category line for X applications. * Emacs modes are placed in the same ports category as the application supported by the mode, not in [.filename]#editors#. For example, an Emacs mode to edit source files of some programming language goes into [.filename]#lang#. * Ports installing loadable kernel modules also have the virtual category [.filename]#kld# in their `CATEGORIES` line. This is one of the things handled automatically by adding `USES=kmod`. * [.filename]#misc# does not appear with any other non-virtual category. If there is `misc` with something else in `CATEGORIES`, that means `misc` can safely be deleted and the port placed only in the other subdirectory. * If the port truly does not belong anywhere else, put it in [.filename]#misc#. If the category is not clearly defined, please put a comment to that effect in the https://bugs.freebsd.org/submit/[port submission] in the bug database so we can discuss it before we import it. As a committer, send a note to the {freebsd-ports} so we can discuss it first. Too often, new ports are imported to the wrong category only to be moved right away. [[proposing-categories]] === Proposing a New Category As the Ports Collection has grown over time, various new categories have been introduced. New categories can either be _virtual_ categories-those that do not have a corresponding subdirectory in the ports tree- or _physical_ categories-those that do. This section discusses the issues involved in creating a new physical category. Read it thoroughly before proposing a new one. Our existing practice has been to avoid creating a new physical category unless either a large number of ports would logically belong to it, or the ports that would belong to it are a logically distinct group that is of limited general interest (for instance, categories related to spoken human languages), or preferably both. The rationale for this is that such a change creates a extref:{committers-guide}[fair amount of work, ports] for both the committers and also for all users who track changes to the Ports Collection. In addition, proposed category changes just naturally seem to attract controversy. (Perhaps this is because there is no clear consensus on when a category is "too big", nor whether categories should lend themselves to browsing (and thus what number of categories would be an ideal number), and so forth.) Here is the procedure: [.procedure] . Propose the new category on {freebsd-ports}. Include a detailed rationale for the new category, including why the existing categories are not sufficient, and the list of existing ports proposed to move. (If there are new ports pending in Bugzilla that would fit this category, list them too.) Indicating that the updater is also the maintainer or submitter may be helpful to the case. . Participate in the discussion. . If it seems that there is support for the idea, file a PR which includes both the rationale and the list of existing ports that need to be moved. Ideally, this PR would also include these patches: ** [.filename]##Makefile##s for the new ports once they are repocopied ** [.filename]#Makefile# for the new category ** [.filename]#Makefile# for the old ports' categories ** [.filename]##Makefile##s for ports that depend on the old ports ** (for extra credit, include the other files that have to change, as per the procedure in the Committer's Guide.) . Since it affects the ports infrastructure and involves moving and patching many ports but also possibly running regression tests on the build cluster, assign the PR to the {portmgr}. . If that PR is approved, a committer will need to follow the rest of the procedure that is extref:{committers-guide}[outlined in the Committer's Guide, ports]. Proposing a new virtual category is similar to the above but much less involved, since no ports will actually have to move. In this case, the only patches to include in the PR would be those to add the new category to `CATEGORIES` of the affected ports. [[proposing-reorg]] === Proposing Reorganizing All the Categories Occasionally someone proposes reorganizing the categories with either a 2-level structure, or some other kind of keyword structure. To date, nothing has come of any of these proposals because, while they are very easy to make, the effort involved to retrofit the entire existing ports collection with any kind of reorganization is daunting to say the very least. Please read the history of these proposals in the mailing list archives before posting this idea. Furthermore, be prepared to be challenged to offer a working prototype. [[makefile-distfiles]] == The Distribution Files The second part of the [.filename]#Makefile# describes the files that must be downloaded to build the port, and where they can be downloaded. [[makefile-distname]] === `DISTNAME` `DISTNAME` is the name of the port as called by the authors of the software. `DISTNAME` defaults to `${PORTNAME}-${DISTVERSIONPREFIX}${DISTVERSION}${DISTVERSIONSUFFIX}`, and if not set, `DISTVERSION` defaults to `${PORTVERSION}` so override `DISTNAME` only if necessary. `DISTNAME` is only used in two places. First, the distribution file list (`DISTFILES`) defaults to `${DISTNAME}${EXTRACT_SUFX}`. Second, the distribution file is expected to extract into a subdirectory named `WRKSRC`, which defaults to [.filename]#work/${DISTNAME}#. Some vendor's distribution names which do not fit into the `${PORTNAME}-${PORTVERSION}`-scheme can be handled automatically by setting `DISTVERSIONPREFIX`, `DISTVERSION`, and `DISTVERSIONSUFFIX`. `PORTVERSION` will be derived from `DISTVERSION` automatically. [IMPORTANT] ==== Only one of `PORTVERSION` and `DISTVERSION` can be set at a time. If `DISTVERSION` does not derive a correct `PORTVERSION`, do not use `DISTVERSION`. ==== If the upstream version scheme can be derived into a ports-compatible version scheme, set some variable to the upstream version, _do not_ use `DISTVERSION` as the variable name. Set `PORTVERSION` to the computed version based on the created variable and set `DISTNAME` accordingly. If the upstream version scheme cannot easily be coerced into a ports-compatible value, set `PORTVERSION` to a sensible value, and set `DISTNAME` with `PORTNAME` with the verbatim upstream version. [[makefile-distname-ex1]] .Deriving `PORTVERSION` Manually [example] ==== BIND9 uses a version scheme that is not compatible with the ports versions (it has `-` in its versions) and cannot be derived using `DISTVERSION` because after the 9.9.9 release, it will release a "patchlevels" in the form of `9.9.9-P1`. DISTVERSION would translate that into `9.9.9.p1`, which, in the ports versioning scheme means 9.9.9 pre-release 1, which is before 9.9.9 and not after. So `PORTVERSION` is manually derived from an `ISCVERSION` variable to output `9.9.9p1`. The order into which the ports framework, and pkg, will sort versions is checked using the `-t` argument of man:pkg-version[8]: [source,shell] .... % pkg version -t 9.9.9 9.9.9.p1 > <.> % pkg version -t 9.9.9 9.9.9p1 < <.> .... <.> The `>` sign means that the first argument passed to `-t` is greater than the second argument. `9.9.9` is after `9.9.9.p1`. <.> The `<` sign means that the first argument passed to `-t` is less than the second argument. `9.9.9` is before `9.9.9p1`. In the port [.filename]#Makefile#, for example package:dns/bind99[], it is achieved by: [.programlisting] .... PORTNAME= bind PORTVERSION= ${ISCVERSION:S/-P/P/:S/b/.b/:S/a/.a/:S/rc/.rc/} CATEGORIES= dns net MASTER_SITES= ISC/bind9/${ISCVERSION} PKGNAMESUFFIX= 99 DISTNAME= ${PORTNAME}-${ISCVERSION} MAINTAINER= mat@FreeBSD.org COMMENT= BIND DNS suite with updated DNSSEC and DNS64 WWW= https://www.isc.org/bind/ LICENSE= ISCL # ISC releases things like 9.8.0-P1 or 9.8.1rc1, which our versioning does not like ISCVERSION= 9.9.9-P6 .... Define upstream version in `ISCVERSION`, with a comment saying _why_ it is needed. Use `ISCVERSION` to get a ports-compatible `PORTVERSION`. Use `ISCVERSION` directly to get the correct URL for fetching the distribution file. Use `ISCVERSION` directly to name the distribution file. ==== [[makefile-distname-ex2]] .Derive `DISTNAME` from `PORTVERSION` [example] ==== From time to time, the distribution file name has little or no relation to the version of the software. In package:comms/kermit[], only the last element of the version is present in the distribution file: [.programlisting] .... PORTNAME= kermit PORTVERSION= 9.0.304 CATEGORIES= comms ftp net MASTER_SITES= ftp://ftp.kermitproject.org/kermit/test/tar/ DISTNAME= cku${PORTVERSION:E}-dev20 .... The `:E` man:make[1] modifier returns the suffix of the variable, in this case, `304`. The distribution file is correctly generated as `cku304-dev20.tar.gz`. ==== [[makefile-distname-ex3]] .Exotic Case 1 [example] ==== Sometimes, there is no relation between the software name, its version, and the distribution file it is distributed in. From package:audio/libworkman[]: [.programlisting] .... PORTNAME= libworkman PORTVERSION= 1.4 CATEGORIES= audio MASTER_SITES= LOCAL/jim DISTNAME= ${PORTNAME}-1999-06-20 .... ==== [[makefile-distname-ex4]] .Exotic Case 2 [example] ==== In package:comms/librs232[], the distribution file is not versioned, so using crossref:makefiles[makefile-dist_subdir,`DIST_SUBDIR`] is needed: [.programlisting] .... PORTNAME= librs232 PORTVERSION= 20160710 CATEGORIES= comms MASTER_SITES= http://www.teuniz.net/RS-232/ DISTNAME= RS-232 DIST_SUBDIR= ${PORTNAME}-${PORTVERSION} .... ==== [NOTE] ==== `PKGNAMEPREFIX` and `PKGNAMESUFFIX` do not affect `DISTNAME`. Also note that if `WRKSRC` is equal to [.filename]#${WRKDIR}/${DISTNAME}# while the original source archive is named something other than `${PORTNAME}-${PORTVERSION}${EXTRACT_SUFX}`, leave `DISTNAME` alone- defining only `DISTFILES` is easier than both `DISTNAME` and `WRKSRC` (and possibly `EXTRACT_SUFX`). ==== [[makefile-master_sites]] === `MASTER_SITES` Record the directory part of the FTP/HTTP-URL pointing at the original tarball in `MASTER_SITES`. Do not forget the trailing slash ([.filename]#/#)! The `make` macros will try to use this specification for grabbing the distribution file with `FETCH` if they cannot find it already on the system. It is recommended that multiple sites are included on this list, preferably from different continents. This will safeguard against wide-area network problems. [IMPORTANT] ==== `MASTER_SITES` must not be blank. It must point to the actual site hosting the distribution files. It cannot point to web archives, or the FreeBSD distribution files cache sites. The only exception to this rule is ports that do not have any distribution files. For example, meta-ports do not have any distribution files, so `MASTER_SITES` does not need to be set. ==== [[makefile-master_sites-shorthand]] ==== Using `MASTER_SITE_*` Variables Shortcut abbreviations are available for popular archives like SourceForge (`SOURCEFORGE`), GNU (`GNU`), or Perl CPAN (`PERL_CPAN`). `MASTER_SITES` can use them directly: [.programlisting] .... MASTER_SITES= GNU/make .... The older expanded format still works, but all ports have been converted to the compact format. The expanded format looks like this: [.programlisting] .... MASTER_SITES= ${MASTER_SITE_GNU} MASTER_SITE_SUBDIR= make .... These values and variables are defined in https://cgit.freebsd.org/ports/tree/Mk/bsd.sites.mk[Mk/bsd.sites.mk]. New entries are added often, so make sure to check the latest version of this file before submitting a port. [TIP] ==== For any `MASTER_SITE_FOO` variable, the shorthand `_FOO_` can be used. For example, use: [.programlisting] .... MASTER_SITES= FOO .... If `MASTER_SITE_SUBDIR` is needed, use this: [.programlisting] .... MASTER_SITES= FOO/bar .... ==== [NOTE] ==== Some `MASTER_SITE_*` names are quite long, and for ease of use, shortcuts have been defined: [[makefile-master_sites-shortcut]] .Shortcuts for `MASTER_SITE_*` Macros [cols="1,1", frame="none", options="header"] |=== | Macro | Shortcut |`PERL_CPAN` |`CPAN` |`GITHUB` |`GH` |`GITHUB_CLOUD` |`GHC` |`LIBREOFFICE_DEV` |`LODEV` |`NETLIB` |`NL` |`RUBYGEMS` |`RG` |`SOURCEFORGE` |`SF` |=== ==== [[makefile-master_sites-magic]] ==== Magic MASTER_SITES Macros Several "magic" macros exist for popular sites with a predictable directory structure. For these, just use the abbreviation and the system will choose a subdirectory automatically. For a port named `Stardict`, of version `1.2.3`, and hosted on SourceForge, adding this line: [.programlisting] .... MASTER_SITES= SF .... infers a subdirectory named `/project/stardict/stardict/1.2.3`. If the inferred directory is incorrect, it can be overridden: [.programlisting] .... MASTER_SITES= SF/stardict/WyabdcRealPeopleTTS/${PORTVERSION} .... This can also be written as [.programlisting] .... MASTER_SITES= SF MASTER_SITE_SUBDIR= stardict/WyabdcRealPeopleTTS/${PORTVERSION} .... [[makefile-master_sites-popular]] .Magic `MASTER_SITES` Macros [cols="1,1", frame="none", options="header"] |=== | Macro | Assumed subdirectory |`APACHE_COMMONS_BINARIES` |`${PORTNAME:S,commons-,,}` |`APACHE_COMMONS_SOURCE` |`${PORTNAME:S,commons-,,}` |`APACHE_JAKARTA` |`${PORTNAME:S,-,/,}/source` |`BERLIOS` |`${PORTNAME:tl}.berlios` |`PYPI` |`source/${DISTNAME:C/(.).\*/\1/}/${DISTNAME:C/(.*)-[0-9].*/\1/}` |`CPAN` |`${PORTNAME:C/-.*//}` |`DEBIAN` |`pool/main/${PORTNAME:C/^((lib)?.).*$/\1/}/${PORTNAME}` |`FARSIGHT` |`${PORTNAME}` |`FESTIVAL` |`${PORTREVISION}` |`GCC` |`releases/${DISTNAME}` |`GENTOO` |`distfiles` |`GIMP` |`${PORTNAME}/${PORTVERSION:R}/` |`GH` |`${GH_ACCOUNT}/${GH_PROJECT}/tar.gz/${GH_TAGNAME}?dummy=/` |`GHC` |`${GH_ACCOUNT}/${GH_PROJECT}/` |`GNOME` |`sources/${PORTNAME}/${PORTVERSION:C/^([0-9]+\.[0-9]+).*/\1/}` |`GNU` |`${PORTNAME}` |`GNUPG` |`${PORTNAME}` |`GNU_ALPHA` |`${PORTNAME}` |`HORDE` |`${PORTNAME}` |`LODEV` |`${PORTNAME}` |`MATE` |`${PORTVERSION:C/^([0-9]+\.[0-9]+).*/\1/}` |`MOZDEV` |`${PORTNAME:tl}` |`NL` |`${PORTNAME}` |`QT` |`archive/qt/${PORTVERSION:R}` |`SAMBA` |`${PORTNAME}` |`SAVANNAH` |`${PORTNAME:tl}` |`SF` |`${PORTNAME:tl}/${PORTNAME:tl}/${PORTVERSION}` |=== [[makefile-master_sites-github]] === `USE_GITHUB` If the distribution file comes from a specific commit or tag on https://github.com/[GitHub] for which there is no officially released file, there is an easy way to set the right `DISTNAME` and `MASTER_SITES` automatically. [WARNING] ==== As of 2023-02-21 link:https://github.blog/2023-02-21-update-on-the-future-stability-of-source-code-archives-and-hashes/[GitHub] have announced that source downloads will be stable for a year. Please switch to release assets and if not available ask upstream to generate ones. ==== These variables are available: [[makefile-master_sites-github-description]] .`USE_GITHUB` Description [cols="1,1,1", options="header"] |=== | Variable | Description | Default |`GH_ACCOUNT` |Account name of the GitHub user hosting the project |`${PORTNAME}` |`GH_PROJECT` |Name of the project on GitHub |`${PORTNAME}` |`GH_TAGNAME` |Name of the tag to download (2.0.1, hash, ...) Using the name of a branch here is incorrect. It is also possible to use the hash of a commit id to do a snapshot. |`${DISTVERSIONPREFIX}${DISTVERSION}${DISTVERSIONSUFFIX}` |`GH_SUBDIR` |When the software needs an additional distribution file to be extracted within `${WRKSRC}`, this variable can be used. See the examples in crossref:makefiles[makefile-master_sites-github-multiple, Fetching Multiple Files from GitHub] for more information. |(none) |`GH_TUPLE` |`GH_TUPLE` allows putting `GH_ACCOUNT`, `GH_PROJECT`, `GH_TAGNAME`, and `GH_SUBDIR` into a single variable. The format is _account_`:`_project_`:`_tagname_`:`_group_`/`_subdir_. The `/`_subdir_ part is optional. It is helpful when there is more than one GitHub project from which to fetch. | |=== [IMPORTANT] ==== Do not use `GH_TUPLE` for the default distribution file, as it has no default. ==== [[makefile-master_sites-github-ex1]] .Simple Use of `USE_GITHUB` [example] ==== While trying to make a port for version `1.2.7` of pkg from the FreeBSD user on github, at https://github.com/freebsd/pkg/[], The [.filename]#Makefile# would end up looking like this (slightly stripped for the example): [.programlisting] .... PORTNAME= pkg DISTVERSION= 1.2.7 USE_GITHUB= yes GH_ACCOUNT= freebsd .... It will automatically have `MASTER_SITES` set to `GH` and `WRKSRC` to `${WRKDIR}/pkg-1.2.7`. ==== [[makefile-master_sites-github-ex2]] .More Complete Use of `USE_GITHUB` [example] ==== While trying to make a port for the bleeding edge version of pkg from the FreeBSD user on github, at https://github.com/freebsd/pkg/[], the [.filename]#Makefile# ends up looking like this (slightly stripped for the example): [.programlisting] .... PORTNAME= pkg-devel DISTVERSION= 2.7.4.20260626 USE_GITHUB= yes GH_ACCOUNT= freebsd GH_PROJECT= pkg GH_TAGNAME= 2678d2b6a8ca3cf80cb4dbc8da557a2998e1b5c0 .... It will automatically have `MASTER_SITES` set to `GH` and `WRKSRC` to `${WRKDIR}/pkg-6dbb17b`. [TIP] ==== `20260626` is the date of the commit referenced in `GH_TAGNAME`, not the date the [.filename]#Makefile# is edited, or the date the commit is made. ==== ==== [[makefile-master_sites-github-ex3]] .Use of `USE_GITHUB` with `DISTVERSIONPREFIX` [example] ==== From time to time, `GH_TAGNAME` is a slight variation from `DISTVERSION`. For example, if the version is `1.0.2`, the tag is `v1.0.2`. In those cases, it is possible to use `DISTVERSIONPREFIX` or `DISTVERSIONSUFFIX`: [.programlisting] .... PORTNAME= foo DISTVERSIONPREFIX= v DISTVERSION= 1.0.2 USE_GITHUB= yes .... It will automatically set `GH_TAGNAME` to `v1.0.2`, while `WRKSRC` will be kept to `${WRKDIR}/foo-1.0.2`. ==== [[makefile-master_sites-github-ex4]] .Using `USE_GITHUB` When Upstream Does Not Use Versions [example] ==== If there never was a version upstream, do not invent one like `0.1` or `1.0`. Create the port with a `DISTVERSION` of `g__YYYYMMDD__`, where `g` is for Git, and `_YYYYMMDD_` represents the date the commit referenced in `GH_TAGNAME`. [.programlisting] .... PORTNAME= bar DISTVERSION= g20260626 USE_GITHUB= yes GH_TAGNAME= c472d66b70dd603bf9e67607f0869639276796ce .... This creates a versioning scheme that increases over time, and that is still before version `0`. See crossref:makefiles[makefile-versions-ex-pkg-version, this secion on how to compare versions] using man:pkg-version[8]: [source,shell] .... % pkg version -t g20260626 0 < .... Which means using `PORTEPOCH` will not be needed in case upstream decides to cut versions in the future. ==== [[makefile-master_sites-github-ex5]] .Using `USE_GITHUB` to Access a Commit Between Two Versions [example] ==== If the current version of the software uses a Git tag, and the port needs to be updated to a newer, intermediate version, without a tag, use man:git-describe[1] to find out the version to use: [source,shell] .... % git describe --tags f0038b1 v0.7.3-14-gf0038b1 .... `v0.7.3-14-gf0038b1` can be split into three parts: `v0.7.3`:: This is the last Git tag that appears in the commit history before the requested commit. `-14`:: This means that the requested commit, `f0038b1`, is the 14th commit after the `v0.7.3` tag. `-gf0038b1`:: The `-g` means "Git", and the `f0038b1` is the commit hash that this reference points to. [.programlisting] .... PORTNAME= bar DISTVERSIONPREFIX= v DISTVERSION= 0.7.3-14 DISTVERSIONSUFFIX= -gf0038b1 USE_GITHUB= yes .... This creates a versioning scheme that increases over time (well, over commits), and does not conflict with the creation of a `0.7.4` version. See crossref:makefiles[makefile-versions-ex-pkg-version, this section for how to compare versions] using man:pkg-version[8]: [source,shell] .... % pkg version -t 0.7.3 0.7.3.14 < % pkg version -t 0.7.3.14 0.7.4 < .... [NOTE] ===== If the requested commit is the same as a tag, a shorter description is shown by default. The longer version is equivalent: [source,shell] .... % git describe --tags c66c71d v0.7.3 % git describe --tags --long c66c71d v0.7.3-0-gc66c71d .... ===== ==== [[makefile-master_sites-github-multiple]] ==== Fetching Multiple Files from GitHub The `USE_GITHUB` framework also supports fetching multiple distribution files from different places in GitHub. It works in a way very similar to crossref:makefiles[porting-master-sites-n, Multiple Distribution or Patches Files from Multiple Locations]. Multiple values are added to `GH_ACCOUNT`, `GH_PROJECT`, and `GH_TAGNAME`. Each different value is assigned a group. The main value can either have no group, or the `:DEFAULT` group. A value can be omitted if it is the same as the default as listed in crossref:makefiles[makefile-master_sites-github-description,`USE_GITHUB` Description]. `GH_TUPLE` can also be used when there are a lot of distribution files. It helps keep the account, project, tagname, and group information at the same place. For each group, a `${WRKSRC_group}` helper variable is created, containing the directory into which the file has been extracted. The `${WRKSRC_group}` variables can be used to move directories around during `post-extract`, or add to `CONFIGURE_ARGS`, or whatever is needed so that the software builds correctly. [CAUTION] ==== The `:__group__` part _must_ be used for _only one_ distribution file. It is used as a unique key and using it more than once will overwrite the previous values. ==== [NOTE] ==== As this is only syntactic sugar above `DISTFILES` and `MASTER_SITES`, the group names must adhere to the restrictions on group names outlined in crossref:makefiles[porting-master-sites-n, Multiple Distribution or Patches Files from Multiple Locations] ==== When fetching multiple files from GitHub, sometimes the default distribution file is not fetched from GitHub. To disable fetching the default distribution, set: [.programlisting] .... USE_GITHUB= nodefault .... [IMPORTANT] ==== When using `USE_GITHUB=nodefault`, the [.filename]#Makefile# must set `DISTFILES` in its crossref:porting-order[porting-order-portname,top block]. The definition should be: [.programlisting] .... DISTFILES= ${DISTNAME}${EXTRACT_SUFX} .... ==== [[makefile-master_sites-github-multi]] .Use of `USE_GITHUB` with Multiple Distribution Files [example] ==== From time to time, there is a need to fetch more than one distribution file. For example, when the upstream git repository uses submodules. This can be done easily using groups in the `GH_*` variables: [.programlisting] .... PORTNAME= foo DISTVERSION= 1.0.2 USE_GITHUB= yes GH_ACCOUNT= bar:icons,contrib GH_PROJECT= foo-icons:icons foo-contrib:contrib GH_TAGNAME= 1.0:icons fa579bc:contrib GH_SUBDIR= ext/icons:icons CONFIGURE_ARGS= --with-contrib=${WRKSRC_contrib} .... This will fetch three distribution files from github. The default one comes from [.filename]#foo/foo# and is version `1.0.2`. The second one, with the `icons` group, comes from [.filename]#bar/foo-icons# and is in version `1.0`. The third one comes from [.filename]#bar/foo-contrib# and uses the Git commit `fa579bc`. The distribution files are named [.filename]#foo-foo-1.0.2_GH0.tar.gz#, [.filename]#bar-foo-icons-1.0_GH0.tar.gz#, and [.filename]#bar-foo-contrib-fa579bc_GH0.tar.gz#. All the distribution files are extracted in `${WRKDIR}` in their respective subdirectories. The default file is still extracted in `${WRKSRC}`, in this case, [.filename]#${WRKDIR}/foo-1.0.2#. Each additional distribution file is extracted in `${WRKSRC_group}`. Here, for the `icons` group, it is called `${WRKSRC_icons}` and it contains [.filename]#${WRKDIR}/foo-icons-1.0#. The file with the `contrib` group is called `${WRKSRC_contrib}` and contains `${WRKDIR}/foo-contrib-fa579bc`. The software's build system expects to find the icons in a [.filename]#ext/icons# subdirectory in its sources, so `GH_SUBDIR` is used. `GH_SUBDIR` makes sure that [.filename]#ext# exists, but that [.filename]#ext/icons# does not already exist. Then it does this: [.programlisting] .... post-extract: @${MV} ${WRKSRC_icons} ${WRKSRC}/ext/icons .... ==== [[makefile-master_sites-github-multi2]] .Use of `USE_GITHUB` with Multiple Distribution Files Using `GH_TUPLE` [example] ==== This is functionally equivalent to crossref:makefiles[makefile-master_sites-github-multi,Use of `USE_GITHUB` with Multiple Distribution Files], but using `GH_TUPLE`: [.programlisting] .... PORTNAME= foo DISTVERSION= 1.0.2 USE_GITHUB= yes GH_TUPLE= bar:foo-icons:1.0:icons/ext/icons \ bar:foo-contrib:fa579bc:contrib CONFIGURE_ARGS= --with-contrib=${WRKSRC_contrib} .... Grouping was used in the previous example with `bar:icons,contrib`. Some redundant information is present with `GH_TUPLE` because grouping is not possible. ==== [[makefile-master_sites-github-submodules]] .How to Use `USE_GITHUB` with Git Submodules? [example] ==== Ports with GitHub as an upstream repository sometimes use submodules. See man:git-submodule[1] for more information. The problem with submodules is that each is a separate repository. As such, they each must be fetched separately. Using package:finance/moneymanagerex[] as an example, its GitHub repository is https://github.com/moneymanagerex/moneymanagerex/[]. It has a https://github.com/moneymanagerex/moneymanagerex/blob/master/.gitmodules[.gitmodules] file at the root. This file describes all the submodules used in this repository, and lists additional repositories needed. This file will tell what additional repositories are needed: [.programlisting] .... [submodule "lib/wxsqlite3"] path = lib/wxsqlite3 url = https://github.com/utelle/wxsqlite3.git [submodule "3rd/mongoose"] path = 3rd/mongoose url = https://github.com/cesanta/mongoose.git [submodule "3rd/LuaGlue"] path = 3rd/LuaGlue url = https://github.com/moneymanagerex/LuaGlue.git [submodule "3rd/cgitemplate"] path = 3rd/cgitemplate url = https://github.com/moneymanagerex/html-template.git [...] .... The only information missing from that file is the commit hash or tag to use as a version. This information is found after cloning the repository: [source,shell] .... % git clone --recurse-submodules https://github.com/moneymanagerex/moneymanagerex.git Cloning into 'moneymanagerex'... remote: Counting objects: 32387, done. [...] Submodule '3rd/LuaGlue' (https://github.com/moneymanagerex/LuaGlue.git) registered for path '3rd/LuaGlue' Submodule '3rd/cgitemplate' (https://github.com/moneymanagerex/html-template.git) registered for path '3rd/cgitemplate' Submodule '3rd/mongoose' (https://github.com/cesanta/mongoose.git) registered for path '3rd/mongoose' Submodule 'lib/wxsqlite3' (https://github.com/utelle/wxsqlite3.git) registered for path 'lib/wxsqlite3' [...] Cloning into '/home/mat/work/freebsd/ports/finance/moneymanagerex/moneymanagerex/3rd/LuaGlue'... Cloning into '/home/mat/work/freebsd/ports/finance/moneymanagerex/moneymanagerex/3rd/cgitemplate'... Cloning into '/home/mat/work/freebsd/ports/finance/moneymanagerex/moneymanagerex/3rd/mongoose'... Cloning into '/home/mat/work/freebsd/ports/finance/moneymanagerex/moneymanagerex/lib/wxsqlite3'... [...] Submodule path '3rd/LuaGlue': checked out 'c51d11a247ee4d1e9817dfa2a8da8d9e2f97ae3b' Submodule path '3rd/cgitemplate': checked out 'cd434eeeb35904ebcd3d718ba29c281a649b192c' Submodule path '3rd/mongoose': checked out '2140e5992ab9a3a9a34ce9a281abf57f00f95cda' Submodule path 'lib/wxsqlite3': checked out 'fb66eb230d8aed21dec273b38c7c054dcb7d6b51' [...] % cd moneymanagerex % git submodule status c51d11a247ee4d1e9817dfa2a8da8d9e2f97ae3b 3rd/LuaGlue (heads/master) cd434eeeb35904ebcd3d718ba29c281a649b192c 3rd/cgitemplate (cd434ee) 2140e5992ab9a3a9a34ce9a281abf57f00f95cda 3rd/mongoose (6.2-138-g2140e59) fb66eb230d8aed21dec273b38c7c054dcb7d6b51 lib/wxsqlite3 (v3.4.0) [...] .... It can also be found on GitHub. Each subdirectory that is a submodule is shown as `_directory @ hash_`, for example, `mongoose @ 2140e59`. [NOTE] ===== While getting the information from GitHub seems more straightforward, the information found using `git submodule status` will provide more meaningful information. For example, here, ``lib/wxsqlite3``'s commit hash `fb66eb2` correspond to `v3.4.0`. Both can be used interchangeably, but when a tag is available, use it. ===== Now that all the required information has been gathered, the [.filename]#Makefile# can be written (only GitHub-related lines are shown): [.programlisting] .... PORTNAME= moneymanagerex DISTVERSIONPREFIX= v DISTVERSION= 1.3.0 USE_GITHUB= yes GH_TUPLE= utelle:wxsqlite3:v3.4.0:wxsqlite3/lib/wxsqlite3 \ moneymanagerex:LuaGlue:c51d11a:lua_glue/3rd/LuaGlue \ moneymanagerex:html-template:cd434ee:html_template/3rd/cgitemplate \ cesanta:mongoose:2140e59:mongoose/3rd/mongoose \ [...] .... ==== [[makefile-master_sites-gitlab]] === `USE_GITLAB` Similar to GitHub, if the distribution file comes from https://gitlab.com/[gitlab.com] or is hosting the GitLab software, these variables are available for use and might need to be set. [[makefile-master_sites-gitlab-description]] .`USE_GITLAB` Description [cols="1,1,1", options="header"] |=== | Variable | Description | Default |`GL_SITE` |Site name hosting the GitLab project |https://gitlab.com/ |`GL_ACCOUNT` |Account name of the GitLab user hosting the project |`${PORTNAME}` |`GL_PROJECT` |Name of the project on GitLab |`${PORTNAME}` |`GL_COMMIT` |The commit hash to download. Must be the full 160 bit, 40 character hex sha1 hash. This is a required variable for GitLab. |`(none)` |`GL_SUBDIR` |When the software needs an additional distribution file to be extracted within `${WRKSRC}`, this variable can be used. See the examples in crossref:makefiles[makefile-master_sites-gitlab-multiple, Fetching Multiple Files from GitLab] for more information. |(none) |`GL_TUPLE` |`GL_TUPLE` allows putting `GL_SITE`, `GL_ACCOUNT`, `GL_PROJECT`, `GL_COMMIT`, and `GL_SUBDIR` into a single variable. The format is _site_`:`_account_`:`_project_`:`_commit_`:`_group_`/`_subdir_. The _site_`:` and `/`_subdir_ part is optional. It is helpful when there are more than one GitLab project from which to fetch. | |=== [[makefile-master_sites-gitlab-ex1]] .Simple Use of `USE_GITLAB` [example] ==== While trying to make a port for version `1.14` of libsignon-glib from the accounts-sso user on gitlab.com, at https://gitlab.com/accounts-sso/libsignon-glib/[], The [.filename]#Makefile# would end up looking like this for fetching the distribution files: [.programlisting] .... PORTNAME= libsignon-glib DISTVERSION= 1.14 USE_GITLAB= yes GL_ACCOUNT= accounts-sso GL_COMMIT= e90302e342bfd27bc8c9132ab9d0ea3d8723fd03 .... It will automatically have `MASTER_SITES` set to https://gitlab.com/[gitlab.com] and `WRKSRC` to `${WRKDIR}/libsignon-glib-e90302e342bfd27bc8c9132ab9d0ea3d8723fd03-e90302e342bfd27bc8c9132ab9d0ea3d8723fd03`. ==== [[makefile-master_sites-gitlab-ex2]] .More Complete Use of `USE_GITLAB` [example] ==== A more complete use of the above if port had no versioning and foobar from the foo user on project bar on a self hosted GitLab site `https://gitlab.example.com/`, the [.filename]#Makefile# ends up looking like this for fetching distribution files: [.programlisting] .... PORTNAME= foobar DISTVERSION= g20170906 USE_GITLAB= yes GL_SITE= https://gitlab.example.com GL_ACCOUNT= foo GL_PROJECT= bar GL_COMMIT= 9c1669ce60c3f4f5eb43df874d7314483fb3f8a6 .... It will have `MASTER_SITES` set to `"https://gitlab.example.com"` and `WRKSRC` to `${WRKDIR}/bar-9c1669ce60c3f4f5eb43df874d7314483fb3f8a6-9c1669ce60c3f4f5eb43df874d7314483fb3f8a6`. [TIP] ====== `20170906` is the date of the commit referenced in `GL_COMMIT`, not the date the [.filename]#Makefile# is edited, or the date the commit to the FreeBSD ports tree is made. ====== [NOTE] ====== ``GL_SITE``'s protocol, port and webroot can all be modified in the same variable. ====== ==== [[makefile-master_sites-gitlab-multiple]] ==== Fetching Multiple Files from GitLab The `USE_GITLAB` framework also supports fetching multiple distribution files from different places from GitLab and GitLab hosted sites. It works in a way very similar to crossref:makefiles[porting-master-sites-n, Multiple Distribution or Patches Files from Multiple Locations] and crossref:makefiles[makefile-master_sites-github-multiple, Fetching Multiple Files from GitHub]. Multiple values are added to `GL_SITE`, `GL_ACCOUNT`, `GL_PROJECT` and `GL_COMMIT`. Each different value is assigned a group. crossref:makefiles[makefile-master_sites-gitlab-description,`USE_GITLAB` Description]. `GL_TUPLE` can also be used when there are a lot of distribution files. It helps keep the site, account, project, commit, and group information at the same place. For each group, a `${WRKSRC_group}` helper variable is created, containing the directory into which the file has been extracted. The `${WRKSRC_group}` variables can be used to move directories around during `post-extract`, or add to `CONFIGURE_ARGS`, or whatever is needed so that the software builds correctly. [CAUTION] ==== The `:__group__` part _must_ be used for _only one_ distribution file. It is used as a unique key and using it more than once will overwrite the previous values. ==== [NOTE] ==== As this is only syntactic sugar above `DISTFILES` and `MASTER_SITES`, the group names must adhere to the restrictions on group names outlined in crossref:makefiles[porting-master-sites-n, Multiple Distribution or Patches Files from Multiple Locations] ==== When fetching multiple files using GitLab, sometimes the default distribution file is not fetched from a GitLab site. To disable fetching the default distribution, set: [.programlisting] .... USE_GITLAB= nodefault .... [IMPORTANT] ==== When using `USE_GITLAB=nodefault`, the [.filename]#Makefile# must set `DISTFILES` in its crossref:makefiles[porting-order-portname,top block]. The definition should be: [.programlisting] .... DISTFILES= ${DISTNAME}${EXTRACT_SUFX} .... ==== [[makefile-master_sites-gitlab-multi]] .Use of `USE_GITLAB` with Multiple Distribution Files [example] ==== From time to time, there is a need to fetch more than one distribution file. For example, when the upstream git repository uses submodules. This can be done easily using groups in the `GL_*` variables: [.programlisting] .... PORTNAME= foo DISTVERSION= 1.0.2 USE_GITLAB= yes GL_SITE= https://gitlab.example.com:9434/gitlab:icons GL_ACCOUNT= bar:icons,contrib GL_PROJECT= foo-icons:icons foo-contrib:contrib GL_COMMIT= c189207a55da45305c884fe2b50e086fcad4724b ae7368cab1ca7ca754b38d49da064df87968ffe4:icons 9e4dd76ad9b38f33fdb417a4c01935958d5acd2a:contrib GL_SUBDIR= ext/icons:icons CONFIGURE_ARGS= --with-contrib=${WRKSRC_contrib} .... This will fetch two distribution files from gitlab.com and one from `gitlab.example.com` hosting GitLab. The default one comes from [.filename]#https://gitlab.com/foo/foo# and commit is `c189207a55da45305c884fe2b50e086fcad4724b`. The second one, with the `icons` group, comes from [.filename]#https://gitlab.example.com:9434/gitlab/bar/foo-icons# and commit is `ae7368cab1ca7ca754b38d49da064df87968ffe4`. The third one comes from [.filename]#https://gitlab.com/bar/foo-contrib# and is commit `9e4dd76ad9b38f33fdb417a4c01935958d5acd2a`. The distribution files are named [.filename]#foo-foo-c189207a55da45305c884fe2b50e086fcad4724b_GL0.tar.gz#, [.filename]#bar-foo-icons-ae7368cab1ca7ca754b38d49da064df87968ffe4_GL0.tar.gz#, and [.filename]#bar-foo-contrib-9e4dd76ad9b38f33fdb417a4c01935958d5acd2a_GL0.tar.gz#. All the distribution files are extracted in `${WRKDIR}` in their respective subdirectories. The default file is still extracted in `${WRKSRC}`, in this case, [.filename]#${WRKDIR}/foo-c189207a55da45305c884fe2b50e086fcad4724b-c189207a55da45305c884fe2b50e086fcad4724b#. Each additional distribution file is extracted in `${WRKSRC_group}`. Here, for the `icons` group, it is called `${WRKSRC_icons}` and it contains [.filename]#${WRKDIR}/foo-icons-ae7368cab1ca7ca754b38d49da064df87968ffe4-ae7368cab1ca7ca754b38d49da064df87968ffe4#. The file with the `contrib` group is called `${WRKSRC_contrib}` and contains `${WRKDIR}/foo-contrib-9e4dd76ad9b38f33fdb417a4c01935958d5acd2a-9e4dd76ad9b38f33fdb417a4c01935958d5acd2a`. The software's build system expects to find the icons in a [.filename]#ext/icons# subdirectory in its sources, so `GL_SUBDIR` is used. `GL_SUBDIR` makes sure that [.filename]#ext# exists, but that [.filename]#ext/icons# does not already exist. Then it does this: [.programlisting] .... post-extract: @${MV} ${WRKSRC_icons} ${WRKSRC}/ext/icons .... ==== [[makefile-master_sites-gitlab-multi2]] .Use of `USE_GITLAB` with Multiple Distribution Files Using `GL_TUPLE` [example] ==== This is functionally equivalent to crossref:makefiles[makefile-master_sites-gitlab-multi,Use of `USE_GITLAB` with Multiple Distribution Files], but using `GL_TUPLE`: [.programlisting] .... PORTNAME= foo DISTVERSION= 1.0.2 USE_GITLAB= yes GL_COMMIT= c189207a55da45305c884fe2b50e086fcad4724b GL_TUPLE= https://gitlab.example.com:9434/gitlab:bar:foo-icons:ae7368cab1ca7ca754b38d49da064df87968ffe4:icons/ext/icons \ bar:foo-contrib:9e4dd76ad9b38f33fdb417a4c01935958d5acd2a:contrib CONFIGURE_ARGS= --with-contrib=${WRKSRC_contrib} .... Grouping was used in the previous example with `bar:icons,contrib`. Some redundant information is present with `GL_TUPLE` because grouping is not possible. ==== [[makefile-extract_sufx]] === `EXTRACT_SUFX` If there is one distribution file, and it uses an odd suffix to indicate the compression mechanism, set `EXTRACT_SUFX`. For example, if the distribution file was named [.filename]#foo.tar.gzip# instead of the more normal [.filename]#foo.tar.gz#, write: [.programlisting] .... DISTNAME= foo EXTRACT_SUFX= .tar.gzip .... The `USES=tar[:__xxx__]`, `USES=lha` or `USES=zip` automatically set `EXTRACT_SUFX` to the most common archives extensions as necessary, see crossref:uses[uses,Using `USES` Macros] for more details. If neither of these are set then `EXTRACT_SUFX` defaults to `.tar.gz`. [NOTE] ==== As `EXTRACT_SUFX` is only used in `DISTFILES`, only set one of them.. ==== [[makefile-distfiles-definition]] === `DISTFILES` Sometimes the names of the files to be downloaded have no resemblance to the name of the port. For example, it might be called [.filename]#source.tar.gz# or similar. In other cases the application's source code might be in several different archives, all of which must be downloaded. If this is the case, set `DISTFILES` to be a space separated list of all the files that must be downloaded. [.programlisting] .... DISTFILES= source1.tar.gz source2.tar.gz .... If not explicitly set, `DISTFILES` defaults to `${DISTNAME}${EXTRACT_SUFX}`. [[makefile-extract_only]] === `EXTRACT_ONLY` If only some of the `DISTFILES` must be extracted-for example, one of them is the source code, while another is an uncompressed document-list the filenames that must be extracted in `EXTRACT_ONLY`. [.programlisting] .... DISTFILES= source.tar.gz manual.html EXTRACT_ONLY= source.tar.gz .... When none of the `DISTFILES` need to be uncompressed, set `EXTRACT_ONLY` to the empty string. [.programlisting] .... EXTRACT_ONLY= .... [[porting-patchfiles]] === `PATCHFILES` If the port requires some additional patches that are available by FTP or HTTP, set `PATCHFILES` to the names of the files and `PATCH_SITES` to the URL of the directory that contains them (the format is the same as `MASTER_SITES`). If the patch is not relative to the top of the source tree (that is, `WRKSRC`) because it contains some extra pathnames, set `PATCH_DIST_STRIP` accordingly. For instance, if all the pathnames in the patch have an extra `foozolix-1.0/` in front of the filenames, then set `PATCH_DIST_STRIP=-p1`. Do not worry if the patches are compressed; they will be decompressed automatically if the filenames end with [.filename]#.Z#, [.filename]#.gz#, [.filename]#.bz2# or [.filename]#.xz#. If the patch is distributed with some other files, such as documentation, in a compressed tarball, using `PATCHFILES` is not possible. If that is the case, add the name and the location of the patch tarball to `DISTFILES` and `MASTER_SITES`. Then, use `EXTRA_PATCHES` to point to those files and [.filename]#bsd.port.mk# will automatically apply them. In particular, do _not_ copy patch files into [.filename]#${PATCHDIR}#. That directory may not be writable. [TIP] ==== If there are multiple patches and they need mixed values for the strip parameter, it can be added alongside the patch name in `PATCHFILES`, e.g: [.programlisting] .... PATCHFILES= patch1 patch2:-p1 .... This does not conflict with crossref:makefiles[porting-master-sites-n,the master site grouping feature], adding a group also works: [.programlisting] .... PATCHFILES= patch2:-p1:source2 .... ==== [NOTE] ==== The tarball will have been extracted alongside the regular source by then, so there is no need to explicitly extract it if it is a regular compressed tarball. Take extra care not to overwrite something that already exists in that directory if extracting it manually. Also, do not forget to add a command to remove the copied patch in the `pre-clean` target. ==== [[porting-master-sites-n]] === Multiple Distribution or Patches Files from Multiple Locations (Consider this to be a somewhat "advanced topic"; those new to this document may wish to skip this section at first). This section has information on the fetching mechanism known as both `MASTER_SITES:n` and `MASTER_SITES_NN`. We will refer to this mechanism as `MASTER_SITES:n`. A little background first. OpenBSD has a neat feature inside `DISTFILES` and `PATCHFILES` which allows files and patches to be postfixed with `:n` identifiers. Here, `n` can be any word containing `[0-9a-zA-Z_]` and denote a group designation. For example: [.programlisting] .... DISTFILES= alpha:0 beta:1 .... In OpenBSD, distribution file [.filename]#alpha# will be associated with variable `MASTER_SITES0` instead of our common `MASTER_SITES` and [.filename]#beta# with `MASTER_SITES1`. This is a very interesting feature which can decrease that endless search for the correct download site. Just picture 2 files in `DISTFILES` and 20 sites in `MASTER_SITES`, the sites slow as hell where [.filename]#beta# is carried by all sites in `MASTER_SITES`, and [.filename]#alpha# can only be found in the 20th site. It would be such a waste to check all of them if the maintainer knew this beforehand, would it not? Not a good start for that lovely weekend! Once the concept is clear, just imagine more `DISTFILES` and more `MASTER_SITES`. Surely our "distfiles survey meister" would appreciate the relief to network strain that this would bring. In the next sections, information will follow on the FreeBSD implementation of this idea. We improved a bit on OpenBSD's concept. [IMPORTANT] ==== The group names cannot have dashes in them (`-`), in fact, they cannot have any characters out of the `[a-zA-Z0-9_]` range. This is because, while man:make[1] is ok with variable names containing dashes, man:sh[1] is not. ==== [[porting-master-sites-n-simplified]] ==== Simplified Information This section explains how to quickly prepare fine grained fetching of multiple distribution files and patches from different sites and subdirectories. We describe here a case of simplified `MASTER_SITES:n` usage. This will be sufficient for most scenarios. More detailed information are available in crossref:makefiles[ports-master-sites-n-detailed, Detailed Information]. Some applications consist of multiple distribution files that must be downloaded from a number of different sites. For example, Ghostscript consists of the core of the program, and then a large number of driver files that are used depending on the user's printer. Some of these driver files are supplied with the core, but many others must be downloaded from a variety of different sites. To support this, each entry in `DISTFILES` may be followed by a colon and a "group name". Each site listed in `MASTER_SITES` is then followed by a colon, and the group that indicates which distribution files are downloaded from this site. For example, consider an application with the source split in two parts, [.filename]#source1.tar.gz# and [.filename]#source2.tar.gz#, which must be downloaded from two different sites. The port's [.filename]#Makefile# would include lines like crossref:makefiles[ports-master-sites-n-example-simple-use-one-file-per-site,Simplified Use of `MASTER_SITES:n` with One File Per Site]. [[ports-master-sites-n-example-simple-use-one-file-per-site]] .Simplified Use of `MASTER_SITES:n` with One File Per Site [example] ==== [.programlisting] .... MASTER_SITES= ftp://ftp1.example.com/:source1 \ http://www.example.com/:source2 DISTFILES= source1.tar.gz:source1 \ source2.tar.gz:source2 .... ==== Multiple distribution files can have the same group. Continuing the previous example, suppose that there was a third distfile, [.filename]#source3.tar.gz#, that is downloaded from `www.example.com`. The [.filename]#Makefile# would then be written like crossref:makefiles[ports-master-sites-n-example-simple-use-more-than-one-file-per-site,Simplified Use of `MASTER_SITES:n` with More Than One File Per Site]. [[ports-master-sites-n-example-simple-use-more-than-one-file-per-site]] .Simplified Use of `MASTER_SITES:n` with More Than One File Per Site [example] ==== [.programlisting] .... MASTER_SITES= ftp://ftp.example.com/:source1 \ http://www.example.com/:source2 DISTFILES= source1.tar.gz:source1 \ source2.tar.gz:source2 \ source3.tar.gz:source2 .... ==== [[ports-master-sites-n-detailed]] ==== Detailed Information Okay, so the previous example did not reflect the new port's needs? In this section we will explain in detail how the fine grained fetching mechanism `MASTER_SITES:n` works and how it can be used. . Elements can be postfixed with `:__n__` where _n_ is `[^:,]\+`, that is, _n_ could conceptually be any alphanumeric string but we will limit it to `[a-zA-Z_][0-9a-zA-Z_]+` for now. + Moreover, string matching is case sensitive; that is, `n` is different from `N`. + However, these words cannot be used for postfixing purposes since they yield special meaning: `default`, `all` and `ALL` (they are used internally in item crossref:makefiles[porting-master-sites-n-what-changes-in-port-targets, ii]). Furthermore, `DEFAULT` is a special purpose word (check item crossref:makefiles[porting-master-sites-n-DEFAULT-group,3]). . Elements postfixed with `:n` belong to the group `n`, `:m` belong to group `m` and so forth. + . [[porting-master-sites-n-DEFAULT-group]] Elements without a postfix are groupless, they all belong to the special group `DEFAULT`. Any elements postfixed with `DEFAULT`, is just being redundant unless an element belongs to both `DEFAULT` and other groups at the same time (check item crossref:makefiles[porting-master-sites-n-comma-operator,5]). + These examples are equivalent but the first one is preferred: + [.programlisting] .... MASTER_SITES= alpha .... + [.programlisting] .... MASTER_SITES= alpha:DEFAULT .... . Groups are not exclusive, an element may belong to several different groups at the same time and a group can either have either several different elements or none at all. + . [[porting-master-sites-n-comma-operator]] When an element belongs to several groups at the same time, use the comma operator (`,`). + Instead of repeating it several times, each time with a different postfix, we can list several groups at once in a single postfix. For instance, `:m,n,o` marks an element that belongs to group `m`, `n` and `o`. + All these examples are equivalent but the last one is preferred: + [.programlisting] .... MASTER_SITES= alpha alpha:SOME_SITE .... + [.programlisting] .... MASTER_SITES= alpha:DEFAULT alpha:SOME_SITE .... + [.programlisting] .... MASTER_SITES= alpha:SOME_SITE,DEFAULT .... + [.programlisting] .... MASTER_SITES= alpha:DEFAULT,SOME_SITE .... . All sites within a given group are sorted according to `MASTER_SORT_AWK`. All groups within `MASTER_SITES` and `PATCH_SITES` are sorted as well. + . [[porting-master-sites-n-group-semantics]] Group semantics can be used in any of the variables `MASTER_SITES`, `PATCH_SITES`, `MASTER_SITE_SUBDIR`, `PATCH_SITE_SUBDIR`, `DISTFILES`, and `PATCHFILES` according to this syntax: .. All `MASTER_SITES`, `PATCH_SITES`, `MASTER_SITE_SUBDIR` and `PATCH_SITE_SUBDIR` elements must be terminated with the forward slash `/` character. If any elements belong to any groups, the group postfix `:__n__` must come right after the terminator `/`. The `MASTER_SITES:n` mechanism relies on the existence of the terminator `/` to avoid confusing elements where a `:n` is a valid part of the element with occurrences where `:n` denotes group `n`. For compatibility purposes, since the `/` terminator was not required before in both `MASTER_SITE_SUBDIR` and `PATCH_SITE_SUBDIR` elements, if the postfix immediate preceding character is not a `/` then `:n` will be considered a valid part of the element instead of a group postfix even if an element is postfixed with `:n`. See both crossref:makefiles[ports-master-sites-n-example-detailed-use-master-site-subdir,Detailed Use of `MASTER_SITES:n` in `MASTER_SITE_SUBDIR`] and - crossref:makefiles[ports-master-sites-n-example-detailed-use-complete-example-master-sites,Detailed Use of `MASTER_SITES:n` with Comma Operator, Multiple Files, Multiple Sites and Multiple Subdirectories]. + crossref:makefiles[ports-master-sites-n-example-detailed-use-complete-example-master-sites,"Detailed Use of `MASTER_SITES:n` with Comma Operator, Multiple Files, Multiple Sites and Multiple Subdirectories"]. + [[ports-master-sites-n-example-detailed-use-master-site-subdir]] .Detailed Use of `MASTER_SITES:n` in `MASTER_SITE_SUBDIR` [example] ==== [.programlisting] .... MASTER_SITE_SUBDIR= old:n new/:NEW .... *** Directories within group `DEFAULT` -> old:n *** Directories within group `NEW` -> new ==== + [[ports-master-sites-n-example-detailed-use-complete-example-master-sites]] .Detailed Use of `MASTER_SITES:n` with Comma Operator, Multiple Files, Multiple Sites and Multiple Subdirectories [example] ==== [.programlisting] .... MASTER_SITES= http://site1/%SUBDIR%/ http://site2/:DEFAULT \ http://site3/:group3 http://site4/:group4 \ http://site5/:group5 http://site6/:group6 \ http://site7/:DEFAULT,group6 \ http://site8/%SUBDIR%/:group6,group7 \ http://site9/:group8 DISTFILES= file1 file2:DEFAULT file3:group3 \ file4:group4,group5,group6 file5:grouping \ file6:group7 MASTER_SITE_SUBDIR= directory-trial:1 directory-n/:groupn \ directory-one/:group6,DEFAULT \ directory .... The previous example results in this fine grained fetching. Sites are listed in the exact order they will be used. *** [.filename]#file1# will be fetched from **** `MASTER_SITE_OVERRIDE` **** http://site1/directory-trial:1/ **** http://site1/directory-one/ **** http://site1/directory/ **** http://site2/ **** http://site7/ **** `MASTER_SITE_BACKUP` *** [.filename]#file2# will be fetched exactly as [.filename]#file1# since they both belong to the same group **** `MASTER_SITE_OVERRIDE` **** http://site1/directory-trial:1/ **** http://site1/directory-one/ **** http://site1/directory/ **** http://site2/ **** http://site7/ **** `MASTER_SITE_BACKUP` *** [.filename]#file3# will be fetched from **** `MASTER_SITE_OVERRIDE` **** http://site3/ **** `MASTER_SITE_BACKUP` *** [.filename]#file4# will be fetched from **** `MASTER_SITE_OVERRIDE` **** http://site4/ **** http://site5/ **** http://site6/ **** http://site7/ **** http://site8/directory-one/ **** `MASTER_SITE_BACKUP` *** [.filename]#file5# will be fetched from **** `MASTER_SITE_OVERRIDE` **** `MASTER_SITE_BACKUP` *** [.filename]#file6# will be fetched from **** `MASTER_SITE_OVERRIDE` **** http://site8/ **** `MASTER_SITE_BACKUP` ==== . How do I group one of the special macros from [.filename]#bsd.sites.mk#, for example, SourceForge (`SF`)? + This has been simplified as much as possible. See crossref:makefiles[ports-master-sites-n-example-detailed-use-master-site-sourceforge,Detailed Use of `MASTER_SITES:n` with SourceForge (`SF`)]. + [[ports-master-sites-n-example-detailed-use-master-site-sourceforge]] .Detailed Use of `MASTER_SITES:n` with SourceForge (`SF`) [example] ==== [.programlisting] .... MASTER_SITES= http://site1/ SF/something/1.0:sourceforge,TEST DISTFILES= something.tar.gz:sourceforge .... [.filename]#something.tar.gz# will be fetched from all sites within SourceForge. ==== . How do I use this with `PATCH*`? + All examples were done with `MASTER*` but they work exactly the same for `PATCH*` ones as can be seen in crossref:makefiles[ports-master-sites-n-example-detailed-use-patch-sites,Simplified Use of `MASTER_SITES:n` with `PATCH_SITES`]. + [[ports-master-sites-n-example-detailed-use-patch-sites]] .Simplified Use of `MASTER_SITES:n` with `PATCH_SITES` [example] ==== [.programlisting] .... PATCH_SITES= http://site1/ http://site2/:test PATCHFILES= patch1:test .... ==== [[port-master-sites-n-what-changed]] ==== What Does Change for Ports? What Does Not? [lowerroman] . All current ports remain the same. The `MASTER_SITES:n` feature code is only activated if there are elements postfixed with `:__n__` like elements according to the aforementioned syntax rules, especially as shown in item crossref:makefiles[porting-master-sites-n-group-semantics, 7]. + . [[porting-master-sites-n-what-changes-in-port-targets]] The port targets remain the same: `checksum`, `makesum`, `patch`, `configure`, `build`, etc. With the obvious exceptions of `do-fetch`, `fetch-list`, `master-sites` and `patch-sites`. ** `do-fetch`: deploys the new grouping postfixed `DISTFILES` and `PATCHFILES` with their matching group elements within both `MASTER_SITES` and `PATCH_SITES` which use matching group elements within both `MASTER_SITE_SUBDIR` and `PATCH_SITE_SUBDIR`. Check - crossref:makefiles[ports-master-sites-n-example-detailed-use-complete-example-master-sites,Detailed Use of `MASTER_SITES:n` with Comma Operator, Multiple Files, Multiple Sites and Multiple Subdirectories]. + crossref:makefiles[ports-master-sites-n-example-detailed-use-complete-example-master-sites,"Detailed Use of `MASTER_SITES:n` with Comma Operator, Multiple Files, Multiple Sites and Multiple Subdirectories"]. ** `fetch-list`: works like old `fetch-list` with the exception that it groups just like `do-fetch`. ** `master-sites` and `patch-sites`: (incompatible with older versions) only return the elements of group `DEFAULT`; in fact, they execute targets `master-sites-default` and `patch-sites-default` respectively. + Furthermore, using target either `master-sites-all` or `patch-sites-all` is preferred to directly checking either `MASTER_SITES` or `PATCH_SITES`. Also, directly checking is not guaranteed to work in any future versions. Check item crossref:makefiles[porting-master-sites-n-new-port-targets-master-sites-all, B] for more information on these new port targets. . New port targets .. There are `master-sites-_n_` and `patch-sites-_n_` targets which will list the elements of the respective group _n_ within `MASTER_SITES` and `PATCH_SITES` respectively. For instance, both `master-sites-DEFAULT` and `patch-sites-DEFAULT` will return the elements of group `DEFAULT`, `master-sites-test` and `patch-sites-test` of group `test`, and thereon. + .. [[porting-master-sites-n-new-port-targets-master-sites-all]] There are new targets `master-sites-all` and `patch-sites-all` which do the work of the old `master-sites` and `patch-sites` ones. They return the elements of all groups as if they all belonged to the same group with the caveat that it lists as many `MASTER_SITE_BACKUP` and `MASTER_SITE_OVERRIDE` as there are groups defined within either `DISTFILES` or `PATCHFILES`; respectively for `master-sites-all` and `patch-sites-all`. [[makefile-dist_subdir]] === `DIST_SUBDIR` Do not let the port clutter [.filename]#/usr/ports/distfiles#. If the port requires a lot of files to be fetched, or contains a file that has a name that might conflict with other ports (for example, [.filename]#Makefile#), set `DIST_SUBDIR` to the name of the port (`${PORTNAME}` or `${PKGNAMEPREFIX}${PORTNAME}` are fine). This will change `DISTDIR` from the default [.filename]#/usr/ports/distfiles# to [.filename]#/usr/ports/distfiles/${DIST_SUBDIR}#, and in effect puts everything that is required for the port into that subdirectory. It will also look at the subdirectory with the same name on the backup master site at http://distcache.FreeBSD.org[http://distcache.FreeBSD.org] (Setting `DISTDIR` explicitly in [.filename]#Makefile# will not accomplish this, so please use `DIST_SUBDIR`.) [NOTE] ==== This does not affect `MASTER_SITES` defined in the [.filename]#Makefile#. ==== [[makefile-maintainer]] == `MAINTAINER` Set the mail-address here. Please. _:-)_ Only a single address without the comment part is allowed as a `MAINTAINER` value. The format used is `user@hostname.domain`. Please do not include any descriptive text such as a real name in this entry. That merely confuses the Ports infrastructure and most tools using it. The maintainer is responsible for keeping the port up to date and making sure that it works correctly. For a detailed description of the responsibilities of a port maintainer, refer to extref:{contributing}[The challenge for port maintainers, maintain-port]. [NOTE] ==== A maintainer volunteers to keep a port in good working order. Maintainers have the primary responsibility for their ports, but not exclusive ownership. Ports exist for the benefit of the community and, in reality, belong to the community. What this means is that people other than the maintainer can make changes to a port. Large changes to the Ports Collection might require changes to many ports. The FreeBSD Ports Management Team or members of other teams might modify ports to fix dependency issues or other problems, like a version bump for a shared library update. Some types of fixes have "blanket approval" from the {portmgr}, allowing any committer to fix those categories of problems on any port. These fixes do not need approval from the maintainer. Blanket approval for most ports applies to fixes like infrastructure changes, or trivial and _tested_ build and runtime fixes. The current list is available in extref:{committers-guide}[Ports section of the Committer's Guide, ports-qa-misc-blanket-approval]. ==== Other changes to the port will be sent to the maintainer for review and approval before being committed. If the maintainer does not respond to an update request after two weeks (excluding major public holidays), then that is considered a maintainer timeout, and the update can be made without explicit maintainer approval. If the maintainer does not respond within three months, or if there have been three consecutive timeouts, then that maintainer is considered absent without leave, and all of their ports can be assigned back to the pool. Exceptions to this are anything maintained by the {portmgr}, or the {security-officer}. No unauthorized commits may ever be made to ports maintained by those groups. We reserve the right to modify the maintainer's submission to better match existing policies and style of the Ports Collection without explicit blessing from the submitter or the maintainer. Also, large infrastructural changes can result in a port being modified without the maintainer's consent. These kinds of changes will never affect the port's functionality. The {portmgr} reserves the right to revoke or override anyone's maintainership for any reason, and the {security-officer} reserves the right to revoke or override maintainership for security reasons. [[makefile-comment]] == `COMMENT` The comment is a one-line description of a port shown by `pkg info`. Please follow these rules when composing it: . The COMMENT string should be 70 characters or less. . Do _not_ include the package name or version number of software. . The comment must begin with a capital and end without a period. . Do not start with an indefinite article (that is, A or An). . Capitalize names such as Apache, JavaScript, or Perl. . Use a serial comma for lists of words: "green, red, and blue." . Check for spelling errors. Here is an example: [.programlisting] .... COMMENT= Cat chasing a mouse all over the screen .... The COMMENT variable immediately follows the MAINTAINER variable in the [.filename]#Makefile#. [[makefile-www]] == Project website Each port should point to a website that provides more information about the software. Whenever possible, this should be the official project website maintained by the developers of the software. [.programlisting] .... WWW= https://ffmpeg.org/ .... But it can also be a directory or resource in the source code repository: [.programlisting] .... WWW= https://sourceforge.net/projects/mpd/ .... The WWW variable immediately follows the COMMENT variable in the [.filename]#Makefile#. If the same content can be accessed via HTTP and HTTPS, the URL starting with `https://` shall be used. If the URI is the root of the website or directory, it must be terminated with a slash. This information used to be placed into the last line of the [.filename]#pkg-descr# file. It has been moved into the Makefile for easier maintenance and processing. Having a `WWW:` line at the end of the [.filename]#pkg-descr# file is deprecated. [[licenses]] == Licenses Each port must document the license under which it is available. If it is not an OSI approved license it must also document any restrictions on redistribution. [[licenses-license]] === `LICENSE` A short name for the license or licenses if more than one license apply. If it is one of the licenses listed in crossref:makefiles[licenses-license-list,Predefined License List], only `LICENSE_FILE` and `LICENSE_DISTFILES` variables can be set. If this is a license that has not been defined in the ports framework (see crossref:makefiles[licenses-license-list,Predefined License List]), the `LICENSE_PERMS` and `LICENSE_NAME` must be set, along with either `LICENSE_FILE` or `LICENSE_TEXT`. `LICENSE_DISTFILES` and `LICENSE_GROUPS` can also be set, but are not required. The predefined licenses are shown in crossref:makefiles[licenses-license-list,Predefined License List]. The current list is always available in [.filename]#Mk/bsd.licenses.db.mk#. [[licenses-license-ex1]] .Simplest Usage, Predefined Licenses [example] ==== When the [.filename]#README# of some software says "This software is under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version." but does not provide the license file, use this: [.programlisting] .... LICENSE= LGPL21+ .... When the software provides the license file, use this: [.programlisting] .... LICENSE= LGPL21+ LICENSE_FILE= ${WRKSRC}/COPYING .... ==== For the predefined licenses, the default permissions are `dist-mirror dist-sell pkg-mirror pkg-sell auto-accept`. [[licenses-license-list]] .Predefined License List [cols="1,1,1,1", frame="none", options="header"] |=== | Short Name | Name | Group | Permissions |`AGPLv3` |GNU Affero General Public License version 3 |`FSF GPL OSI` |(default) |`AGPLv3+` |GNU Affero General Public License version 3 (or later) |`FSF GPL OSI` |(default) |`APACHE10` |Apache License 1.0 |`FSF` |(default) |`APACHE11` |Apache License 1.1 |`FSF OSI` |(default) |`APACHE20` |Apache License 2.0 |`FSF OSI` |(default) |`ART10` |Artistic License version 1.0 |`OSI` |(default) |`ART20` |Artistic License version 2.0 |`FSF GPL OSI` |(default) |`ARTPERL10` |Artistic License (perl) version 1.0 |`OSI` |(default) |`BSD` |BSD license Generic Version (deprecated) |`FSF OSI COPYFREE` |(default) |`BSD2CLAUSE` |BSD 2-clause "Simplified" License |`FSF OSI COPYFREE` |(default) |`BSD3CLAUSE` |BSD 3-clause "New" or "Revised" License |`FSF OSI COPYFREE` |(default) |`BSD4CLAUSE` |BSD 4-clause "Original" or "Old" License |`FSF` |(default) |`BSL` |Boost Software License |`FSF OSI COPYFREE` |(default) |`CC-BY-1.0` |Creative Commons Attribution 1.0 | |(default) |`CC-BY-2.0` |Creative Commons Attribution 2.0 | |(default) |`CC-BY-2.5` |Creative Commons Attribution 2.5 | |(default) |`CC-BY-3.0` |Creative Commons Attribution 3.0 | |(default) |`CC-BY-4.0` |Creative Commons Attribution 4.0 | |(default) |`CC-BY-NC-1.0` |Creative Commons Attribution Non Commercial 1.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-2.0` |Creative Commons Attribution Non Commercial 2.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-2.5` |Creative Commons Attribution Non Commercial 2.5 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-3.0` |Creative Commons Attribution Non Commercial 3.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-4.0` |Creative Commons Attribution Non Commercial 4.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-ND-1.0` |Creative Commons Attribution Non Commercial No Derivatives 1.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-ND-2.0` |Creative Commons Attribution Non Commercial No Derivatives 2.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-ND-2.5` |Creative Commons Attribution Non Commercial No Derivatives 2.5 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-ND-3.0` |Creative Commons Attribution Non Commercial No Derivatives 3.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-ND-4.0` |Creative Commons Attribution Non Commercial No Derivatives 4.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-SA-1.0` |Creative Commons Attribution Non Commercial Share Alike 1.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-SA-2.0` |Creative Commons Attribution Non Commercial Share Alike 2.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-SA-2.5` |Creative Commons Attribution Non Commercial Share Alike 2.5 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-SA-3.0` |Creative Commons Attribution Non Commercial Share Alike 3.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-NC-SA-4.0` |Creative Commons Attribution Non Commercial Share Alike 4.0 | |`dist-mirror``pkg-mirror``auto-accept` |`CC-BY-ND-1.0` |Creative Commons Attribution No Derivatives 1.0 | |(default) |`CC-BY-ND-2.0` |Creative Commons Attribution No Derivatives 2.0 | |(default) |`CC-BY-ND-2.5` |Creative Commons Attribution No Derivatives 2.5 | |(default) |`CC-BY-ND-3.0` |Creative Commons Attribution No Derivatives 3.0 | |(default) |`CC-BY-ND-4.0` |Creative Commons Attribution No Derivatives 4.0 | |(default) |`CC-BY-SA-1.0` |Creative Commons Attribution Share Alike 1.0 | |(default) |`CC-BY-SA-2.0` |Creative Commons Attribution Share Alike 2.0 | |(default) |`CC-BY-SA-2.5` |Creative Commons Attribution Share Alike 2.5 | |(default) |`CC-BY-SA-3.0` |Creative Commons Attribution Share Alike 3.0 | |(default) |`CC-BY-SA-4.0` |Creative Commons Attribution Share Alike 4.0 | |(default) |`CC0-1.0` |Creative Commons Zero v1.0 Universal |`FSF GPL COPYFREE` |(default) |`CDDL` |Common Development and Distribution License |`FSF OSI` |(default) |`CPAL-1.0` |Common Public Attribution License |`FSF OSI` |(default) |`ClArtistic` |Clarified Artistic License |`FSF GPL OSI` |(default) |`EPL` |Eclipse Public License |`FSF OSI` |(default) |`GFDL` |GNU Free Documentation License |`FSF` |(default) |`GMGPL` |GNAT Modified General Public License |`FSF GPL OSI` |(default) |`GPLv1` |GNU General Public License version 1 |`FSF GPL OSI` |(default) |`GPLv1+` |GNU General Public License version 1 (or later) |`FSF GPL OSI` |(default) |`GPLv2` |GNU General Public License version 2 |`FSF GPL OSI` |(default) |`GPLv2+` |GNU General Public License version 2 (or later) |`FSF GPL OSI` |(default) |`GPLv3` |GNU General Public License version 3 |`FSF GPL OSI` |(default) |`GPLv3+` |GNU General Public License version 3 (or later) |`FSF GPL OSI` |(default) |`GPLv3RLE` |GNU GPL version 3 Runtime Library Exception |`FSF GPL OSI` |(default) |`GPLv3RLE+` |GNU GPL version 3 Runtime Library Exception (or later) |`FSF GPL OSI` |(default) |`ISCL` |Internet Systems Consortium License |`FSF GPL OSI COPYFREE` |(default) |`LGPL20` |GNU Library General Public License version 2.0 |`FSF GPL OSI` |(default) |`LGPL20+` |GNU Library General Public License version 2.0 (or later) |`FSF GPL OSI` |(default) |`LGPL21` |GNU Lesser General Public License version 2.1 |`FSF GPL OSI` |(default) |`LGPL21+` |GNU Lesser General Public License version 2.1 (or later) |`FSF GPL OSI` |(default) |`LGPL3` |GNU Lesser General Public License version 3 |`FSF GPL OSI` |(default) |`LGPL3+` |GNU Lesser General Public License version 3 (or later) |`FSF GPL OSI` |(default) |`LPPL10` |LaTeX Project Public License version 1.0 |`FSF OSI` |`dist-mirror dist-sell` |`LPPL11` |LaTeX Project Public License version 1.1 |`FSF OSI` |`dist-mirror dist-sell` |`LPPL12` |LaTeX Project Public License version 1.2 |`FSF OSI` |`dist-mirror dist-sell` |`LPPL13` |LaTeX Project Public License version 1.3 |`FSF OSI` |`dist-mirror dist-sell` |`LPPL13a` |LaTeX Project Public License version 1.3a |`FSF OSI` |`dist-mirror dist-sell` |`LPPL13b` |LaTeX Project Public License version 1.3b |`FSF OSI` |`dist-mirror dist-sell` |`LPPL13c` |LaTeX Project Public License version 1.3c |`FSF OSI` |`dist-mirror dist-sell` |`MIT` |MIT license / X11 license |`COPYFREE FSF GPL OSI` |(default) |`MPL10` |Mozilla Public License version 1.0 |`FSF OSI` |(default) |`MPL11` |Mozilla Public License version 1.1 |`FSF OSI` |(default) |`MPL20` |Mozilla Public License version 2.0 |`FSF OSI` |(default) |`NCSA` |University of Illinois/NCSA Open Source License |`COPYFREE FSF GPL OSI` |(default) |`NONE` |No license specified | |`none` |`OFL10` |SIL Open Font License version 1.0 (https://scripts.sil.org/OFL/) |`FONTS` |(default) |`OFL11` |SIL Open Font License version 1.1 (https://scripts.sil.org/OFL/) |`FONTS` |(default) |`OWL` |Open Works License (owl.apotheon.org) |`COPYFREE` |(default) |`OpenSSL` |OpenSSL License |`FSF` |(default) |`PD` |Public Domain |`GPL COPYFREE` |(default) |`PHP202` |PHP License version 2.02 |`FSF OSI` |(default) |`PHP30` |PHP License version 3.0 |`FSF OSI` |(default) |`PHP301` |PHP License version 3.01 |`FSF OSI` |(default) |`PSFL` |Python Software Foundation License |`FSF GPL OSI` |(default) |`PostgreSQL` |PostgreSQL License |`FSF GPL OSI COPYFREE` |(default) |`RUBY` |Ruby License |`FSF` |(default) |`UNLICENSE` |The Unlicense |`COPYFREE FSF GPL` |(default) |`WTFPL` |Do What the Fuck You Want To Public License version 2 |`GPL FSF COPYFREE` |(default) |`WTFPL1` |Do What the Fuck You Want To Public License version 1 |`GPL FSF COPYFREE` |(default) |`ZLIB` |zlib License |`GPL FSF OSI` |(default) |`ZPL21` |Zope Public License version 2.1 |`GPL OSI` |(default) |=== [[licenses-license_perms]] === `LICENSE_PERMS` and `LICENSE_PERMS_NAME_` Permissions. use `none` if empty. .License Permissions List [[licenses-license_perms-dist-mirror]] `dist-mirror`:: Redistribution of the distribution files is permitted. The distribution files will be added to the FreeBSD `MASTER_SITE_BACKUP` CDN. [[licenses-license_perms-no-dist-mirror]] `no-dist-mirror`:: Redistribution of the distribution files is prohibited. This is equivalent to setting crossref:special[porting-restrictions-restricted,`RESTRICTED`]. The distribution files will _not_ be added to the FreeBSD `MASTER_SITE_BACKUP` CDN. [[licenses-license_perms-dist-sell]] `dist-sell`:: Selling of distribution files is permitted. The distribution files will be present on the installer images. [[licenses-license_perms-no-dist-sell]] `no-dist-sell`:: Selling of distribution files is prohibited. This is equivalent to setting crossref:special[porting-restrictions-no_cdrom,`NO_CDROM`]. [[licenses-license_perms-pkg-mirror]] `pkg-mirror`:: Free redistribution of package is permitted. The package will be distributed on the FreeBSD package CDN https://pkg.freebsd.org/[https://pkg.freebsd.org/]. [[licenses-license_perms-no-pkg-mirror]] `no-pkg-mirror`:: Free redistribution of package is prohibited. Equivalent to setting crossref:special[porting-restrictions-no_package,`NO_PACKAGE`]. The package will _not_ be distributed from the FreeBSD package CDN https://pkg.freebsd.org/[https://pkg.freebsd.org/]. [[licenses-license_perms-pkg-sell]] `pkg-sell`:: Selling of package is permitted. The package will be present on the installer images. [[licenses-license_perms-no-pkg-sell]] `no-pkg-sell`:: Selling of package is prohibited. This is equivalent to setting crossref:special[porting-restrictions-no_cdrom,`NO_CDROM`]. The package will _not_ be present on the installer images. [[licenses-license_perms-auto-accept]] `auto-accept`:: License is accepted by default. Prompts to accept a license are not displayed unless the user has defined `LICENSES_ASK`. Use this unless the license states the user must accept the terms of the license. [[licenses-license_perms-no-auto-accept]] `no-auto-accept`:: License is not accepted by default. The user will always be asked to confirm the acceptance of this license. This must be used if the license states that the user must accept its terms. When both `_permission_` and `no-_permission_` is present the `no-_permission_` will cancel `_permission_`. When `_permission_` is not present, it is considered to be a `no-_permission_`. [WARNING] ==== Some missing permissions will prevent a port (and all ports depending on it) from being usable by package users: A port without the `auto-accept` permission will never be built and all the ports depending on it will be ignored. A port without the `pkg-mirror` permission, and any ports that depend on it, will be removed after the build, thus ensuring they are not distributed. ==== [[licenses-license_perms-ex1]] .Nonstandard License [example] ==== Read the terms of the license and translate those using the available permissions. [.programlisting] .... LICENSE= UNKNOWN LICENSE_NAME= unknown LICENSE_TEXT= This program is NOT in public domain.\ It can be freely distributed for non-commercial purposes only. LICENSE_PERMS= dist-mirror no-dist-sell pkg-mirror no-pkg-sell auto-accept .... ==== [[licenses-license_perms-ex2]] .Standard and Nonstandard Licenses [example] ==== Read the terms of the license and express those using the available permissions. In case of doubt, please ask for guidance on the {freebsd-ports}. [.programlisting] .... LICENSE= WARSOW GPLv2 LICENSE_COMB= multi LICENSE_NAME_WARSOW= Warsow Content License LICENSE_FILE_WARSOW= ${WRKSRC}/docs/license.txt LICENSE_PERMS_WARSOW= dist-mirror pkg-mirror auto-accept .... When the permissions of the GPLv2 and the UNKNOWN licenses are mixed, the port ends up with `dist-mirror dist-sell pkg-mirror pkg-sell auto-accept dist-mirror no-dist-sell pkg-mirror no-pkg-sell auto-accept`. The `no-_permissions_` cancel the _permissions_. The resulting list of permissions are _dist-mirror pkg-mirror auto-accept_. The distribution files and the packages will not be available on the installer images. ==== [[licenses-license_groups]] === `LICENSE_GROUPS` and `LICENSE_GROUPS_NAME` Groups the license belongs. .Predefined License Groups List [[licenses-license_groups-FSF]] `FSF`:: Free Software Foundation Approved, see the https://www.fsf.org/licensing/[FSF Licensing & Compliance Team]. [[licenses-license_groups-GPL]] `GPL`:: GPL Compatible [[licenses-license_groups-OSI]] `OSI`:: OSI Approved, see the Open Source Initiative https://opensource.org/licenses/[Open Source Licenses] page. [[licenses-license_groups-COPYFREE]] `COPYFREE`:: Comply with Copyfree Standard Definition, see the https://copyfree.org/standard/licenses/[Copyfree Licenses] page. [[licenses-license_groups-FONTS]] `FONTS`:: Font licenses [[licenses-license_name]] === `LICENSE_NAME` and `LICENSE_NAME_NAME` Full name of the license. [[licenses-license_name-ex1]] .`LICENSE_NAME` [example] ==== [.programlisting] .... LICENSE= UNRAR LICENSE_NAME= UnRAR License LICENSE_FILE= ${WRKSRC}/license.txt LICENSE_PERMS= dist-mirror dist-sell pkg-mirror pkg-sell auto-accept .... ==== [[licenses-license_file]] === `LICENSE_FILE` and `LICENSE_FILE_NAME` Full path to the file containing the license text, usually [.filename]#${WRKSRC}/some/file#. If the file is not in the distfile, and its content is too long to be put in crossref:makefiles[licenses-license_text,`LICENSE_TEXT`], put it in a new file in [.filename]#${FILESDIR}#. [[licenses-license_file-ex1]] .`LICENSE_FILE` [example] ==== [.programlisting] .... LICENSE= GPLv3+ LICENSE_FILE= ${WRKSRC}/COPYING .... ==== [[licenses-license_text]] === `LICENSE_TEXT` and `LICENSE_TEXT_NAME` Text to use as a license. Useful when the license is not in the distribution files and its text is short. [[licenses-license_text-ex1]] .`LICENSE_TEXT` [example] ==== [.programlisting] .... LICENSE= UNKNOWN LICENSE_NAME= unknown LICENSE_TEXT= This program is NOT in public domain.\ It can be freely distributed for non-commercial purposes only,\ and THERE IS NO WARRANTY FOR THIS PROGRAM. LICENSE_PERMS= dist-mirror no-dist-sell pkg-mirror no-pkg-sell auto-accept .... ==== [[licenses-license_distfiles]] === `LICENSE_DISTFILES` and `LICENSE_DISTFILES_NAME` The distribution files to which the licenses apply. Defaults to all the distribution files. [[licenses-license_distfiles-ex1]] .`LICENSE_DISTFILES` [example] ==== Used when the distribution files do not all have the same license. For example, one has a code license, and another has some artwork that cannot be redistributed: [.programlisting] .... MASTER_SITES= SF/some-game DISTFILES= ${DISTNAME}${EXTRACT_SUFX} artwork.zip LICENSE= BSD3CLAUSE ARTWORK LICENSE_COMB= dual LICENSE_NAME_ARTWORK= The game artwork license LICENSE_TEXT_ARTWORK= The README says that the files cannot be redistributed LICENSE_PERMS_ARTWORK= pkg-mirror pkg-sell auto-accept LICENSE_DISTFILES_BSD3CLAUSE= ${DISTNAME}${EXTRACT_SUFX} LICENSE_DISTFILES_ARTWORK= artwork.zip .... ==== [[licenses-license_comb]] === `LICENSE_COMB` Set to `multi` if all licenses apply. Set to `dual` if any license applies. Defaults to `single`. [[licenses-license_comb-ex1]] .Dual Licenses [example] ==== When a port says "This software may be distributed under the GNU General Public License or the Artistic License", it means that either license can be used. Use this: [.programlisting] .... LICENSE= ART10 GPLv1 LICENSE_COMB= dual .... If license files are provided, use this: [.programlisting] .... LICENSE= ART10 GPLv1 LICENSE_COMB= dual LICENSE_FILE_ART10= ${WRKSRC}/Artistic LICENSE_FILE_GPLv1= ${WRKSRC}/Copying .... ==== [[licenses-license_comb-ex2]] .Multiple Licenses [example] ==== When part of a port has one license, and another part has a different license, use `multi`: [.programlisting] .... LICENSE= GPLv2 LGPL21+ LICENSE_COMB= multi .... ==== [[makefile-portscout]] == `PORTSCOUT` Portscout is an automated distfile check utility for the FreeBSD Ports Collection, described in detail in crossref:keeping-up[distfile-survey,Portscout: the FreeBSD Ports Distfile Scanner]. `PORTSCOUT` defines special conditions within which the Portscout distfile scanner is restricted. Situations where `PORTSCOUT` is set include: * When distfiles have to be ignored for specific versions. For example, to exclude version _8.2_ and version _8.3_ from distfile version checks because they are known to be broken, add: + [.programlisting] .... PORTSCOUT= skipv:8.2,8.3 .... * When distfile version checks have to be disabled completely. For example, if a port is not going to be updated ever again, add: + [.programlisting] .... PORTSCOUT= ignore:1 .... * When specific versions or specific major and minor revisions of a distfile must be checked. For example, if only version _0.6.4_ must be monitored because newer versions have compatibility issues with FreeBSD, add: + [.programlisting] .... PORTSCOUT= limit:^0\.6\.4 .... * When URLs listing the available versions differ from the download URLs. For example, to limit distfile version checks to the download page for the package:databases/pgtune[] port, add: + [.programlisting] .... PORTSCOUT= site:http://www.renpy.org/dl/release/ .... [[makefile-depend]] == Dependencies Many ports depend on other ports. This is a very convenient feature of most Unix-like operating systems, including FreeBSD. Multiple ports can share a common dependency, rather than bundling that dependency with every port or package that needs it. There are seven variables that can be used to ensure that all the required bits will be on the user's machine. There are also some pre-supported dependency variables for common cases, plus a few more to control the behavior of dependencies. [IMPORTANT] ==== When software has extra dependencies that provide extra features, the base dependencies listed in `*_DEPENDS` should include the extra dependencies that would benefit most users. The base dependencies should never be a "minimal" dependency set. The goal is not to include every dependency possible. Only include those that will benefit most people. ==== [[makefile-lib_depends]] === `LIB_DEPENDS` This variable specifies the shared libraries this port depends on. It is a list of `_lib:dir_` tuples where `_lib_` is the name of the shared library, `_dir_` is the directory in which to find it in case it is not available. For example, [.programlisting] .... LIB_DEPENDS= libjpeg.so:graphics/jpeg .... will check for a shared jpeg library with any version, and descend into the [.filename]#graphics/jpeg# subdirectory of the ports tree to build and install it if it is not found. The dependency is checked twice, once from within the `build` target and then from within the `install` target. Also, the name of the dependency is put into the package so that `pkg install` (see man:pkg-install[8]) will automatically install it if it is not on the user's system. [[makefile-run_depends]] === `RUN_DEPENDS` This variable specifies executables or files this port depends on during run-time. It is a list of ``_path:dir_``[:``_target_``] tuples where `_path_` is the name of the executable or file, _dir_ is the directory in which to find it in case it is not available, and _target_ is the target to call in that directory. If _path_ starts with a slash (`/`), it is treated as a file and its existence is tested with `test -e`; otherwise, it is assumed to be an executable, and `which -s` is used to determine if the program exists in the search path. For example, [.programlisting] .... RUN_DEPENDS= ${LOCALBASE}/news/bin/innd:news/inn \ xmlcatmgr:textproc/xmlcatmgr .... will check if the file or directory [.filename]#/usr/local/news/bin/innd# exists, and build and install it from the [.filename]#news/inn# subdirectory of the ports tree if it is not found. It will also see if an executable called `xmlcatmgr` is in the search path, and descend into [.filename]#textproc/xmlcatmgr# to build and install it if it is not found. [NOTE] ==== In this case, `innd` is actually an executable; if an executable is in a place that is not expected to be in the search path, use the full pathname. ==== [NOTE] ==== The official search `PATH` used on the ports build cluster is [.programlisting] .... /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin .... ==== The dependency is checked from within the `install` target. Also, the name of the dependency is put into the package so that `pkg install` (see man:pkg-install[8]) will automatically install it if it is not on the user's system. The _target_ part can be omitted if it is the same as `DEPENDS_TARGET`. A quite common situation is when `RUN_DEPENDS` is literally the same as `BUILD_DEPENDS`, especially if ported software is written in a scripted language or if it requires the same build and run-time environment. In this case, it is both tempting and intuitive to directly assign one to the other: [.programlisting] .... RUN_DEPENDS= ${BUILD_DEPENDS} .... However, such assignment can pollute run-time dependencies with entries not defined in the port's original `BUILD_DEPENDS`. This happens because of man:make[1]'s lazy evaluation of variable assignment. Consider a [.filename]#Makefile# with `USE_*`, which are processed by [.filename]#ports/Mk/bsd.*.mk# to augment initial build dependencies. For example, `USES= gmake` adds package:devel/gmake[] to `BUILD_DEPENDS`. To prevent such additional dependencies from polluting `RUN_DEPENDS`, create another variable with the current content of `BUILD_DEPENDS` and assign it to both `BUILD_DEPENDS` and `RUN_DEPENDS`: [.programlisting] .... MY_DEPENDS= some:devel/some \ other:lang/other BUILD_DEPENDS= ${MY_DEPENDS} RUN_DEPENDS= ${MY_DEPENDS} .... [IMPORTANT] ==== _Do not_ use `:=` to assign `BUILD_DEPENDS` to `RUN_DEPENDS` or vice-versa. All variables are expanded immediately, which is exactly the wrong thing to do and almost always a failure. ==== [[makefile-build_depends]] === `BUILD_DEPENDS` This variable specifies executables or files this port requires to build. Like `RUN_DEPENDS`, it is a list of ``_path:dir_``[:``_target_``] tuples. For example, [.programlisting] .... BUILD_DEPENDS= unzip:archivers/unzip .... will check for an executable called `unzip`, and descend into the [.filename]#archivers/unzip# subdirectory of the ports tree to build and install it if it is not found. [NOTE] ==== "build" here means everything from extraction to compilation. The dependency is checked from within the `extract` target. The _target_ part can be omitted if it is the same as `DEPENDS_TARGET` ==== [[makefile-fetch_depends]] === `FETCH_DEPENDS` This variable specifies executables or files this port requires to fetch. Like the previous two, it is a list of ``_path:dir_``[:``_target_``] tuples. For example, [.programlisting] .... FETCH_DEPENDS= ncftp2:net/ncftp2 .... will check for an executable called `ncftp2`, and descend into the [.filename]#net/ncftp2# subdirectory of the ports tree to build and install it if it is not found. The dependency is checked from within the `fetch` target. The _target_ part can be omitted if it is the same as `DEPENDS_TARGET`. [[makefile-extract_depends]] === `EXTRACT_DEPENDS` This variable specifies executables or files this port requires for extraction. Like the previous, it is a list of ``_path:dir_``[:``_target_``] tuples. For example, [.programlisting] .... EXTRACT_DEPENDS= unzip:archivers/unzip .... will check for an executable called `unzip`, and descend into the [.filename]#archivers/unzip# subdirectory of the ports tree to build and install it if it is not found. The dependency is checked from within the `extract` target. The _target_ part can be omitted if it is the same as `DEPENDS_TARGET`. [NOTE] ==== Use this variable only if the extraction does not already work (the default assumes `tar`) and cannot be made to work using `USES=tar`, `USES=lha` or `USES=zip` described in crossref:uses[uses,Using `USES` Macros]. ==== [[makefile-patch_depends]] === `PATCH_DEPENDS` This variable specifies executables or files this port requires to patch. Like the previous, it is a list of ``_path:dir_``[:``_target_``] tuples. For example, [.programlisting] .... PATCH_DEPENDS= ${NONEXISTENT}:java/jfc:extract .... will descend into the [.filename]#java/jfc# subdirectory of the ports tree to extract it. The dependency is checked from within the `patch` target. The _target_ part can be omitted if it is the same as `DEPENDS_TARGET`. [[makefile-uses]] === `USES` Parameters can be added to define different features and dependencies used by the port. They are specified by adding this line to the [.filename]#Makefile#: [.programlisting] .... USES= feature[:arguments] .... For the complete list of values, please see crossref:uses[uses,Using `USES` Macros]. [WARNING] ==== `USES` cannot be assigned after inclusion of [.filename]#bsd.port.pre.mk#. ==== [[makefile-use-vars]] === `USE_*` Several variables exist to define common dependencies shared by many ports. Their use is optional, but helps to reduce the verbosity of the port [.filename]##Makefile##s. Each of them is styled as `USE_*`. These variables may be used only in the port [.filename]##Makefile##s and [.filename]#ports/Mk/bsd.*.mk#. They are not meant for user-settable options - use `PORT_OPTIONS` for that purpose. [NOTE] ==== It is _always_ incorrect to set any `USE_*` in [.filename]#/etc/make.conf#. For instance, setting [.programlisting] .... USE_GCC=X.Y .... (where X.Y is version number) would add a dependency on gccXY for every port, including `lang/gccXY` itself! ==== [[makefile-use-vars-table]] .`USE_*` [cols="1,1", frame="none", options="header"] |=== | Variable | Means |`USE_GCC` a| The port requires GCC (`gcc` or `{g-plus-plus}`) to build. Some ports need a specific, old GCC version, some require modern, recent versions. It is typically set to `yes` (means always use stable, modern GCC from ports per `GCC_DEFAULT` in [.filename]#Mk/bsd.default-versions.mk#). This is also the default value. The exact version can also be specified, with a value such as `10`. GCC from the base system is used when it satisfies the requested version, otherwise an appropriate compiler is built from ports, and `CC` and `CXX` are adjusted accordingly. The `:build` argument following the version specifier adds only a build time dependency to the port. For example: [example] ==== [.programlisting] .... USE_GCC=yes # port requires a current version of GCC USE_GCC=11:build # port requires GCC 11 at build time only .... ==== [NOTE] ==== `USE_GCC=any` is deprecated and should not be used in new ports ==== |=== Variables related to gmake and [.filename]#configure# are described in crossref:special[building,Building Mechanisms], while autoconf, automake and libtool are described in crossref:special[using-autotools,Using GNU Autotools]. Perl related variables are described in crossref:special[using-perl,Using Perl]. X11 variables are listed in crossref:special[using-x11,Using X11]. crossref:special[using-gnome,Using Gnome] deals with GNOME and crossref:special[using-kde,Using KDE] with KDE related variables. -crossref:special[using-java,Using Java] documents Java variables, while crossref:special[using-php,Web Applications, Apache and PHP] contains information on Apache, PHP and PEAR modules. +crossref:special[using-java,Using Java] documents Java variables, while crossref:special[using-php,"Web Applications, Apache and PHP"] contains information on Apache, PHP and PEAR modules. Python is discussed in crossref:special[using-python,Using Python], while Ruby in crossref:uses[uses-ruby,Ruby]. crossref:special[using-sdl,Using SDL] provides variables used for SDL applications and finally, crossref:special[using-xfce,Using Xfce] contains information on Xfce. [[makefile-version-dependency]] === Minimal Version of a Dependency A minimal version of a dependency can be specified in any `*_DEPENDS` except `LIB_DEPENDS` using this syntax: [.programlisting] .... p5-Spiffy>=0.26:devel/p5-Spiffy .... The first field contains a dependent package name, which must match the entry in the package database, a comparison sign, and a package version. The dependency is satisfied if p5-Spiffy-0.26 or newer is installed on the machine. [[makefile-note-on-dependencies]] === Notes on Dependencies As mentioned above, the default target to call when a dependency is required is `DEPENDS_TARGET`. It defaults to `install`. This is a user variable; it is never defined in a port's [.filename]#Makefile#. If the port needs a special way to handle a dependency, use the `:target` part of `*_DEPENDS` instead of redefining `DEPENDS_TARGET`. When running `make clean`, the port dependencies are automatically cleaned too. If this is not desirable, define `NOCLEANDEPENDS` in the environment. This may be particularly desirable if the port has something that takes a long time to rebuild in its dependency list, such as KDE, GNOME or Mozilla. To depend on another port unconditionally, use the variable `${NONEXISTENT}` as the first field of `BUILD_DEPENDS` or `RUN_DEPENDS`. Use this only when the source of the other port is needed. Compilation time can be saved by specifying the target too. For instance [.programlisting] .... BUILD_DEPENDS= ${NONEXISTENT}:graphics/jpeg:extract .... will always descend to the `jpeg` port and extract it. [[makefile-circular-dependencies]] === Circular Dependencies Are Fatal [IMPORTANT] ==== Do not introduce any circular dependencies into the ports tree! ==== The ports building technology does not tolerate circular dependencies. If one is introduced, someone, somewhere in the world, will have their FreeBSD installation broken almost immediately, with many others quickly to follow. These can really be hard to detect. If in doubt, before making that change, make sure to run: `cd /usr/ports; make index`. That process can be quite slow on older machines, but it may be able to save a large number of people, including yourself, a lot of grief in the process. [[makefile-automatic-dependencies]] === Problems Caused by Automatic Dependencies Dependencies must be declared either explicitly or by using the crossref:makefiles[makefile-options,OPTIONS framework]. Using other methods like automatic detection complicates indexing, which causes problems for port and package management. [[makefile-automatic-dependencies-bad]] .Wrong Declaration of an Optional Dependency [example] ==== [.programlisting] .... .include .if exists(${LOCALBASE}/bin/foo) LIB_DEPENDS= libbar.so:foo/bar .endif .... ==== The problem with trying to automatically add dependencies is that files and settings outside an individual port can change at any time. For example: an index is built, then a batch of ports are installed. But one of the ports installs the tested file. The index is now incorrect, because an installed port unexpectedly has a new dependency. The index may still be wrong even after rebuilding if other ports also determine their need for dependencies based on the existence of other files. [[makefile-automatic-dependencies-good]] .Correct Declaration of an Optional Dependency [example] ==== [.programlisting] .... OPTIONS_DEFINE= BAR BAR_DESC= Calling cellphones via bar BAR_LIB_DEPENDS= libbar.so:foo/bar .... ==== Testing option variables is the correct method. It will not cause inconsistencies in the index of a batch of ports, provided the options were defined prior to the index build. Simple scripts can then be used to automate the building, installation, and updating of these ports and their packages. [[makefile-masterdir]] == Slave Ports and `MASTERDIR` If the port needs to build slightly different versions of packages by having a variable (for instance, resolution, or paper size) take different values, create one subdirectory per package to make it easier for users to see what to do, but try to share as many files as possible between ports. Typically, by using variables cleverly, only a very short [.filename]#Makefile# is needed in all but one of the directories. In the sole [.filename]#Makefile#, use `MASTERDIR` to specify the directory where the rest of the files are. Also, use a variable as part of crossref:makefiles[porting-pkgname,`PKGNAMESUFFIX`] so the packages will have different names. This will be best demonstrated by an example. This is part of [.filename]#print/pkfonts300/Makefile#; [.programlisting] .... PORTNAME= pkfonts${RESOLUTION} PORTVERSION= 1.0 DISTFILES= pk${RESOLUTION}.tar.gz PLIST= ${PKGDIR}/pkg-plist.${RESOLUTION} .if !defined(RESOLUTION) RESOLUTION= 300 .else .if ${RESOLUTION} != 118 && ${RESOLUTION} != 240 && \ ${RESOLUTION} != 300 && ${RESOLUTION} != 360 && \ ${RESOLUTION} != 400 && ${RESOLUTION} != 600 .BEGIN: @${ECHO_MSG} "Error: invalid value for RESOLUTION: \"${RESOLUTION}\"" @${ECHO_MSG} "Possible values are: 118, 240, 300, 360, 400 and 600." @${FALSE} .endif .endif .... package:print/pkfonts300[] also has all the regular patches, package files, etc. Running `make` there, it will take the default value for the resolution (300) and build the port normally. As for other resolutions, this is the _entire_ [.filename]#print/pkfonts360/Makefile#: [.programlisting] .... RESOLUTION= 360 MASTERDIR= ${.CURDIR}/../pkfonts300 .include "${MASTERDIR}/Makefile" .... ([.filename]#print/pkfonts118/Makefile#, [.filename]#print/pkfonts600/Makefile#, and all the other are similar). `MASTERDIR` definition tells [.filename]#bsd.port.mk# that the regular set of subdirectories like `FILESDIR` and `SCRIPTDIR` are to be found under [.filename]#pkfonts300#. The `RESOLUTION=360` line will override the `RESOLUTION=300` line in [.filename]#pkfonts300/Makefile# and the port will be built with resolution set to 360. [[makefile-manpages]] == Man Pages If the port anchors its man tree somewhere other than `PREFIX`, use `MANDIRS` to specify those directories. Note that the files corresponding to manual pages must be placed in [.filename]#pkg-plist# along with the rest of the files. The purpose of `MANDIRS` is to enable automatic compression of manual pages, therefore the file names are suffixed with [.filename]#.gz#. [[makefile-info]] == Info Files If the package needs to install GNU info files, list them in `INFO` (without the trailing `.info`), one entry per document. These files are assumed to be installed to [.filename]#PREFIX/INFO_PATH#. Change `INFO_PATH` if the package uses a different location. However, this is not recommended. These entries contain just the path relative to [.filename]#PREFIX/INFO_PATH#. For example, package:lang/gcc34[] installs info files to [.filename]#PREFIX/INFO_PATH/gcc34#, and `INFO` will be something like this: [.programlisting] .... INFO= gcc34/cpp gcc34/cppinternals gcc34/g77 ... .... Appropriate installation/de-installation code will be automatically added to the temporary [.filename]#pkg-plist# before package registration. [[makefile-options]] == Makefile Options Many applications can be built with optional or differing configurations. Examples include choice of natural (human) language, GUI versus command-line, or type of database to support. Users may need a different configuration than the default, so the ports system provides hooks the port author can use to control which variant will be built. Supporting these options properly will make users happy, and effectively provide two or more ports for the price of one. [[makefile-options-options]] === `OPTIONS` [[makefile-options-background]] ==== Background `OPTIONS_*` give the user installing the port a dialog showing the available options, and then saves those options to [.filename]#${PORT_DBDIR}/${OPTIONS_NAME}/options#. The next time the port is built, the options are reused. `PORT_DBDIR` defaults to [.filename]#/var/db/ports#. `OPTIONS_NAME` is to the port origin with an underscore as the space separator, for example, for package:dns/bind99[] it will be `dns_bind99`. When the user runs `make config` (or runs `make build` for the first time), the framework checks for [.filename]#${PORT_DBDIR}/${OPTIONS_NAME}/options#. If that file does not exist, the values of `OPTIONS_*` are used, and a dialog box is displayed where the options can be enabled or disabled. Then [.filename]#options# is saved and the configured variables are used when building the port. If a new version of the port adds new `OPTIONS`, the dialog will be presented to the user with the saved values of old `OPTIONS` prefilled. `make showconfig` shows the saved configuration. Use `make rmconfig` to remove the saved configuration. [[makefile-options-syntax]] ==== Syntax `OPTIONS_DEFINE` contains a list of `OPTIONS` to be used. These are independent of each other and are not grouped: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 .... Once defined, `OPTIONS` are described (optional, but strongly recommended): [.programlisting] .... OPT1_DESC= Describe OPT1 OPT2_DESC= Describe OPT2 OPT3_DESC= Describe OPT3 OPT4_DESC= Describe OPT4 OPT5_DESC= Describe OPT5 OPT6_DESC= Describe OPT6 .... [.filename]#ports/Mk/bsd.options.desc.mk# has descriptions for many common `OPTIONS`. While often useful, override them if the description is insufficient for the port. [TIP] ==== When describing options, view it from the perspective of the user: "What functionality does it change?" and "Why would I want to enable this?" Do not just repeat the name. For example, describing the `NLS` option as "include NLS support" does not help the user, who can already see the option name but may not know what it means. Describing it as "Native Language Support via gettext utilities" is much more helpful. ==== [IMPORTANT] ==== Option names are always in all uppercase. They cannot use mixed case or lowercase. ==== `OPTIONS` can be grouped as radio choices, where only one choice from each group is allowed: [.programlisting] .... OPTIONS_SINGLE= SG1 OPTIONS_SINGLE_SG1= OPT3 OPT4 .... [WARNING] ==== There _must_ be one of each `OPTIONS_SINGLE` group selected at all times for the options to be valid. One option of each group _must_ be added to `OPTIONS_DEFAULT`. ==== `OPTIONS` can be grouped as radio choices, where none or only one choice from each group is allowed: [.programlisting] .... OPTIONS_RADIO= RG1 OPTIONS_RADIO_RG1= OPT7 OPT8 .... `OPTIONS` can also be grouped as "multiple-choice" lists, where _at least one_ option must be enabled: [.programlisting] .... OPTIONS_MULTI= MG1 OPTIONS_MULTI_MG1= OPT5 OPT6 .... `OPTIONS` can also be grouped as "multiple-choice" lists, where none or any option can be enabled: [.programlisting] .... OPTIONS_GROUP= GG1 OPTIONS_GROUP_GG1= OPT9 OPT10 .... `OPTIONS` are unset by default, unless they are listed in `OPTIONS_DEFAULT`: [.programlisting] .... OPTIONS_DEFAULT= OPT1 OPT3 OPT6 .... `OPTIONS` definitions must appear before the inclusion of [.filename]#bsd.port.options.mk#. `PORT_OPTIONS` values can only be tested after the inclusion of [.filename]#bsd.port.options.mk#. Inclusion of [.filename]#bsd.port.pre.mk# can be used instead, too, and is still widely used in ports written before the introduction of [.filename]#bsd.port.options.mk#. But be aware that some variables will not work as expected after the inclusion of [.filename]#bsd.port.pre.mk#, typically some `USE_*` flags. [[ports-options-simple-use]] .Simple Use of `OPTIONS` [example] ==== [.programlisting] .... OPTIONS_DEFINE= FOO BAR OPTIONS_DEFAULT=FOO FOO_DESC= Option foo support BAR_DESC= Feature bar support # Will add --with-foo / --without-foo FOO_CONFIGURE_WITH= foo BAR_RUN_DEPENDS= bar:bar/bar .include .... ==== [[ports-options-check-unset]] .Check for Unset Port `OPTIONS` [example] ==== [.programlisting] .... .if ! ${PORT_OPTIONS:MEXAMPLES} CONFIGURE_ARGS+=--without-examples .endif .... The form shown above is discouraged. The preferred method is using a configure knob to really enable and disable the feature to match the option: [.programlisting] .... # Will add --with-examples / --without-examples EXAMPLES_CONFIGURE_WITH= examples .... ==== [[ports-options-practical-use]] .Practical Use of `OPTIONS` [example] ==== [.programlisting] .... OPTIONS_DEFINE= EXAMPLES OPTIONS_DEFAULT= PGSQL LDAP SSL OPTIONS_SINGLE= BACKEND OPTIONS_SINGLE_BACKEND= MYSQL PGSQL BDB OPTIONS_MULTI= AUTH OPTIONS_MULTI_AUTH= LDAP PAM SSL EXAMPLES_DESC= Install extra examples MYSQL_DESC= Use MySQL as backend PGSQL_DESC= Use PostgreSQL as backend BDB_DESC= Use Berkeley DB as backend LDAP_DESC= Build with LDAP authentication support PAM_DESC= Build with PAM support SSL_DESC= Build with OpenSSL support # Will add USE_PGSQL=yes PGSQL_USE= pgsql=yes # Will add --enable-postgres / --disable-postgres PGSQL_CONFIGURE_ENABLE= postgres ICU_LIB_DEPENDS= libicuuc.so:devel/icu # Will add --with-examples / --without-examples EXAMPLES_CONFIGURE_WITH= examples # Check other OPTIONS .include .... ==== [[makefile-options-default]] ==== Default Options These options are always on by default. * `DOCS` - build and install documentation. * `NLS` - Native Language Support. * `EXAMPLES` - build and install examples. * `IPV6` - IPv6 protocol support. [NOTE] ==== There is no need to add these to `OPTIONS_DEFAULT`. To have them active, and show up in the options selection dialog, however, they must be added to `OPTIONS_DEFINE`. ==== [[makefile-options-auto-activation]] === Feature Auto-Activation When using a GNU configure script, keep an eye on which optional features are activated by auto-detection. Explicitly disable optional features that are not needed by adding `--without-xxx` or `--disable-xxx` in `CONFIGURE_ARGS`. [[makefile-options-auto-activation-bad]] .Wrong Handling of an Option [example] ==== [.programlisting] .... .if ${PORT_OPTIONS:MFOO} LIB_DEPENDS+= libfoo.so:devel/foo CONFIGURE_ARGS+= --enable-foo .endif .... ==== In the example above, imagine a library libfoo is installed on the system. The user does not want this application to use libfoo, so he toggled the option off in the `make config` dialog. But the application's configure script detects the library present in the system and includes its support in the resulting executable. Now when the user decides to remove libfoo from the system, the ports system does not protest (no dependency on libfoo was recorded) but the application breaks. [[makefile-options-auto-activation-good]] .Correct Handling of an Option [example] ==== [.programlisting] .... FOO_LIB_DEPENDS= libfoo.so:devel/foo # Will add --enable-foo / --disable-foo FOO_CONFIGURE_ENABLE= foo .... ==== [NOTE] ==== Under some circumstances, the shorthand conditional syntax can cause problems with complex constructs. The errors are usually `Malformed conditional`, an alternative syntax can be used. [.programlisting] .... .if !empty(VARIABLE:MVALUE) .... as an alternative to [.programlisting] .... .if ${VARIABLE:MVALUE} .... ==== [[options-helpers]] === Options Helpers There are some macros to help simplify conditional values which differ based on the options set. For easier access, a comprehensive list is provided: `PLIST_SUB`, `SUB_LIST`:: For automatic `%%_OPT_%%` and `%%NO__OPT__%%` generation, see crossref:makefiles[options_sub, `OPTIONS_SUB`]. + -For more complex usage, see crossref:makefiles[options-variables, Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF`]. +For more complex usage, see crossref:makefiles[options-variables,"Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF`"]. `CONFIGURE_ARGS`:: For `--enable-_x_` and `--disable-_x_`, see crossref:makefiles[options-configure_enable, `OPT_CONFIGURE_ENABLE`]. + For `--with-_x_` and `--without-_x_`, see crossref:makefiles[options-configure_with, `OPT_CONFIGURE_WITH`]. + For all other cases, see crossref:makefiles[options-configure_on, `OPT_CONFIGURE_ON` and `OPT_CONFIGURE_OFF`]. `CMAKE_ARGS`:: For arguments that are booleans (`on`, `off`, `true`, `false`, `0`, `1`) see crossref:makefiles[options-cmake_bool, `OPT_CMAKE_BOOL` and `OPT_CMAKE_BOOL_OFF`]. + For all other cases, see crossref:makefiles[options-cmake_on, `OPT_CMAKE_ON` and `OPT_CMAKE_OFF`]. `MESON_ARGS`:: For arguments that take `true` or `false`, see crossref:makefiles[options-meson_true, `OPT_MESON_TRUE` and `OPT_MESON_FALSE`]. + For arguments that take `yes` or `no`, use crossref:makefiles[options-meson_yes, `OPT_MESON_YES` and `OPT_MESON_NO`]. + For arguments that take `enabled` or `disabled`, see crossref:makefiles[options-meson_enabled, `OPT_MESON_ENABLED` and `OPT_MESON_DISABLED`]. + For all other cases, use crossref:makefiles[options-meson_on, `OPT_MESON_ON` and `OPT_MESON_OFF`]. `QMAKE_ARGS`:: See crossref:makefiles[options-qmake_on, `OPT_QMAKE_ON` and `OPT_QMAKE_OFF`]. `USE_*`:: See crossref:makefiles[options-use, `OPT_USE` and `OPT_USE_OFF`]. `*_DEPENDS`:: -See crossref:makefiles[options-dependencies, Dependencies, `OPT_DEPTYPE` and `OPT_DEPTYPE_OFF`]. +See crossref:makefiles[options-dependencies,"Dependencies, `OPT_DEPTYPE` and `OPT_DEPTYPE_OFF`"]. `*` (Any variable):: The most used variables have direct helpers, see -crossref:makefiles[options-variables, Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF`]. +crossref:makefiles[options-variables,"Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF`"]. + For any variable without a specific helper, see crossref:makefiles[options-vars, `OPT_VARS` and `OPT_VARS_OFF`]. Options dependencies:: When an option need another option to work, see crossref:makefiles[options-implies, `OPT_IMPLIES`]. Options conflicts:: When an option cannot work if another is also enabled, see crossref:makefiles[options-prevents, `OPT_PREVENTS` and `OPT_PREVENTS_MSG`]. Build targets:: When an option need some extra processing, see -crossref:makefiles[options-targets, Additional Build Targets, `_target_-_OPT_-on` and `_target_-_OPT_-off`]. +crossref:makefiles[options-targets,"Additional Build Targets, `_target_-_OPT_-on` and `_target_-_OPT_-off`"]. [[options_sub]] ==== `OPTIONS_SUB` If `OPTIONS_SUB` is set to `yes` then each of the options added to `OPTIONS_DEFINE` will be added to `PLIST_SUB` and `SUB_LIST`, for example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPTIONS_SUB= yes .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} PLIST_SUB+= OPT1="" NO_OPT1="@comment " SUB_LIST+= OPT1="" NO_OPT1="@comment " .else PLIST_SUB+= OPT1="@comment " NO_OPT1="" SUB_LIST+= OPT1="@comment " NO_OPT1="" .endif .... [NOTE] ==== The value of `OPTIONS_SUB` is ignored. Setting it to any value will add `PLIST_SUB` and `SUB_LIST` entries for _all_ options. ==== [[options-use]] ==== `OPT_USE` and `OPT_USE_OFF` When option _OPT_ is selected, for each `_key=value_` pair in ``OPT_USE``, _value_ is appended to the corresponding `USE_KEY`. If _value_ has spaces in it, replace them with commas and they will be changed back to spaces during processing. `OPT_USE_OFF` works the same way, but when `OPT` is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_USES= xorg OPT1_USE= mysql=yes xorg=x11,xextproto,xext,xrandr OPT1_USE_OFF= openssl=yes .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} USE_MYSQL= yes USES+= xorg USE_XORG= x11 xextproto xext xrandr .else USE_OPENSSL= yes .endif .... [[options-configure-helpers]] ==== `CONFIGURE_ARGS` Helpers [[options-configure_enable]] ===== `OPT_CONFIGURE_ENABLE` When option _OPT_ is selected, for each _entry_ in `OPT_CONFIGURE_ENABLE` then `--enable-_entry_` is appended to `CONFIGURE_ARGS`. When option _OPT_ is _not_ selected, `--disable-_entry_` is appended to `CONFIGURE_ARGS`. An optional argument can be specified with an `=` symbol. This argument is only appended to the `--enable-_entry_` configure option. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 OPT1_CONFIGURE_ENABLE= test1 test2 OPT2_CONFIGURE_ENABLE= test2=exhaustive .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} CONFIGURE_ARGS+= --enable-test1 --enable-test2 .else CONFIGURE_ARGS+= --disable-test1 --disable-test2 .endif .if ${PORT_OPTIONS:MOPT2} CONFIGURE_ARGS+= --enable-test2=exhaustive .else CONFIGURE_ARGS+= --disable-test2 .endif .... [[options-configure_with]] ===== `OPT_CONFIGURE_WITH` When option _OPT_ is selected, for each _entry_ in `OPT_CONFIGURE_WITH` then `--with-_entry_` is appended to `CONFIGURE_ARGS`. When option _OPT_ is _not_ selected, `--without-_entry_` is appended to `CONFIGURE_ARGS`. An optional argument can be specified with an `=` symbol. This argument is only appended to the `--with-_entry_` configure option. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 OPT1_CONFIGURE_WITH= test1 OPT2_CONFIGURE_WITH= test2=exhaustive .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 .include .if ${PORT_OPTIONS:MOPT1} CONFIGURE_ARGS+= --with-test1 .else CONFIGURE_ARGS+= --without-test1 .endif .if ${PORT_OPTIONS:MOPT2} CONFIGURE_ARGS+= --with-test2=exhaustive .else CONFIGURE_ARGS+= --without-test2 .endif .... [[options-configure_on]] ===== `OPT_CONFIGURE_ON` and `OPT_CONFIGURE_OFF` When option _OPT_ is selected, the value of `OPT_CONFIGURE_ON`, if defined, is appended to `CONFIGURE_ARGS`. `OPT_CONFIGURE_OFF` works the same way, but when `OPT` is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_CONFIGURE_ON= --add-test OPT1_CONFIGURE_OFF= --no-test .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} CONFIGURE_ARGS+= --add-test .else CONFIGURE_ARGS+= --no-test .endif .... [TIP] ==== Most of the time, the helpers in crossref:makefiles[options-configure_enable, `OPT_CONFIGURE_ENABLE`] and crossref:makefiles[options-configure_with, `OPT_CONFIGURE_WITH`] provide a shorter and more comprehensive functionality. ==== [[options-cmake-helpers]] ==== `CMAKE_ARGS` Helpers [[options-cmake_on]] ===== `OPT_CMAKE_ON` and `OPT_CMAKE_OFF` When option _OPT_ is selected, the value of `OPT_CMAKE_ON`, if defined, is appended to `CMAKE_ARGS`. `OPT_CMAKE_OFF` works the same way, but when `OPT` is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_CMAKE_ON= -DTEST:BOOL=true -DDEBUG:BOOL=true OPT1_CMAKE_OFF= -DOPTIMIZE:BOOL=true .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} CMAKE_ARGS+= -DTEST:BOOL=true -DDEBUG:BOOL=true .else CMAKE_ARGS+= -DOPTIMIZE:BOOL=true .endif .... [TIP] ==== See crossref:makefiles[options-cmake_bool, `OPT_CMAKE_BOOL` and `OPT_CMAKE_BOOL_OFF`] for a shorter helper when the value is boolean. ==== [[options-cmake_bool]] ===== `OPT_CMAKE_BOOL` and `OPT_CMAKE_BOOL_OFF` When option _OPT_ is selected, for each _entry_ in `OPT_CMAKE_BOOL` then `-D__entry__:BOOL=true` is appended to `CMAKE_ARGS`. When option _OPT_ is _not_ selected, `-D__entry__:BOOL=false` is appended to `CMAKE_ARGS`. `OPT_CMAKE_BOOL_OFF` is the opposite, `-D__entry__:BOOL=false` is appended to `CMAKE_ARGS` when the option is selected, and `-D__entry__:BOOL=true` when the option is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_CMAKE_BOOL= TEST DEBUG OPT1_CMAKE_BOOL_OFF= OPTIMIZE .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} CMAKE_ARGS+= -DTEST:BOOL=true -DDEBUG:BOOL=true \ -DOPTIMIZE:BOOL=false .else CMAKE_ARGS+= -DTEST:BOOL=false -DDEBUG:BOOL=false \ -DOPTIMIZE:BOOL=true .endif .... [[options-meson-helpers]] ==== `MESON_ARGS` Helpers [[options-meson_on]] ===== `OPT_MESON_ON` and `OPT_MESON_OFF` When option _OPT_ is selected, the value of `OPT_MESON_ON`, if defined, is appended to `MESON_ARGS`. `OPT_MESON_OFF` works the same way, but when `OPT` is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_MESON_ON= -Dopt=1 OPT1_MESON_OFF= -Dopt=2 .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} MESON_ARGS+= -Dopt=1 .else MESON_ARGS+= -Dopt=2 .endif .... [[options-meson_true]] ===== `OPT_MESON_TRUE` and `OPT_MESON_FALSE` When option _OPT_ is selected, for each _entry_ in `OPT_MESON_TRUE` then `-D__entry__=true` is appended to `MESON_ARGS`. When option _OPT_ is _not_ selected, `-D__entry__=false` is appended to `MESON_ARGS`. `OPT_MESON_FALSE` is the opposite, `-D__entry__=false` is appended to `MESON_ARGS` when the option is selected, and `-D__entry__=true` when the option is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_MESON_TRUE= test debug OPT1_MESON_FALSE= optimize .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} MESON_ARGS+= -Dtest=true -Ddebug=true \ -Doptimize=false .else MESON_ARGS+= -Dtest=false -Ddebug=false \ -Doptimize=true .endif .... [[options-meson_yes]] ===== `OPT_MESON_YES` and `OPT_MESON_NO` When option _OPT_ is selected, for each _entry_ in `OPT_MESON_YES` then `-D__entry__=yes` is appended to `MESON_ARGS`. When option _OPT_ is _not_ selected, `-D__entry__=no` is appended to `MESON_ARGS`. `OPT_MESON_NO` is the opposite, `-D__entry__=no` is appended to `MESON_ARGS` when the option is selected, and `-D__entry__=yes` when the option is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_MESON_YES= test debug OPT1_MESON_NO= optimize .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} MESON_ARGS+= -Dtest=yes -Ddebug=yes \ -Doptimize=no .else MESON_ARGS+= -Dtest=no -Ddebug=no \ -Doptimize=yes .endif .... [[options-meson_enabled]] ===== `OPT_MESON_ENABLED` and `OPT_MESON_DISABLED` When option _OPT_ is selected, for each _entry_ in `OPT_MESON_ENABLED` then `-D__entry__=enabled` is appended to `MESON_ARGS`. When option _OPT_ is _not_ selected, `-D__entry__=disabled` is appended to `MESON_ARGS`. `OPT_MESON_DISABLED` is the opposite, `-D__entry__=disabled` is appended to `MESON_ARGS` when the option is selected, and `-D__entry__=enabled` when the option is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_MESON_ENABLED= test OPT1_MESON_DISABLED= debug .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} MESON_ARGS+= -Dtest=enabled -Ddebug=disabled .else MESON_ARGS+= -Dtest=disabled -Ddebug=enabled .endif .... [[options-qmake_on]] ==== `OPT_QMAKE_ON` and `OPT_QMAKE_OFF` When option _OPT_ is selected, the value of `OPT_QMAKE_ON`, if defined, is appended to `QMAKE_ARGS`. `OPT_QMAKE_OFF` works the same way, but when `OPT` is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_QMAKE_ON= -DTEST:BOOL=true OPT1_QMAKE_OFF= -DPRODUCTION:BOOL=true .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} QMAKE_ARGS+= -DTEST:BOOL=true .else QMAKE_ARGS+= -DPRODUCTION:BOOL=true .endif .... [[options-implies]] ==== `OPT_IMPLIES` Provides a way to add dependencies between options. When _OPT_ is selected, all the options listed in this variable will be selected too. Using the crossref:makefiles[options-configure_enable,`OPT_CONFIGURE_ENABLE`] described earlier to illustrate: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 OPT1_IMPLIES= OPT2 OPT1_CONFIGURE_ENABLE= opt1 OPT2_CONFIGURE_ENABLE= opt2 .... Is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 .include .if ${PORT_OPTIONS:MOPT1} CONFIGURE_ARGS+= --enable-opt1 .else CONFIGURE_ARGS+= --disable-opt1 .endif .if ${PORT_OPTIONS:MOPT2} || ${PORT_OPTIONS:MOPT1} CONFIGURE_ARGS+= --enable-opt2 .else CONFIGURE_ARGS+= --disable-opt2 .endif .... [[options-implies-ex1]] .Simple Use of `OPT_IMPLIES` [example] ==== This port has a `X11` option, and a `GNOME` option that needs the `X11` option to be selected to build. [.programlisting] .... OPTIONS_DEFINE= X11 GNOME OPTIONS_DEFAULT= X11 X11_USES= xorg X11_USE= xorg=xi,xextproto GNOME_USE= gnome=gtk30 GNOME_IMPLIES= X11 .... ==== [[options-prevents]] ==== `OPT_PREVENTS` and `OPT_PREVENTS_MSG` Provides a way to add conflicts between options. When _OPT_ is selected, all the options listed in `OPT_PREVENTS` must be un-selected. If `OPT_PREVENTS_MSG` is set and a conflict is triggered, its content will be shown explaining why they conflict. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 OPT1_PREVENTS= OPT2 OPT1_PREVENTS_MSG= OPT1 and OPT2 enable conflicting options .... Is roughly equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 .include .if ${PORT_OPTIONS:MOPT2} && ${PORT_OPTIONS:MOPT1} BROKEN= Option OPT1 conflicts with OPT2 (select only one) .endif .... The only difference is that the first one will write an error after running `make config`, suggesting changing the selected options. [[options-prevents-ex1]] .Simple Use of `OPT_PREVENTS` [example] ==== This port has `X509` and `SCTP` options. Both options add patches, but the patches conflict with each other, so they cannot be selected at the same time. [.programlisting] .... OPTIONS_DEFINE= X509 SCTP SCTP_PATCHFILES= ${PORTNAME}-6.8p1-sctp-2573.patch.gz:-p1 SCTP_CONFIGURE_WITH= sctp X509_PATCH_SITES= http://www.roumenpetrov.info/openssh/x509/:x509 X509_PATCHFILES= ${PORTNAME}-7.0p1+x509-8.5.diff.gz:-p1:x509 X509_PREVENTS= SCTP X509_PREVENTS_MSG= X509 and SCTP patches conflict .... ==== [[options-vars]] ==== `OPT_VARS` and `OPT_VARS_OFF` Provides a generic way to set and append to variables. [WARNING] ==== Before using `OPT_VARS` and `OPT_VARS_OFF`, see if there is already a more -specific helper available in crossref:makefiles[options-variables, Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF`]. +specific helper available in crossref:makefiles[options-variables,"Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF`"]. ==== When option _OPT_ is selected, and `OPT_VARS` defined, `_key_=_value_` and `_key_+=_value_` pairs are evaluated from `OPT_VARS`. An `=` cause the existing value of `KEY` to be overwritten, an `+=` appends to the value. `OPT_VARS_OFF` works the same way, but when `OPT` is _not_ selected. [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 OPT3 OPT1_VARS= also_build+=bin1 OPT2_VARS= also_build+=bin2 OPT3_VARS= bin3_build=yes OPT3_VARS_OFF= bin3_build=no MAKE_ARGS= ALSO_BUILD="${ALSO_BUILD}" BIN3_BUILD="${BIN3_BUILD}" .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT2 OPT3 MAKE_ARGS= ALSO_BUILD="${ALSO_BUILD}" BIN3_BUILD="${BIN3_BUILD}" .include .if ${PORT_OPTIONS:MOPT1} ALSO_BUILD+= bin1 .endif .if ${PORT_OPTIONS:MOPT2} ALSO_BUILD+= bin2 .endif .if ${PORT_OPTIONS:MOPT3} BIN3_BUILD= yes .else BIN3_BUILD= no .endif .... [IMPORTANT] ==== Values containing whitespace must be enclosed in quotes: [.programlisting] .... OPT_VARS= foo="bar baz" .... This is due to the way man:make[1] variable expansion deals with whitespace. When `OPT_VARS= foo=bar baz` is expanded, the variable ends up containing two strings, `foo=bar` and `baz`. But the submitter probably intended there to be only one string, `foo=bar baz`. Quoting the value prevents whitespace from being used as a delimiter. Also, _do not_ add extra spaces after the `_var_=` sign and before the value, it would also be split into two strings. _This will not work_: [.programlisting] .... OPT_VARS= foo= bar .... ==== [[options-dependencies]] ==== Dependencies, `OPT_DEPTYPE` and `OPT_DEPTYPE_OFF` For any of these dependency types: * `PKG_DEPENDS` * `EXTRACT_DEPENDS` * `PATCH_DEPENDS` * `FETCH_DEPENDS` * `BUILD_DEPENDS` * `LIB_DEPENDS` * `RUN_DEPENDS` When option _OPT_ is selected, the value of `OPT_DEPTYPE`, if defined, is appended to `DEPTYPE`. `OPT_DEPTYPE_OFF` works the same, but when `OPT` is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_LIB_DEPENDS= liba.so:devel/a OPT1_LIB_DEPENDS_OFF= libb.so:devel/b .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} LIB_DEPENDS+= liba.so:devel/a .else LIB_DEPENDS+= libb.so:devel/b .endif .... [[options-variables]] ==== Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF` For any of these variables: * `ALL_TARGET` * `BINARY_ALIAS` * `BROKEN` * `CATEGORIES` * `CFLAGS` * `CONFIGURE_ENV` * `CONFLICTS` * `CONFLICTS_BUILD` * `CONFLICTS_INSTALL` * `CPPFLAGS` * `CXXFLAGS` * `DESKTOP_ENTRIES` * `DISTFILES` * `EXTRACT_ONLY` * `EXTRA_PATCHES` * `GH_ACCOUNT` * `GH_PROJECT` * `GH_SUBDIR` * `GH_TAGNAME` * `GH_TUPLE` * `GL_ACCOUNT` * `GL_COMMIT` * `GL_PROJECT` * `GL_SITE` * `GL_SUBDIR` * `GL_TUPLE` * `IGNORE` * `INFO` * `INSTALL_TARGET` * `LDFLAGS` * `LIBS` * `MAKE_ARGS` * `MAKE_ENV` * `MASTER_SITES` * `PATCHFILES` * `PATCH_SITES` * `PLIST_DIRS` * `PLIST_FILES` * `PLIST_SUB` * `PORTDOCS` * `PORTEXAMPLES` * `SUB_FILES` * `SUB_LIST` * `TEST_TARGET` * `USES` When option _OPT_ is selected, the value of `OPT_ABOVEVARIABLE`, if defined, is appended to `_ABOVEVARIABLE_`. `OPT_ABOVEVARIABLE_OFF` works the same way, but when `OPT` is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 OPT1_USES= gmake OPT1_CFLAGS_OFF= -DTEST .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include .if ${PORT_OPTIONS:MOPT1} USES+= gmake .else CFLAGS+= -DTEST .endif .... [NOTE] ==== Some variables are not in this list, in particular `PKGNAMEPREFIX` and `PKGNAMESUFFIX`. This is intentional. A port _must not_ change its name when its option set changes. ==== [WARNING] ==== Some of these variables, at least `ALL_TARGET`, `DISTFILES` and `INSTALL_TARGET`, have their default values set _after_ the options are processed. With these lines in the [.filename]#Makefile#: [.programlisting] .... ALL_TARGET= all DOCS_ALL_TARGET= doc .... If the `DOCS` option is enabled, `ALL_TARGET` will have a final value of `all doc`; if the option is disabled, it would have a value of `all`. With only the options helper line in the [.filename]#Makefile#: [.programlisting] .... DOCS_ALL_TARGET= doc .... If the `DOCS` option is enabled, `ALL_TARGET` will have a final value of `doc`; if the option is disabled, it would have a value of `all`. ==== [[options-targets]] ==== Additional Build Targets, `_target_-_OPT_-on` and `_target_-_OPT_-off` These [.filename]#Makefile# targets can accept optional extra build targets: * `pre-fetch` * `do-fetch` * `post-fetch` * `pre-extract` * `do-extract` * `post-extract` * `pre-patch` * `do-patch` * `post-patch` * `pre-configure` * `do-configure` * `post-configure` * `pre-build` * `do-build` * `post-build` * `pre-install` * `do-install` * `post-install` * `post-stage` * `pre-package` * `do-package` * `post-package` When option _OPT_ is selected, the target `_TARGET_-_OPT_-on`, if defined, is executed after `_TARGET_`. `_TARGET_-_OPT_-off` works the same way, but when `OPT` is _not_ selected. For example: [.programlisting] .... OPTIONS_DEFINE= OPT1 post-patch-OPT1-on: @${REINPLACE_CMD} -e '/opt1/s|/usr/bin/|${EXAMPLESDIR}/|' ${WRKSRC}/Makefile post-patch-OPT1-off: @${REINPLACE_CMD} -e '/opt1/s|/usr/bin/|${PREFIX}/bin/|' ${WRKSRC}/Makefile .... is equivalent to: [.programlisting] .... OPTIONS_DEFINE= OPT1 .include post-patch: .if ${PORT_OPTIONS:MOPT1} @${REINPLACE_CMD} -e '/opt1/s|/usr/bin/|${EXAMPLESDIR}/|' ${WRKSRC}/Makefile .else @${REINPLACE_CMD} -e '/opt1/s|/usr/bin/|${PREFIX}/bin/|' ${WRKSRC}/Makefile .endif .... [[makefile-wrkdir]] == Specifying the Working Directory Each port is extracted into a working directory, which must be writable. The ports system defaults to having `DISTFILES` unpack in to a directory called `${DISTNAME}`. In other words, if the [.filename]#Makefile# has: [.programlisting] .... PORTNAME= foo DISTVERSION= 1.0 .... then the port's distribution files contain a top-level directory, [.filename]#foo-1.0#, and the rest of the files are located under that directory. A number of variables can be overridden if that is not the case. [[makefile-wrksrc]] === `WRKSRC` The variable lists the name of the directory that is created when the application's distfiles are extracted. If our previous example extracted into a directory called [.filename]#foo# (and not [.filename]#foo-1.0#) write: [.programlisting] .... WRKSRC= ${WRKDIR}/foo .... or possibly [.programlisting] .... WRKSRC= ${WRKDIR}/${PORTNAME} .... [[makefile-wrksrc_subdir]] === `WRKSRC_SUBDIR` If the source files needed for the port are in a subdirectory of the extracted distribution file, set `WRKSRC_SUBDIR` to that directory. [.programlisting] .... WRKSRC_SUBDIR= src .... [[makefile-no_wrksubdir]] === `NO_WRKSUBDIR` If the port does not extract in to a subdirectory at all, then set `NO_WRKSUBDIR` to indicate that. [.programlisting] .... NO_WRKSUBDIR= yes .... [NOTE] ==== Because `WRKDIR` is the only directory that is supposed to be writable during the build, and is used to store many files recording the status of the build, the port's extraction will be forced into a subdirectory. ==== [[conflicts]] == Conflict Handling There are three different variables to register a conflict between packages and ports: `CONFLICTS`, `CONFLICTS_INSTALL` and `CONFLICTS_BUILD`. [NOTE] ==== -The conflict variables automatically set the variable `IGNORE`, which is more fully documented in crossref:porting-dads[dads-noinstall,Marking a Port Not Installable with `BROKEN`, `FORBIDDEN`, or `IGNORE`]. +The conflict variables automatically set the variable `IGNORE`, which is more fully documented in crossref:porting-dads[dads-noinstall,"Marking a Port Not Installable with `BROKEN`, `FORBIDDEN`, or `IGNORE`"]. ==== When removing one of several conflicting ports, it is advisable to retain `CONFLICTS` in those other ports for a few months to cater for users who only update once in a while. [[conclicts-conflicts_install]] `CONFLICTS_INSTALL`:: If the package cannot coexist with other packages (because of file conflicts, runtime incompatibilities, etc.). `CONFLICTS_INSTALL` check is done after the build stage and prior to the install stage. [[conclicts-conflicts_build]] `CONFLICTS_BUILD`:: If the port cannot be built when other specific ports are already installed. Build conflicts are not recorded in the resulting package. [[conclicts-conflicts]] `CONFLICTS`:: If the port cannot be built if a certain port is already installed and the resulting package cannot coexist with the other package. `CONFLICTS` check is done prior to the build stage and prior to the install stage. Each space-separated item in the `CONFLICTS*` variable values is matched against packages except the one being built, using shell globbing rules. This allows listing all flavors of a port in a conflict list instead of having to take pains to exclude the flavor being built from that list. For example, if git-lite is installed, `CONFLICTS_INSTALL=git git-lite` would allow to perform: [source,shell] .... % make -C devel/git FLAVOR=lite all deinstall install .... But the following command would report a conflict, since the package base name installed is `git-lite`, while `git` would be built, but cannot be installed in addition to `git-lite`: [source,shell] .... % make -C devel/git FLAVOR=default all deinstall install .... Without that feature, the Makefile would need one `_flavor__CONFLICTS_INSTALL` for each flavor, listing every other flavor. The most common content of one of these variable is the package base of another port. The package base is the package name without the appended version, it can be obtained by running `make -V PKGBASE`. [[conflicts-ex1]] .Basic usage of `CONFLICTS*` [example] ==== package:dns/bind99[] cannot be installed if package:dns/bind910[] is present because they install same files. First gather the package base to use: [source,shell] .... % make -C dns/bind99 -V PKGBASE bind99 % make -C dns/bind910 -V PKGBASE bind910 .... Then add to the [.filename]#Makefile# of package:dns/bind99[]: [.programlisting] .... CONFLICTS_INSTALL= bind910 .... And add to the [.filename]#Makefile# of package:dns/bind910[]: [.programlisting] .... CONFLICTS_INSTALL= bind99 .... ==== Sometimes, only certain versions of another port are incompatible. When this is the case, use the full package name including the version. If necessary, use shell globs like `*` and `?` so that all necessary versions are matched. [[conflicts-ex2]] .Using `CONFLICTS*` With Globs. [example] ==== From versions from 2.0 and up-to 2.4.1_2, package:deskutils/gnotime[] used to install a bundled version of package:databases/qof[]. To reflect this past, the [.filename]#Makefile# of package:databases/qof[] contains: [.programlisting] .... CONFLICTS_INSTALL= gnotime-2.[0-3]* \ gnotime-2.4.0* gnotime-2.4.1 \ gnotime-2.4.1_[12] .... The first entry match versions `2.0` through `2.3`, the second all the revisions of `2.4.0`, the third the exact `2.4.1` version, and the last the first and second revisions of the `2.4.1` version. package:deskutils/gnotime[] does not have any conflicts line because its current version does not conflict with anything else. ==== The variable `DISABLE_CONFLICTS` may be temporarily set when making targets that are not affected by conflicts. The variable is not to be set in port Makefiles. [source,shell] .... % make -DDISABLE_CONFLICTS patch .... [[install]] == Installing Files [IMPORTANT] ==== The `install` phase is very important to the end user because it adds files to their system. All the additional commands run in the port [.filename]#Makefile#'s `*-install` targets should be echoed to the screen. _Do not_ silence these commands with `@` or `.SILENT`. ==== [[install-macros]] === `INSTALL_*` Macros Use the macros provided in [.filename]#bsd.port.mk# to ensure correct modes of files in the port's `*-install` targets. Set ownership directly in [.filename]#pkg-plist# with the corresponding entries, such as `@(_owner_,_group_,)`, `@owner _owner_`, and `@group _group_`. These operators work until overridden, or until the end of [.filename]#pkg-plist#, so remember to reset them after they are no longer needed. The default ownership is `root:wheel`. See crossref:plist[plist-keywords-base,Base Keywords] for more information. * `INSTALL_PROGRAM` is a command to install binary executables. * `INSTALL_SCRIPT` is a command to install executable scripts. * `INSTALL_LIB` is a command to install shared libraries (but not static libraries). * `INSTALL_KLD` is a command to install kernel loadable modules. Some architectures do not like having the modules stripped, so use this command instead of `INSTALL_PROGRAM`. * `INSTALL_DATA` is a command to install sharable data, including static libraries. * `INSTALL_MAN` is a command to install manpages and other documentation (it does not compress anything). These variables are set to the man:install[1] command with the appropriate flags for each situation. [IMPORTANT] ==== Do not use `INSTALL_LIB` to install static libraries, because stripping them renders them useless. Use `INSTALL_DATA` instead. ==== [[install-strip]] === Stripping Binaries and Shared Libraries Installed binaries should be stripped. Do not strip binaries manually unless absolutely required. The `INSTALL_PROGRAM` macro installs and strips a binary at the same time. The `INSTALL_LIB` macro does the same thing to shared libraries. When a file must be stripped, but neither `INSTALL_PROGRAM` nor `INSTALL_LIB` macros are desirable, `${STRIP_CMD}` strips the program or shared library. This is typically done within the `post-install` target. For example: [.programlisting] .... post-install: ${STRIP_CMD} ${STAGEDIR}${PREFIX}/bin/xdl .... When multiple files need to be stripped: [.programlisting] .... post-install: .for l in geometry media body track world ${STRIP_CMD} ${STAGEDIR}${PREFIX}/lib/lib${PORTNAME}-${l}.so.0 .endfor .... Use man:file[1] on a file to determine if it has been stripped. Binaries are reported by man:file[1] as `stripped`, or `not stripped`. Additionally, man:strip[1] will detect programs that have already been stripped and exit cleanly. [IMPORTANT] ==== When `WITH_DEBUG` is defined, elf files _must not_ be stripped. The variables (`STRIP_CMD`, `INSTALL_PROGRAM`, `INSTALL_LIB`, ...) and crossref:uses[uses,`USES`] provided by the framework handle this automatically. Some software, add `-s` to their `LDFLAGS`, in this case, either remove `-s` if `WITH_DEBUG` is set, or remove it unconditionally and use `STRIP_CMD` in `post-install`. ==== [[install-copytree]] === Installing a Whole Tree of Files Sometimes, a large number of files must be installed while preserving their hierarchical organization. For example, copying over a whole directory tree from `WRKSRC` to a target directory under `PREFIX`. Note that `PREFIX`, `EXAMPLESDIR`, `DATADIR`, and other path variables must always be prepended with `STAGEDIR` to respect staging (see crossref:special[staging,Staging]). Two macros exist for this situation. The advantage of using these macros instead of `cp` is that they guarantee proper file ownership and permissions on target files. The first macro, `COPYTREE_BIN`, will set all the installed files to be executable, thus being suitable for installing into [.filename]#PREFIX/bin#. The second macro, `COPYTREE_SHARE`, does not set executable permissions on files, and is therefore suitable for installing files under [.filename]#PREFIX/share# target. [.programlisting] .... post-install: ${MKDIR} ${STAGEDIR}${EXAMPLESDIR} (cd ${WRKSRC}/examples && ${COPYTREE_SHARE} . ${STAGEDIR}${EXAMPLESDIR}) .... This example will install the contents of the [.filename]#examples# directory in the vendor distfile to the proper examples location of the port. [.programlisting] .... post-install: ${MKDIR} ${STAGEDIR}${DATADIR}/summer (cd ${WRKSRC}/temperatures && ${COPYTREE_SHARE} "June July August" ${STAGEDIR}${DATADIR}/summer) .... And this example will install the data of summer months to the [.filename]#summer# subdirectory of a [.filename]#DATADIR#. Additional `find` arguments can be passed via the third argument to `COPYTREE_*` macros. For example, to install all files from the first example except Makefiles, one can use these commands. [.programlisting] .... post-install: ${MKDIR} ${STAGEDIR}${EXAMPLESDIR} (cd ${WRKSRC}/examples && \ ${COPYTREE_SHARE} . ${STAGEDIR}${EXAMPLESDIR} "! -name Makefile") .... These macros do not add the installed files to [.filename]#pkg-plist#. They must be added manually. For optional documentation (`PORTDOCS`, see crossref:makefiles[install-documentation, Install Additional Documentation]) and examples (`PORTEXAMPLES`), the `%%PORTDOCS%%` or `%%PORTEXAMPLES%%` prefixes must be prepended in [.filename]#pkg-plist#. [[install-documentation]] === Install Additional Documentation If the software has some documentation other than the standard man and info pages that is useful for the user, install it under `DOCSDIR`. This can be done, like the previous item, in the `post-install` target. Create a new directory for the port. The directory name is `DOCSDIR`. This usually equals `PORTNAME`. However, if the user might want different versions of the port to be installed at the same time, the whole `PKGNAME` can be used. Since only the files listed in [.filename]#pkg-plist# are installed, it is safe to always install documentation to `STAGEDIR` (see crossref:special[staging,Staging]). Hence `.if` blocks are only needed when the installed files are large enough to cause significant I/O overhead. [.programlisting] .... post-install: ${MKDIR} ${STAGEDIR}${DOCSDIR} ${INSTALL_DATA} ${WRKSRC}/docs/xvdocs.ps ${STAGEDIR}${DOCSDIR} .... On the other hand, if there is a DOCS option in the port, install the documentation in a `post-install-DOCS-on` target. -These targets are described in crossref:makefiles[options-targets, Additional Build Targets, `_target_-_OPT_-on` and `_target_-_OPT_-off`]. +These targets are described in crossref:makefiles[options-targets,"Additional Build Targets, `_target_-_OPT_-on` and `_target_-_OPT_-off`"]. Here are some handy variables and how they are expanded by default when used in the [.filename]#Makefile#: * `DATADIR` gets expanded to [.filename]#PREFIX/share/PORTNAME#. * `DATADIR_REL` gets expanded to [.filename]#share/PORTNAME#. * `DOCSDIR` gets expanded to [.filename]#PREFIX/share/doc/PORTNAME#. * `DOCSDIR_REL` gets expanded to [.filename]#share/doc/PORTNAME#. * `EXAMPLESDIR` gets expanded to [.filename]#PREFIX/share/examples/PORTNAME#. * `EXAMPLESDIR_REL` gets expanded to [.filename]#share/examples/PORTNAME#. [NOTE] ==== The `DOCS` option only controls additional documentation installed in `DOCSDIR`. It does not apply to standard man pages and info pages. Things installed in `EXAMPLESDIR` are controlled by the `EXAMPLES` option. ==== These variables are exported to `PLIST_SUB`. Their values will appear there as pathnames relative to [.filename]#PREFIX# if possible. That is, [.filename]#share/doc/PORTNAME# will be substituted for `%%DOCSDIR%%` in the packing list by default, and so on. (See more on [.filename]#pkg-plist# substitution crossref:plist[plist-sub,here].) All conditionally installed documentation files and directories are included in [.filename]#pkg-plist# with the `%%PORTDOCS%%` prefix, for example: [.programlisting] .... %%PORTDOCS%%%%DOCSDIR%%/AUTHORS %%PORTDOCS%%%%DOCSDIR%%/CONTACT .... As an alternative to enumerating the documentation files in [.filename]#pkg-plist#, a port can set the variable `PORTDOCS` to a list of file names and shell glob patterns to add to the final packing list. The names will be relative to `DOCSDIR`. Therefore, a port that utilizes `PORTDOCS`, and uses a non-default location for its documentation, must set `DOCSDIR` accordingly. If a directory is listed in `PORTDOCS` or matched by a glob pattern from this variable, the entire subtree of contained files and directories will be registered in the final packing list. If the `DOCS` option has been unset then files and directories listed in `PORTDOCS` would not be installed or added to port packing list. Installing the documentation at `PORTDOCS` as shown above remains up to the port itself. A typical example of utilizing `PORTDOCS`: [.programlisting] .... PORTDOCS= README.* ChangeLog docs/* .... [NOTE] ==== The equivalents of `PORTDOCS` for files installed under `DATADIR` and `EXAMPLESDIR` are `PORTDATA` and `PORTEXAMPLES`, respectively. The contents of [.filename]#pkg-message# are displayed upon installation. See crossref:pkg-files[porting-message,the section on using [.filename]#pkg-message#] for details. [.filename]#pkg-message# does not need to be added to [.filename]#pkg-plist#. ==== [[install-subdirs]] === Subdirectories Under `PREFIX` Try to let the port put things in the right subdirectories of `PREFIX`. Some ports lump everything and put it in the subdirectory with the port's name, which is incorrect. Also, many ports put everything except binaries, header files and manual pages in a subdirectory of [.filename]#lib#, which does not work well with the BSD paradigm. Many of the files must be moved to one of these directories: [.filename]#etc# (setup/configuration files), [.filename]#libexec# (executables started internally), [.filename]#sbin# (executables for superusers/managers), [.filename]#info# (documentation for info browser) or [.filename]#share# (architecture independent files). See man:hier[7] for details; the rules governing [.filename]#/usr# pretty much apply to [.filename]#/usr/local# too. The exception are ports dealing with USENET "news". They may use [.filename]#PREFIX/news# as a destination for their files. [[binary-alias]] == Use `BINARY_ALIAS` to Rename Commands Instead of Patching the Build When `BINARY_ALIAS` is defined it will create symlinks of the given commands in a directory which will be prepended to `PATH`. Use it to substitute hardcoded commands the build phase relies on without having to patch any build files. [[binary-alias-ex1]] .Using `BINARY_ALIAS` to Make `gsed` Available as `sed` [example] ==== Some ports expect `sed` to behave like GNU sed and use features that man:sed[1] does not provide. GNU sed is available from package:textproc/gsed[] on FreeBSD. Use `BINARY_ALIAS` to substitute `sed` with `gsed` for the duration of the build: [.programlisting] .... BUILD_DEPENDS= gsed:textproc/gsed ... BINARY_ALIAS= sed=gsed .... ==== [[binary-alias-ex2]] .Using `BINARY_ALIAS` to Provide Aliases for Hardcoded `python3` Commands [example] ==== A port that has a hardcoded reference to `python3` in its build scripts will need to have it available in `PATH` at build time. Use `BINARY_ALIAS` to create an alias that points to the right Python 3 binary: [.programlisting] .... USES= python:3.4+,build ... BINARY_ALIAS= python3=${PYTHON_CMD} .... See crossref:special[using-python,Using Python] for more information about `USES=python`. ==== [NOTE] ==== Binary aliases are created after the dependencies provided via `BUILD_DEPENDS` and `LIB_DEPENDS` are processed and before the `configure` target. This leads to various limitations. For example, programs installed via `TEST_DEPENDS` cannot be used to create a binary alias as test dependencies specified this way are processed after binary aliases are created. ==== diff --git a/documentation/content/en/books/porters-handbook/plist/_index.adoc b/documentation/content/en/books/porters-handbook/plist/_index.adoc index 9a76a8e3e8..ea8cd92d84 100644 --- a/documentation/content/en/books/porters-handbook/plist/_index.adoc +++ b/documentation/content/en/books/porters-handbook/plist/_index.adoc @@ -1,694 +1,694 @@ --- title: Chapter 8. Advanced pkg-plist Practices prev: books/porters-handbook/flavors next: books/porters-handbook/pkg-files description: Advanced pkg-plist Practices tags: ["pkg-plist", "practices", "configuration"] showBookMenu: true weight: 8 params: path: "/books/porters-handbook/plist/" --- [[plist]] = Advanced pkg-plist Practices :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumoffset: 8 :partnums: :source-highlighter: rouge :experimental: :images-path: books/porters-handbook/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[plist-sub]] == Changing pkg-plist Based on Make Variables Some ports, particularly the `p5-` ports, need to change their [.filename]#pkg-plist# depending on what options they are configured with (or version of `perl`, in the case of `p5-` ports). To make this easy, any instances in [.filename]#pkg-plist# of `%%OSREL%%`, `%%PERL_VER%%`, and `%%PERL_VERSION%%` will be substituted appropriately. The value of `%%OSREL%%` is the numeric revision of the operating system (for example, `4.9`). `%%PERL_VERSION%%` and `%%PERL_VER%%` is the full version number of `perl` (for example, `5.8.9`). Several other `%%_VARS_%%` related to port's documentation files are described in crossref:makefiles[install-documentation,the relevant section]. To make other substitutions, set `PLIST_SUB` with a list of `_VAR=VALUE_` pairs and instances of `%%_VAR_%%` will be substituted with _VALUE_ in [.filename]#pkg-plist#. For instance, if a port installs many files in a version-specific subdirectory, use a placeholder for the version so that [.filename]#pkg-plist# does not have to be regenerated every time the port is updated. For example, set: [.programlisting] .... OCTAVE_VERSION= ${PORTREVISION} PLIST_SUB= OCTAVE_VERSION=${OCTAVE_VERSION} .... in the [.filename]#Makefile# and use `%%OCTAVE_VERSION%%` wherever the version shows up in [.filename]#pkg-plist#. When the port is upgraded, it will not be necessary to edit dozens (or in some cases, hundreds) of lines in [.filename]#pkg-plist#. If files are installed conditionally on the options set in the port, the usual way of handling it is prefixing [.filename]#pkg-plist# lines with a `%%OPT%%` for lines needed when the option is enabled, or `%%NO_OPT%%` when the option is disabled, and adding `OPTIONS_SUB=yes` to the [.filename]#Makefile#. See crossref:makefiles[options_sub,`OPTIONS_SUB`] for more information. For instance, if there are files that are only installed when the `X11` option is enabled, and [.filename]#Makefile# has: [.programlisting] .... OPTIONS_DEFINE= X11 OPTIONS_SUB= yes .... In [.filename]#pkg-plist#, put `%%X11%%` in front of the lines only being installed when the option is enabled, like this : [.programlisting] .... %%X11%%bin/foo-gui .... This substitution will be done between the `pre-install` and `do-install` targets, by reading from [.filename]#PLIST# and writing to [.filename]#TMPPLIST# (default: [.filename]#WRKDIR/.PLIST.mktmp#). So if the port builds [.filename]#PLIST# on the fly, do so in or before `pre-install`. Also, if the port needs to edit the resulting file, do so in `post-install` to a file named [.filename]#TMPPLIST#. Another way of modifying a port's packing list is based on setting the variables `PLIST_FILES` and `PLIST_DIRS`. The value of each variable is regarded as a list of pathnames to write to [.filename]#TMPPLIST# along with [.filename]#PLIST# contents. While names listed in `PLIST_FILES` and `PLIST_DIRS` are subject to `%%_VAR_%%` substitution as described above, it is better to use the `${_VAR_}` directly. Except for that, names from `PLIST_FILES` will appear in the final packing list unchanged, while `@dir` will be prepended to names from `PLIST_DIRS`. To take effect, `PLIST_FILES` and `PLIST_DIRS` must be set before [.filename]#TMPPLIST# is written, that is, in `pre-install` or earlier. From time to time, using `OPTIONS_SUB` is not enough. In those cases, adding a specific `_TAG_` to `PLIST_SUB` inside the [.filename]#Makefile# with a special value of `@comment`, makes package tools to ignore the line. For instance, if some files are only installed when the `X11` option is on and the architecture is `i386`: [.programlisting] .... .include .if ${PORT_OPTIONS:MX11} && ${ARCH} == "i386" PLIST_SUB+= X11I386="" .else PLIST_SUB+= X11I386="@comment " .endif .... [[plist-cleaning]] == Empty Directories [[plist-dir-cleaning]] === Cleaning Up Empty Directories When being de-installed, a port has to remove empty directories it created. Most of these directories are removed automatically by man:pkg[8], but for directories created outside of [.filename]#${PREFIX}#, or empty directories, some more work needs to be done. This is usually accomplished by adding `@dir` lines for those directories. Subdirectories must be deleted before deleting parent directories. [.programlisting] .... [...] @dir /var/games/oneko/saved-games @dir /var/games/oneko .... [[plist-dir-empty]] === Creating Empty Directories Empty directories created during port installation need special attention. They must be present when the package is created. If they are not created by the port code, create them in the [.filename]#Makefile#: [.programlisting] .... post-install: ${MKDIR} ${STAGEDIR}${PREFIX}/some/directory .... Add the directory to [.filename]#pkg-plist# like any other. For example: [.programlisting] .... @dir some/directory .... [[plist-config]] == Configuration Files If the port installs configuration files to [.filename]#PREFIX/etc# (or elsewhere) do _not_ list them in [.filename]#pkg-plist#. That will cause `pkg delete` to remove files that have been carefully edited by the user, and a re-installation will wipe them out. Instead, install sample files with a [.filename]#filename.sample# extension. The `@sample` macro automates this, see crossref:plist[plist-keywords-sample, Expanding Package List with Keywords] for what it does exactly. For each sample file, add a line to [.filename]#pkg-plist#: [.programlisting] .... @sample etc/orbit.conf.sample .... If there is a very good reason not to install a working configuration file by default, only list the sample filename in [.filename]#pkg-plist#, without the `@sample` followed by a space part, and add a crossref:pkg-files[porting-message,message] pointing out that the user must copy and edit the file before the software will work. [TIP] ==== When a port installs its configuration in a subdirectory of [.filename]#${PREFIX}/etc#, use `ETCDIR`, which defaults to `${PREFIX}/etc/${PORTNAME}`, it can be overridden in the ports [.filename]#Makefile# if there is a convention for the port to use some other directory. The `%%ETCDIR%%` macro will be used in its stead in [.filename]#pkg-plist#. ==== [NOTE] ==== The sample configuration files should always have the [.filename]#.sample# suffix. If for some historical reason using the standard suffix is not possible, or if the sample files come from some other directory, use this construct: [.programlisting] .... @sample etc/orbit.conf-dist etc/orbit.conf .... or [.programlisting] .... @sample %%EXAMPLESDIR%%/orbit.conf etc/orbit.conf .... The format is `@sample _sample-file actual-config-file_`. ==== [[plist-dynamic]] == Dynamic Versus Static Package List A _static package list_ is a package list which is available in the Ports Collection either as [.filename]#pkg-plist# (with or without variable substitution), or embedded into the [.filename]#Makefile# via `PLIST_FILES` and `PLIST_DIRS`. Even if the contents are auto-generated by a tool or a target in the Makefile _before_ the inclusion into the Ports Collection by a committer (for example, using `make makeplist`), this is still considered a static list, since it is possible to examine it without having to download or compile the distfile. A _dynamic package list_ is a package list which is generated at the time the port is compiled based upon the files and directories which are installed. It is not possible to examine it before the source code of the ported application is downloaded and compiled, or after running a `make clean`. While the use of dynamic package lists is not forbidden, maintainers should use static package lists wherever possible, as it enables users to man:grep[1] through available ports to discover, for example, which port installs a certain file. Dynamic lists should be primarily used for complex ports where the package list changes drastically based upon optional features of the port (and thus maintaining a static package list is infeasible), or ports which change the package list based upon the version of dependent software used. For example, ports which generate docs with Javadoc. [[plist-autoplist]] == Automated Package List Creation First, make sure the port is almost complete, with only [.filename]#pkg-plist# missing. Running `make makeplist` will show an example for [.filename]#pkg-plist#. The output of `makeplist` must be double checked for correctness as it tries to automatically guess a few things, and can get it wrong. User configuration files should be installed as [.filename]#filename.sample#, as it is described in crossref:plist[plist-config, Configuration Files]. [.filename]#info/dir# must not be listed and appropriate [.filename]#install-info# lines must be added as noted in the crossref:makefiles[makefile-info,info files] section. Any libraries installed by the port must be listed as specified in the crossref:special[porting-shlibs,shared libraries] section. [[plist-autoplist-regex]] === Expanding `PLIST_SUB` with Regular Expressions Strings to be replaced sometimes need to be very specific to avoid undesired replacements. This is a common problem with shorter values. To address this problem, for each `_PLACEHOLDER_=_value_`, a `PLACEHOLDER_regex=regex` can be set, with the `_regex_` part matching _value_ more precisely. [[plist-autoplist-regex-ex1]] .Using PLIST_SUB with Regular Expressions [example] ==== Perl ports can install architecture dependent files in a specific tree. On FreeBSD to ease porting, this tree is called `mach`. For example, a port that installs a file whose path contains `mach` could have that part of the path string replaced with the wrong values. Consider this [.filename]#Makefile#: [.programlisting] .... PORTNAME= Machine-Build DISTVERSION= 1 CATEGORIES= devel perl5 MASTER_SITES= CPAN PKGNAMEPREFIX= p5- MAINTAINER= perl@FreeBSD.org COMMENT= Building machine WWW= https://search.cpan.org/dist/Machine-Build USES= perl5 USE_PERL5= configure PLIST_SUB= PERL_ARCH=mach .... The files installed by the port are: [.programlisting] .... /usr/local/bin/machine-build /usr/local/lib/perl5/site_perl/man/man1/machine-build.1.gz /usr/local/lib/perl5/site_perl/man/man3/Machine::Build.3.gz /usr/local/lib/perl5/site_perl/Machine/Build.pm /usr/local/lib/perl5/site_perl/mach/5.20/Machine/Build/Build.so .... Running `make makeplist` wrongly generates: [.programlisting] .... bin/%%PERL_ARCH%%ine-build %%PERL5_MAN1%%/%%PERL_ARCH%%ine-build.1.gz %%PERL5_MAN3%%/Machine::Build.3.gz %%SITE_PERL%%/Machine/Build.pm %%SITE_PERL%%/%%PERL_ARCH%%/%%PERL_VER%%/Machine/Build/Build.so .... Change the `PLIST_SUB` line from the [.filename]#Makefile# to: [.programlisting] .... PLIST_SUB= PERL_ARCH=mach \ PERL_ARCH_regex=\bmach\b .... Now `make makeplist` correctly generates: [.programlisting] .... bin/machine-build %%PERL5_MAN1%%/machine-build.1.gz %%PERL5_MAN3%%/Machine::Build.3.gz %%SITE_PERL%%/Machine/Build.pm %%SITE_PERL%%/%%PERL_ARCH%%/%%PERL_VER%%/Machine/Build/Build.so .... ==== [[plist-keywords]] == Expanding Package List with Keywords All keywords can also take optional arguments in parentheses. The arguments are owner, group, and mode. This argument is used on the file or directory referenced. To change the owner, group, and mode of a configuration file, use: [.programlisting] .... @sample(games,games,640) etc/config.sample .... The arguments are optional. If only the group and mode need to be changed, use: [.programlisting] .... @sample(,games,660) etc/config.sample .... [WARNING] ==== If a keyword is used on an crossref:makefiles[makefile-options,optional] entry, it must to be added after the helper: [.programlisting] .... %%FOO%%@sample etc/orbit.conf.sample .... This is because the options plist helpers are used to comment out the line, so they need to be put first. See crossref:makefiles[options_sub,`OPTIONS_SUB`] for more information. ==== [[plist-keywords-desktop-file-utils]] === `@desktop-file-utils` Will run `update-desktop-database -q` after installation and deinstallation. _Never_ use directly, add crossref:uses[uses-desktop-file-utils,`USES=desktop-file-utils`] to the [.filename]#Makefile#. [[plist-keywords-fc]] === `@fc` _directory_ Add a `@dir` entry for the directory passed as an argument, and run `fc-cache -fs` on that directory after installation and deinstallation. [[plist-keywords-fontsdir]] === `@fontsdir` _directory_ Add a `@dir` entry for the directory passed as an argument, and run `mkfontscale` and `mkfontdir` on that directory after installation and deinstallation. Additionally, on deinstallation, it removes the [.filename]#fonts.scale# and [.filename]#fonts.dir# cache files if they are empty. [[plist-keywords-info]] === `@info` _file_ Add the file passed as argument to the plist, and updates the info document index on installation and deinstallation. Additionally, it removes the index if empty on deinstallation. This should never be used manually, but always through `INFO`. See crossref:makefiles[makefile-info,Info Files] for more information. [[plist-keywords-kld]] === `@kld` _directory_ Runs `kldxref` on the directory on installation and deinstallation. Additionally, on deinstallation, it will remove the directory if empty. [[plist-keywords-rmtry]] === `@rmtry` _file_ Will remove the file on deinstallation, and not give an error if the file is not there. [[plist-keywords-sample]] === `@sample` _file_ [_file_] This is used to handle installation of configuration files, through example files bundled with the package. The "actual", non-sample, file is either the second filename, if present, or the first filename without the [.filename]#.sample# extension. This does three things. First, add the first file passed as argument, the sample file, to the plist. Then, on installation, if the actual file is not found, copy the sample file to the actual file. And finally, on deinstallation, remove the actual file if it has not been modified. See crossref:plist[plist-config, Configuration Files] for more information. [[plist-keywords-shared-mime-info]] === `@shared-mime-info` _directory_ Runs `update-mime-database` on the directory on installation and deinstallation. [[plist-keywords-shell]] === `@shell` _file_ Add the file passed as argument to the plist. On installation, add the full path to _file_ to [.filename]#/etc/shells#, while making sure it is not added twice. On deinstallation, remove it from [.filename]#/etc/shells#. [[plist-keywords-terminfo]] === `@terminfo` Do not use by itself. If the port installs [.filename]#*.terminfo# files, add crossref:uses[uses-terminfo,`USES=terminfo`] to its [.filename]#Makefile#. On installation and deinstallation, if `tic` is present, refresh [.filename]#${PREFIX}/shared/misc/terminfo.db# from the [.filename]#*.terminfo# files in [.filename]#${PREFIX}/shared/misc#. [[plist-keywords-base]] === Base Keywords There are a few keywords that are hardcoded, and documented in man:pkg-create[8]. For the sake of completeness, they are also documented here. [[plist-keywords-base-empty]] ==== `@` [_file_] The empty keyword is a placeholder to use when the file's owner, group, or mode need to be changed. For example, to set the group of the file to `games` and add the setgid bit, add: [.programlisting] .... @(,games,2755) sbin/daemon .... [[plist-keywords-base-exec]] ==== `@preexec` _command_, `@postexec` _command_, `@preunexec` _command_, `@postunexec` _command_ Execute _command_ as part of the package installation or deinstallation process. `@preexec` _command_:: Execute _command_ as part of the [.filename]#pre-install# scripts. `@postexec` _command_:: Execute _command_ as part of the [.filename]#post-install# scripts. `@preunexec` _command_:: Execute _command_ as part of the [.filename]#pre-deinstall# scripts. `@postunexec` _command_:: Execute _command_ as part of the [.filename]#post-deinstall# scripts. If _command_ contains any of these sequences somewhere in it, they are expanded inline. For these examples, assume that `@cwd` is set to [.filename]#/usr/local# and the last extracted file was [.filename]#bin/emacs#. `%F`:: Expand to the last filename extracted (as specified). In the example case [.filename]#bin/emacs#. `%D`:: Expand to the current directory prefix, as set with `@cwd`. In the example case [.filename]#/usr/local#. `%B`:: Expand to the basename of the fully qualified filename, that is, the current directory prefix plus the last filespec, minus the trailing filename. In the example case, that would be [.filename]#/usr/local/bin#. `%f`:: Expand to the filename part of the fully qualified name, or the converse of `%B`. In the example case, [.filename]#emacs#. [IMPORTANT] ==== These keywords are here to help you set up the package so that it is as ready to use as possible. They _must not_ be abused to start services, stop services, or run any other commands that will modify the currently running system. ==== [[plist-keywords-base-mode]] ==== `@mode` _mode_ Set default permission for all subsequently extracted files to _mode_. Format is the same as that used by man:chmod[1]. Use without an arg to set back to default permissions (mode of the file while being packed). [IMPORTANT] ==== This must be a numeric mode, like `644`, `4755`, or `600`. It cannot be a relative mode like `u+s`. ==== [[plist-keywords-base-owner]] ==== `@owner` _user_ Set default ownership for all subsequent files to _user_. Use without an argument to set back to default ownership (`root`). [[plist-keywords-base-group]] ==== `@group` _group_ Set default group ownership for all subsequent files to _group_. Use without an arg to set back to default group ownership (`wheel`). [[plist-keywords-base-comment]] ==== `@comment` _string_ This line is ignored when packing. [[plist-keywords-base-dir]] ==== `@dir` _directory_ Declare directory name. By default, directories created under `PREFIX` by a package installation are automatically removed. Use this when an empty directory under `PREFIX` needs to be created, or when the directory needs to have non default owner, group, or mode. Directories outside of `PREFIX` need to be registered. For example, [.filename]#/var/db/${PORTNAME}# needs to have a `@dir` entry whereas [.filename]#${PREFIX}/shared/${PORTNAME}# does not if it contains files or uses the default owner, group, and mode. [[plist-keywords-base-exec-deprecated]] ==== `@exec` _command_, `@unexec` _command_ (Deprecated) Execute _command_ as part of the installation or deinstallation process. -Please use crossref:plist[plist-keywords-base-exec, `@preexec` _command_, `@postexec` _command_, `@preunexec` _command_, `@postunexec` _command_] instead. +Please use crossref:plist[plist-keywords-base-exec,"`@preexec` _command_, `@postexec` _command_, `@preunexec` _command_, `@postunexec` _command_"] instead. [[plist-keywords-base-dirrm]] ==== `@dirrm` _directory_ (Deprecated) Declare directory name to be deleted at deinstall time. By default, directories created under `PREFIX` by a package installation are deleted when the package is deinstalled. [[plist-keywords-base-dirrmtry]] ==== `@dirrmtry` _directory_ (Deprecated) Declare directory name to be removed, as for `@dirrm`, but does not issue a warning if the directory cannot be removed. [[plist-keywords-creating-new]] === Creating New Keywords Package list files can be extended by keywords that are defined in the [.filename]#${PORTSDIR}/Keywords# directory. The settings for each keyword are stored in a UCL file named [.filename]#keyword.ucl#. The file must contain at least one of these sections: * `attributes` * `action` * `pre-install` * `post-install` * `pre-deinstall` * `post-deinstall` * `pre-upgrade` * `post-upgrade` [[plist-keywords-attributes]] ==== `attributes` Changes the owner, group, or mode used by the keyword. Contains an associative array where the possible keys are `owner`, `group`, and `mode`. The values are, respectively, a user name, a group name, and a file mode. For example: [.programlisting] .... attributes: { owner: "games", group: "games", mode: 0555 } .... [[plist-keywords-action]] ==== `action` Defines what happens to the keyword's parameter. Contains an array where the possible values are: `setprefix`:: Set the prefix for the next plist entries. `dir`:: Register a directory to be created on install and removed on deinstall. `dirrm`:: Register a directory to be deleted on deinstall. Deprecated. `dirrmtry`:: Register a directory to try and deleted on deinstall. Deprecated. `file`:: Register a file. `setmode`:: Set the mode for the next plist entries. `setowner`:: Set the owner for the next plist entries. `setgroup`:: Set the group for the next plist entries. `comment`:: Does not do anything, equivalent to not entering an `action` section. `ignore_next`:: Ignore the next entry in the plist. [[plist-keywords-arguments]] ==== `arguments` If set to `true`, adds argument handling, splitting the whole line, `%@`, into numbered arguments, `%1`, `%2`, and so on. For example, for this line: [.programlisting] .... @foo some.content other.content .... `%1` and `%2` will contain: [.programlisting] .... some.content other.content .... It also affects how the crossref:plist[plist-keywords-action,`action`] entry works. When there is more than one argument, the argument number must be specified. For example: [.programlisting] .... actions: [file(1)] .... [[plist-keywords-pre-post]] ==== `pre-install`, `post-install`, `pre-deinstall`, `post-deinstall`, `pre-upgrade`, `post-upgrade` These keywords contains a man:sh[1] script to be executed before or after installation, deinstallation, or upgrade of the package. In addition to the usual `@exec %_foo_` placeholders described in -crossref:plist[plist-keywords-base-exec, `@preexec` _command_, `@postexec` _command_, `@preunexec` _command_, `@postunexec` _command_], there is a new one, `%@`, which represents the argument of the keyword. +crossref:plist[plist-keywords-base-exec,"`@preexec` _command_, `@postexec` _command_, `@preunexec` _command_, `@postunexec` _command_"], there is a new one, `%@`, which represents the argument of the keyword. [[plist-keywords-examples]] ==== Custom Keyword Examples [[plist-keywords-fc-example]] .Example of a `@dirrmtryecho` Keyword [example] ==== This keyword does two things, it adds a `@dirrmtry _directory_` line to the packing list, and echoes the fact that the directory is removed when deinstalling the package. [.programlisting] .... actions: [dirrmtry] post-deinstall: < patch-pathname-file .... When generating patches for new, added files, `-N` is used to tell man:diff[1] to treat the non-existent original file as if it existed but was empty: [source,shell] .... % diff -u -N newfile.orig newfile > patch-pathname-newfile .... Using the recurse (`-r`) option to man:diff[1] to generate patches is fine, but please look at the resulting patches to make sure there is no unnecessary junk in there. In particular, diffs between two backup files, [.filename]##Makefile##s when the port uses `Imake` or GNU `configure`, etc., are unnecessary and have to be deleted. If it was necessary to edit [.filename]#configure.in# and run `autoconf` to regenerate `configure`, do not take the diffs of `configure` (it often grows to a few thousand lines!). Instead, define `USES=autoreconf` and take the diffs of [.filename]#configure.in#. [[slow-patch-automatic-replacements]] === Simple Automatic Replacements Simple replacements can be performed directly from the port [.filename]#Makefile# using the in-place mode of man:sed[1]. This is useful when changes use the value of a variable: [.programlisting] .... post-patch: @${REINPLACE_CMD} -e 's|/usr/local|${PREFIX}|g' ${WRKSRC}/Makefile .... [IMPORTANT] ==== Only use man:sed[1] to replace variable content. You must use patch files instead of man:sed[1] to replace static content. ==== Quite often, software being ported uses the CR/LF convention in source files. This may cause problems with further patching, compiler warnings, or script execution (like `/bin/sh^M not found`.) To quickly convert all files from CR/LF to just LF, add this entry to the port [.filename]#Makefile#: [.programlisting] .... USES= dos2unix .... A list of specific files to convert can be given: [.programlisting] .... USES= dos2unix DOS2UNIX_FILES= util.c util.h .... Use `DOS2UNIX_REGEX` to convert a group of files across subdirectories. Its argument is a man:find[1]-compatible regular expression. More on the format is in man:re_format[7]. This option is useful for converting all files of a given extension. For example, convert all source code files, leaving binary files intact: [.programlisting] .... USES= dos2unix DOS2UNIX_REGEX= .*\.([ch]|cpp) .... A similar option is `DOS2UNIX_GLOB`, which runs `find` for each element listed in it. [.programlisting] .... USES= dos2unix DOS2UNIX_GLOB= *.c *.cpp *.h .... The base directory for the conversion can be set. This is useful when there are multiple distfiles and several contain files which require line-ending conversion. [.programlisting] .... USES= dos2unix DOS2UNIX_WRKSRC= ${WRKDIR} .... [[slow-patch-extra]] === Patching Conditionally Some ports need patches that are only applied for specific FreeBSD versions or when a particular option is enabled or disabled. Conditional patches are specified by placing the full paths to the patch files in `EXTRA_PATCHES`. Conditional patch file names usually start with [.filename]#extra-# although this is not necessary. However, their file names _must not_ start with [.filename]#patch-#. If they do, they are applied unconditionally by the framework which is undesired for conditional patches. [[slow-patch-extra-ex1]] .Applying a Patch for a Specific FreeBSD Version [example] ==== [.programlisting] .... .include # Patch in the iconv const qualifier before this .if ${OPSYS} == FreeBSD && ${OSVERSION} < 1100069 EXTRA_PATCHES= ${PATCHDIR}/extra-patch-fbsd10 .endif .include .... ==== [[slow-patch-extra-ex2]] .Optionally Applying a Patch [example] ==== When an crossref:makefiles[makefile-options,option] requires a patch, use ``opt_EXTRA_PATCHES`` and ``opt_EXTRA_PATCHES_OFF`` to make the patch conditional on the `opt` option. -See crossref:makefiles[options-variables,Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF`] for more information. +See crossref:makefiles[options-variables,"Generic Variables Replacement, `OPT_VARIABLE` and `OPT_VARIABLE_OFF`"] for more information. [.programlisting] .... OPTIONS_DEFINE= FOO BAR FOO_EXTRA_PATCHES= ${PATCHDIR}/extra-patch-foo BAR_EXTRA_PATCHES_OFF= ${PATCHDIR}/extra-patch-bar.c \ ${PATCHDIR}/extra-patch-bar.h .... ==== [[slow-patch-extra-ex-dirs]] .Using `EXTRA_PATCHES` With a Directory [example] ==== Sometimes, there are many patches that are needed for a feature, in this case, it is possible to point `EXTRA_PATCHES` to a directory, and it will automatically apply all files named [.filename]#patch-*# in it. Create a subdirectory in [.filename]#${PATCHDIR}#, and move the patches in it. For example: [source,shell] .... % ls -l files/foo-patches -rw-r--r-- 1 root wheel 350 Jan 16 01:27 patch-Makefile.in -rw-r--r-- 1 root wheel 3084 Jan 18 15:37 patch-configure.ac .... Then add this to the [.filename]#Makefile#: [.programlisting] .... OPTIONS_DEFINE= FOO FOO_EXTRA_PATCHES= ${PATCHDIR}/foo-patches .... The framework will then use all the files named [.filename]#patch-*# in that directory. ==== [[slow-configure]] == Configuring Include any additional customization commands in the [.filename]#configure# script and save it in the [.filename]#scripts# subdirectory. As mentioned above, it is also possible do this with [.filename]#Makefile# targets and/or scripts with the name [.filename]#pre-configure# or [.filename]#post-configure#. [[slow-user-input]] == Handling User Input If the port requires user input to build, configure, or install, set `IS_INTERACTIVE` in the [.filename]#Makefile#. This will allow "overnight builds" to skip it. If the user sets the variable `BATCH` in their environment (and if the user sets the variable `INTERACTIVE`, then _only_ those ports requiring interaction are built). This will save a lot of wasted time on the set of machines that continually build ports (see below). It is also recommended that if there are reasonable default answers to the questions, `PACKAGE_BUILDING` be used to turn off the interactive script when it is set. This will allow us to build the packages for CDROMs and FTP. diff --git a/documentation/content/it/books/handbook/vinum/_index.adoc b/documentation/content/it/books/handbook/vinum/_index.adoc index a312e40cc4..c848d4edff 100644 --- a/documentation/content/it/books/handbook/vinum/_index.adoc +++ b/documentation/content/it/books/handbook/vinum/_index.adoc @@ -1,713 +1,713 @@ --- title: Capitolo 20. Il Gestore di Volumi Vinum part: Parte II. Compiti Ordinari prev: books/handbook/filesystems next: books/handbook/virtualization showBookMenu: true weight: 24 params: path: "/books/handbook/vinum/" --- [[vinum-vinum]] = Il Gestore di Volumi Vinum :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumoffset: 20 :partnums: :source-highlighter: rouge :experimental: :images-path: books/handbook/vinum/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[vinum-synopsis]] == Sinossi Qualunque siano i dischi che hai, ci sono sempre dei problemi potenziali: * Potrebbero essere troppo piccoli. * Potrebbero essere troppo lenti. * Potrebbero essere troppo inaffidabili. Un modo in cui alcuni utenti salvaguardano sè stessi contro questi problemi è attraverso l'uso di dischi multipli, e talvolta ridondanti. In aggiunta a supportare diverse schede e controller per sistemi RAID hardware, il sistema FreeBSD base include il gestore di volumi Vinum, un driver di dispositivo a blocchi che implementa dischi virtuali. Vinum fornisce più flessibilità, prestazioni, e affidabilità rispetto all'archiviazione su disco tradizionale e implementa i modelli RAID-0, RAID-1, e RAID-5 sia singolarmente che in combinazione. Questo capitolo fornisce una panoramica sui potenziali problemi dell'archiviazione su disco tradizionale e un'introduzione al gestore di volumi Vinum. [[vinum-intro]] == Dischi Troppo Piccoli _Vinum_ è un _Volume Manager_, ovvero un driver virtuale di disco che si occupa dei tre problemi indicati nella sinossi. Diamo un'occhiata in dettaglio a questi problemi, per i quali sono state proposte e implementate varie soluzioni. I dischi stanno diventando sempre più grandi, ma questo è vero anche per le necessità di spazio per i dati. Spesso sentirai il bisogno di avere un file system più grande dei dischi che possiedi. Effettivamente questo problema non è così grave come lo era dieci anni fa, ma è sempre presente. Alcuni sistemi risolvono la questione creando un dispositivo astratto che ripartisce i suoi dati su vari dischi. [[vinum-access-bottlenecks]] == Colli di Bottiglia nell'Accesso I moderni sistemi hanno frequentemente la necessità di accedere ai dati in modo concorrente. Ad esempio, un grande server FTP o HTTP può avere migliaia di sessioni concorrenti e molteplici connessioni da 100 Mbit/s verso il mondo esterno, ben oltre il transfer rate (velocità di trasferimento) che la maggior parte dei dischi può sostenere. I dischi odierni possono trasferire sequenzialmente dati fino a 70 MB/s, ma questo valore ha poca importanza in un ambiente dove molti processi indipendenti accedono al disco, in quanto raggiungerebbero solo una frazione di quella velocità. In questi casi è più interessante vedere il problema dal punto di vista del sottosistema dischi: il parametro importante è il carico che il trasferimento pone sul sottosistema, in altre parole il tempo per cui il trasferimento occupa i dischi necessari per lo stesso. In ogni trasferimento da disco il drive deve prima posizionare le testine, poi aspettare che il primo settore passi sotto la testina di lettura e solo dopo può effettuare il trasferimento. Queste azioni possono essere considerate atomiche: non ha alcun senso interromperle. [[vinum-latency]]Considera un tipico trasferimento di circa 10 kB: l'attuale generazione di dischi ad alte prestazioni può posizionare le testine in circa 3,5 ms. I dischi più veloci ruotano a 15.000 rpm, quindi la latenza media rotazionale (mezzo giro) è di 2 ms. A 70 MB/s, il trasferimento in sé occupa circa 150 μs, quasi nulla in confronto al tempo di posizionamento. In questo caso il transfer rate effettivo può scendere fino a poco oltre 1 MB/s e questo è chiaramente molto dipendente dalla dimensione del trasferimento. La tradizionale e ovvia soluzione a questo collo di bottiglia è "più assi": invece di usare un grande disco si usano molti piccoli dischi con la stessa dimensione totale. Ogni disco è capace di posizionarsi e trasferire dati indipendentemente quindi la velocità effettiva aumenta di un fattore vicino al numero di dischi usati. L'esatto fattore di miglioramento è, ovviamente, più piccolo del numero di dischi: benché ogni disco sia capace di trasferire in parallelo non c'è modo di assicurare che le richieste siano distribuite uniformemente tra tutti i dischi. Inevitabilmente il carico su uno dei dischi è più alto che sugli altri. L'uniformità della distribuzione del carico sui dischi è fortemente dipendente dal modo in cui i dati sono condivisi tra i dischi stessi. Nella seguente discussione è conveniente pensare allo spazio di immagazzinamento come se fosse diviso in un gran numero di settori identificati da un indirizzo numerico, come pagine in un libro. Il metodo più ovvio è di dividere il disco virtuale in gruppi di settori consecutivi della dimensione dei dischi fisici e immagazzinarli in questa maniera, come strappare un grosso libro in piccole sezioni. Questo metodo è chiamato _concatenazione_ e ha il vantaggio di non avere particolari richieste sulla dimensione degli specifici dischi. Funziona bene quando l'accesso al disco virtuale è ben ripartito tra tutto il suo spazio di indirizzamento. Quando l'accesso è concentrato in una piccola area il miglioramento è meno marcato. La <> illustra la sequenza in cui le unità di immagazzinamento sono allocate nell'organizzazione concatenata. [[vinum-concat]] .Organizzazione Concatenata image::vinum-concat.png[] Un metodo alternativo è dividere lo spazio di indirizzamento in più piccole componenti di egual dimensione e immagazzinarle sequenzialmente su differenti dispositivi. Per esempio i primi 256 settori potrebbero essere immagazzinati sul primo disco, i seguenti 256 settori sul disco seguente e così via. Dopo aver immagazzinato i dati sull'ultimo disco il processo si ripete finché i dischi non sono pieni. Questo mappamento è chiamato _striping_ (letteralmente "a bande") o RAID-0. Lo striping richiede qualche sforzo aggiuntivo per localizzare i dati e può causare carico di I/O aggiuntivo quando il trasferimento è distribuito tra vari dischi, ma aiuta il carico a essere ben distribuito tra i vari dischi. La <> illustra la sequenza in cui i blocchi di dati sono allocati nell'organizzazione in striping. [[vinum-striped]] .Organizzazione in Striping image::vinum-striped.png[] [[vinum-data-integrity]] == Integrità dei Dati L'ultimo problema dei dischi attuali è che sono inaffidabili. Benché la loro affidabilità sia aumentata tremendamente durante gli ultimi anni sono tuttora il componente di un server che ha la maggior probabilità di rompersi. Quando succede i risultati possono essere catastrofici: rimpiazzare un disco rotto e riempirlo dei dati originari può richiedere giorni. Il metodo tradizionale per affrontare questo problema si chiama _mirroring_ (letteralmente "specchiatura") e consiste nel tenere due copie dei dati su hardware fisici differenti. Con l'avvento dei livelli RAID questa tecnica è stata chiamata RAID di livello 1 o RAID-1. Ogni scrittura su disco scrive in entrambe le locazioni; una lettura può essere soddisfatta da entrambi quindi se un disco si rompe i dati sono sempre disponibili sull'altro disco. Il mirroring ha due problemi: * Il prezzo. Richiede il doppio dello spazio di immagazzinamento delle soluzioni non ridondanti. * L'impatto sulle prestazioni. La scrittura deve essere compiuta su entrambi i dischi quindi la banda occupata raddoppia. Le letture non soffrono di problemi sulle prestazioni: possono perfino essere più veloci. Una soluzione alternativa è la _parità_, implementata nel RAID di livello 2, 3, 4 e 5. Di questi, il RAID-5 è il più interessante. La sua implementazione in Vinum è una variante dell'organizzazione in striping che dedica un blocco di ogni banda alla parità degli altri blocchi. Per come è implementato in Vinum, ogni blocco RAID-5 è simile a un blocco in striping, con la differenza che implementa il RAID-5 includendo un blocco di parità per ogni banda. Come richiesto dal RAID-5 la locazione di questi blocchi di parità cambia da ogni banda alla successiva. I numeri nei blocchi dati indicano il numero dei blocchi relativi. [[vinum-raid5-org]] .Organizzazione RAID-5 image::vinum-raid5-org.png[] Comparandolo al mirroring, il RAID-5 ha il vantaggio di richiedere molto meno spazio di immagazzinamento. La velocità di lettura è simile all'organizzazione in striping, ma in scrittura l'accesso è significativamente più lento, circa il 25% della performance di lettura. Se uno dei dischi si rompe l'aggregato continua a lavorare con performance peggiorate: la lettura da uno dei dischi rimanenti continua normalmente, ma la lettura dal disco rotto è ricalcolata dai corrispondenti blocchi dei dischi rimanenti. [[vinum-objects]] == Oggetti Vinum Per risolvere questi problemi Vinum implementa una categoria di oggetti a quattro livelli: * L'oggetto più visibile è il disco virtuale, chiamato _volume_. I volumi hanno essenzialmente le stesse proprietà di un disco UNIX(R), benché ci sia qualche differenza minore. Non hanno limiti di dimensione. * I volumi sono composti da _plex_, ognuno dei quali rappresenta il completo spazio di indirizzamento del volume. È quindi questo il livello della gerarchia che gestisce la ridondanza dei dati. Pensa ai plex come a singoli dischi collegati tra loro in mirroring, ognuno contenente gli stessi dati. * Dato che Vinum vive all'interno del framework UNIX(R) di immagazzinamento dei dati sarebbe possibile utilizzare le partizioni UNIX(R) come blocchi basilari per costruire i plex multidisco, ma questo approccio sarebbe in effetti troppo poco flessibile: i dischi UNIX(R) possono avere solo un limitato numero di partizioni; al contrario Vinum suddivide le singole partizioni UNIX(R) (_drive_, ovvero dischi) in aree contigue chiamate _subdisks_ (sottodischi), che sono a loro volta utilizzati come elementi per costruire i plex. * I sottodischi risiedono su _drive_ Vinum, che attualmente sono partizioni UNIX(R). I drive Vinum possono contenere qualsiasi quantità di sottodischi. Con l'eccezione di una piccola area all'inizio del drive, che è usata per immagazzinare informazioni sulla configurazione e sullo stato, l'intero drive è disponibile per l'immagazzinamento dei dati. La sezione seguente descrive come gli oggetti sopra discussi possano dare le funzionalità richieste. === Considerazioni sulle Dimensioni dei Volumi I plex possono contenere molteplici sottodischi distribuiti tra tutti i drive presenti nella configurazione di Vinum, questo permette alla dimensione dei plex, e quindi anche dei volumi, di non essere limitata dalla dimensione dei singoli dischi. === Immagazzinamento Ridondante dei Dati Vinum implementa il mirroring collegando più plex allo stesso volume, ogni plex contiene la rappresentazione di tutti i dati del volume. Un volume può contenere da uno ad otto plex. Nonostante un plex rappresenti i dati di un volume per intero, è possibile che parti di questa rappresentazione vengano a mancare o per scelta (non definendo dei sottodischi per alcune parti del plex) o per accidente (come risultato della rottura del disco che le conteneva). Finché almeno un plex contiene i dati di tutto lo spazio d'indirizzamento del volume, il volume stesso è completamente funzionale. === Considerazioni sulle Prestazioni Vinum implementa sia la concatenazione che lo striping al livello di plex: * Un _plex concatenato_ usa lo spazio di indirizzamento di ogni sottodisco a turno. * Un _plex in striping_ suddivide i dati tra ogni sottodisco. I sottodischi devono tutti avere la stessa dimensione e devono essere presenti almeno due sottodischi perché esista differenza da un plex concatenato. === Quale Organizzazione per i Plex? La versione di Vinum distribuita con FreeBSD {rel120-current} implementa due tipi di plex: * I plex concatenati, che sono i più flessibili: possono contenere qualsiasi numero di sottodischi e questi possono avere qualsiasi dimensione. Il plex può essere esteso aggiungendo sottodischi. Richiede meno tempo di CPU di un plex in striping, benché la differenza in carico di CPU non sia misurabile. D'altro canto sono più suscettibili agli hot spot (letteralmente "zona calda"): ovvero situazioni in cui un disco è molto attivo mentre gli altri sono fermi. * Il più grande vantaggio dei plex in striping (RAID-0) è la loro capacità di ridurre gli hot spot: scegliendo una dimensione di striping ottimale (circa 256 kB) si può ridistribuire il carico sui drive. Gli svantaggi di questo approccio sono codice più complesso e restrizioni sui sottodischi: devono essere tutti della stessa dimensione, inoltre estendere il plex aggiungendo sottodischi è così complicato che attualmente Vinum non lo implementa. Vinum aggiunge anche un'ulteriore restrizione elementare: un plex in striping deve contenere almeno due sottodischi, dato che sarebbe altrimenti indistinguibile da un plex concatenato. La <> riassume vantaggi e svantaggi di ogni tipo di organizzazione dei plex. [[vinum-comparison]] .Organizzazione dei Plex Vinum [cols="1,1,1,1,1", frame="none", options="header"] |=== | Tipo di plex | Sottodischi minimi | Sottodischi aggiungibili | Dimensioni forzatamente uguali | Applicazione |concatenato |1 |sì |no |Immagazzinamento di grandi moli di dati con la massima flessibilità e prestazioni moderate |striping |2 |no |sì |Alte prestazioni in casi di accessi altamente concorrenti |=== [[vinum-examples]] == Alcuni Esempi Vinum mantiene un _database della configurazione_ che descrive gli oggetti del sistema conosciuti. Inizialmente l'utente crea il database della configurazione da uno o più file di configurazione, con l'aiuto del programma man:vinum[8]. Vinum immagazzina una copia del database di configurazione in ogni slice del disco (che Vinum chiama _device_, ovvero "dispositivo") sotto il suo controllo. Questo database è aggiornato a ogni cambio di stato in modo che un riavvio possa recuperare accuratamente lo stato di ogni oggetto Vinum. === Il File di Configurazione Il file di configurazione descrive singoli oggetti Vinum. La definizione di un semplice volume potrebbe essere: [.programlisting] .... drive a device /dev/da3h volume myvol plex org concat sd length 512m drive a .... Questo file descrive quattro oggetti Vinum: * La linea _drive_ descrive la partizione del disco (_drive_) e la sua locazione relativa all'hardware sottostante. Gli viene assegnato il nome simbolico _a_. Questa separazione dei nomi simbolici dai nomi di dispositivo permette di muovere i dischi da una locazione ad un'altra senza confusione. * La linea _volume_ descrive un volume. L'unico attributo richiesto è il nome, in questo caso _myvol_. * La linea _plex_ definisce un plex. L'unico parametro richiesto è il tipo di organizzazione, in questo caso _concat_. Non è necessario un nome: il sistema genera un nome automaticamente a partire dal nome del volume, aggiungendo un suffisso _.px_, dove _x_ indica il numero del plex nel volume. Il plex verrà quindi chiamato _myvol.p0_. * La linea _sd_ descrive un sottodisco. Le specifiche minime sono il nome del drive su cui immagazzinarlo e la lunghezza del sottodisco. Come per i plex non è necessario un nome: il sistema assegna automaticamente nomi derivati dal nome del plex, aggiungendo il suffisso _.sx_, dove _x_ indica il numero del sottodisco nel plex, quindi Vinum darà a questo sottodisco il nome di _myvol.p0.s0_. Dopo aver elaborato questo file, man:vinum[8] produce il seguente output: [.programlisting] .... # vinum -> create config1 Configuration summary Drives: 1 (4 configured) Volumes: 1 (4 configured) Plexes: 1 (8 configured) Subdisks: 1 (16 configured) D a State: up Device /dev/da3h Avail: 2061/2573 MB (80%) V myvol State: up Plexes: 1 Size: 512 MB P myvol.p0 C State: up Subdisks: 1 Size: 512 MB S myvol.p0.s0 State: up PO: 0 B Size: 512 MB .... Questo output mostra il formato di elenco breve di man:vinum[8], che è rappresentato graficamente nella <>. [[vinum-simple-vol]] .Un Semplice Volume Vinum image::vinum-simple-vol.png[] Questa figura e quelle che seguono rappresentano un volume contenente dei plex che a loro volta contengono dei sottodischi. In questo semplice esempio il volume contiene un plex e il plex contiene un sottodisco. Questo particolare volume non ha specifici vantaggi su una convenzionale partizione di disco. Contiene un singolo plex, quindi non è ridondante. Il plex contiene un solo sottodisco, quindi non c'è differenza nell'immagazzinamento dei dati. Le sezioni seguenti mostrano vari metodi di configurazione più interessanti. === Aumentare la Resistenza alle Rotture: il Mirroring Il mirroring può aumentare, in un volume, la resistenza alle rotture. Quando si definisce un volume in mirroring è importante assicurarsi che i sottodischi di ogni plex siano su dischi differenti, in modo che la rottura di un drive non blocchi entrambi i plex. La seguente configurazione mostra un volume in mirroring: [.programlisting] .... drive b device /dev/da4h volume mirror plex org concat sd length 512m drive a plex org concat sd length 512m drive b .... In questo esempio non è necessario specificare nuovamente la definizione del drive _a_, dato che Vinum mantiene traccia di tutti gli oggetti nel suo database di configurazione. Dopo aver elaborato questa definizione, la configurazione appare così: [.programlisting] .... Drives: 2 (4 configured) Volumes: 2 (4 configured) Plexes: 3 (8 configured) Subdisks: 3 (16 configured) D a State: up Device /dev/da3h Avail: 1549/2573 MB (60%) D b State: up Device /dev/da4h Avail: 2061/2573 MB (80%) V myvol State: up Plexes: 1 Size: 512 MB V mirror State: up Plexes: 2 Size: 512 MB P myvol.p0 C State: up Subdisks: 1 Size: 512 MB P mirror.p0 C State: up Subdisks: 1 Size: 512 MB P mirror.p1 C State: initializing Subdisks: 1 Size: 512 MB S myvol.p0.s0 State: up PO: 0 B Size: 512 MB S mirror.p0.s0 State: up PO: 0 B Size: 512 MB S mirror.p1.s0 State: empty PO: 0 B Size: 512 MB .... <> mostra la struttura graficamente. [[vinum-mirrored-vol]] .Un Volume Vinum in Mirroring image::vinum-mirrored-vol.png[] In questo esempio ogni plex contiene l'intero spazio di indirizzamento di 512 MB. Come nel precedente esempio ogni plex contiene un solo sottodisco. === Ottimizzazione delle Prestazioni Il volume in mirroring dell'esempio precedente è più resistente alle rotture di un volume non in mirroring, ma le sue prestazioni sono inferiori: ogni scrittura sul volume richiede una scrittura su ognuno dei drive, utilizzando quindi una maggior frazione della banda passante totale dei dischi. Considerazioni sulle prestazioni portano ad un differente approccio: al posto del mirroring, i dati vengono posti sul maggior numero di dischi possibile utilizzando lo striping. La seguente configurazione mostra un volume con un plex in striping su quattro dischi: [.programlisting] .... drive c device /dev/da5h drive d device /dev/da6h volume stripe plex org striped 512k sd length 128m drive a sd length 128m drive b sd length 128m drive c sd length 128m drive d .... Come prima non è necessario definire i drive che Vinum già conosce. Dopo aver elaborato queste definizioni la configurazione appare così: [.programlisting] .... Drives: 4 (4 configured) Volumes: 3 (4 configured) Plexes: 4 (8 configured) Subdisks: 7 (16 configured) D a State: up Device /dev/da3h Avail: 1421/2573 MB (55%) D b State: up Device /dev/da4h Avail: 1933/2573 MB (75%) D c State: up Device /dev/da5h Avail: 2445/2573 MB (95%) D d State: up Device /dev/da6h Avail: 2445/2573 MB (95%) V myvol State: up Plexes: 1 Size: 512 MB V mirror State: up Plexes: 2 Size: 512 MB V striped State: up Plexes: 1 Size: 512 MB P myvol.p0 C State: up Subdisks: 1 Size: 512 MB P mirror.p0 C State: up Subdisks: 1 Size: 512 MB P mirror.p1 C State: initializing Subdisks: 1 Size: 512 MB P striped.p1 State: up Subdisks: 1 Size: 512 MB S myvol.p0.s0 State: up PO: 0 B Size: 512 MB S mirror.p0.s0 State: up PO: 0 B Size: 512 MB S mirror.p1.s0 State: empty PO: 0 B Size: 512 MB S striped.p0.s0 State: up PO: 0 B Size: 128 MB S striped.p0.s1 State: up PO: 512 kB Size: 128 MB S striped.p0.s2 State: up PO: 1024 kB Size: 128 MB S striped.p0.s3 State: up PO: 1536 kB Size: 128 MB .... [[vinum-striped-vol]] .Un Volume Vinum in Striping image::vinum-striped-vol.png[] Questo volume è rappresentato nella <>. L'intensità del colore delle strisce indica la posizione all'interno dello spazio di indirizzamento del plex: le più chiare all'inizio, le più scure alla fine. === Resistenza alle Rotture e Prestazioni [[vinum-resilience]]Con hardware a sufficenza è possibile creare volumi con miglioramenti sia nella resistenza alle rotture che nelle prestazioni, comparati alle normali partizioni UNIX(R). Una tipica configurazione potrebbe essere: [.programlisting] .... volume raid10 plex org striped 512k sd length 102480k drive a sd length 102480k drive b sd length 102480k drive c sd length 102480k drive d sd length 102480k drive e plex org striped 512k sd length 102480k drive c sd length 102480k drive d sd length 102480k drive e sd length 102480k drive a sd length 102480k drive b .... I sottodischi del secondo plex sono spostati di due posti rispetto a quelli del primo plex: questo aumenta le probabilità che le scritture non utilizzino lo stesso sottodisco anche nel caso in cui un trasferimento utilizzi entrambi i drive. La <> rappresenta la struttura di questo volume. [[vinum-raid10-vol]] .Un Volume Vinum in Mirroring e Striping image::vinum-raid10-vol.png[] [[vinum-object-naming]] == Nomenclatura degli Oggetti Come descritto sopra, Vinum assegna nomi di default a plex e sottodischi, benché questi possano essere cambiati. Cambiare il nome di default non è raccomandato: l'esperienza con il VERITAS volume manager, che permette la nomenclatura arbitraria degli oggetti, ha mostrato che questa flessibilità non porta vantaggi significativi e può causare confusione. I nomi possono contenere ogni carattere non blank (i caratteri di spazio, tabulazione, cambio riga) ma è consigliato limitarsi a lettere, cifre e il carattere di underscore. I nomi di volumi, plex e sottodischi possono essere lunghi fino a 64 caratteri, i nomi di drive invece hanno un massimo di 32 caratteri. I nomi assegnati agli oggetti Vinum sono nella gerarchia [.filename]#/dev/vinum#. La configurazione di Vinum mostrata sopra creerebbe i seguenti dispositivi: * I dispositivi di controllo [.filename]#/dev/vinum/control# e [.filename]#/dev/vinum/controld#, utilizzati rispettivamente da man:vinum[8] e dal demone Vinum. * Voci di dispositivi a blocchi e a caratteri per ogni volume. Questi sono i principali dispositivi utilizzati da Vinum. I dispositivi a blocchi hanno il nome dei relativi volumi, quelli a caratteri, seguendo la tradizione BSD, hanno una lettera _r_ all'inizio del nome. Quindi la configurazione vista sopra genererebbe i dispositivi a blocchi [.filename]#/dev/vinum/myvol#, [.filename]#/dev/vinum/mirror#, [.filename]#/dev/vinum/striped#, [.filename]#/dev/vinum/raid5# e [.filename]#/dev/vinum/raid10#, e i dispositivi a caratteri [.filename]#/dev/vinum/rmyvol#, [.filename]#/dev/vinum/rmirror#, [.filename]#/dev/vinum/rstriped#, [.filename]#/dev/vinum/rraid5# e [.filename]#/dev/vinum/rraid10#. In questo c'è un ovvio problema: è possibile avere due volumi chiamati _r_ e _rr_ che avrebbero un conflitto nel creare il nodo [.filename]#/dev/vinum/rr#: sarebbe il dispositivo a caratteri per il volume _r_ o il dispositivo a blocchi per il volume _rr_? Attualmente Vinum non si interessa di questo conflitto: il volume definito per primo prende il nome. * Una directory [.filename]#/dev/vinum/drive# con voci per ogni disco. Queste voci sono in effetti dei collegamenti simbolici ai rispettivi nodi di disco. * Una directory [.filename]#/dev/vinum/volume# con voci per ogni volume. Contiene sottodirectory per ogni plex, che a loro volta contengono sottodirectory per ogni sottodisco. * Le directory [.filename]#/dev/vinum/plex#, [.filename]#/dev/vinum/sd# e [.filename]#/dev/vinum/rsd# contengono i dispositivi a blocchi per ogni plex, dispositivo a blocchi e dispositivo a caratteri per ogni sottodisco rispettivamente. Ad esempio, considera il seguente file di configurazione: [.programlisting] .... drive drive1 device /dev/sd1h drive drive2 device /dev/sd2h drive drive3 device /dev/sd3h drive drive4 device /dev/sd4h volume s64 setupstate plex org striped 64k sd length 100m drive drive1 sd length 100m drive drive2 sd length 100m drive drive3 sd length 100m drive drive4 .... Dopo aver elaborato questo file, man:vinum[8] crea la seguente struttura in [.filename]#/dev/vinum#: [.programlisting] .... brwx------ 1 root wheel 25, 0x40000001 Apr 13 16:46 Control brwx------ 1 root wheel 25, 0x40000002 Apr 13 16:46 control brwx------ 1 root wheel 25, 0x40000000 Apr 13 16:46 controld drwxr-xr-x 2 root wheel 512 Apr 13 16:46 drive drwxr-xr-x 2 root wheel 512 Apr 13 16:46 plex crwxr-xr-- 1 root wheel 91, 2 Apr 13 16:46 rs64 drwxr-xr-x 2 root wheel 512 Apr 13 16:46 rsd drwxr-xr-x 2 root wheel 512 Apr 13 16:46 rvol brwxr-xr-- 1 root wheel 25, 2 Apr 13 16:46 s64 drwxr-xr-x 2 root wheel 512 Apr 13 16:46 sd drwxr-xr-x 3 root wheel 512 Apr 13 16:46 vol /dev/vinum/drive: total 0 lrwxr-xr-x 1 root wheel 9 Apr 13 16:46 drive1 -> /dev/sd1h lrwxr-xr-x 1 root wheel 9 Apr 13 16:46 drive2 -> /dev/sd2h lrwxr-xr-x 1 root wheel 9 Apr 13 16:46 drive3 -> /dev/sd3h lrwxr-xr-x 1 root wheel 9 Apr 13 16:46 drive4 -> /dev/sd4h /dev/vinum/plex: total 0 brwxr-xr-- 1 root wheel 25, 0x10000002 Apr 13 16:46 s64.p0 /dev/vinum/rsd: total 0 crwxr-xr-- 1 root wheel 91, 0x20000002 Apr 13 16:46 s64.p0.s0 crwxr-xr-- 1 root wheel 91, 0x20100002 Apr 13 16:46 s64.p0.s1 crwxr-xr-- 1 root wheel 91, 0x20200002 Apr 13 16:46 s64.p0.s2 crwxr-xr-- 1 root wheel 91, 0x20300002 Apr 13 16:46 s64.p0.s3 /dev/vinum/rvol: total 0 crwxr-xr-- 1 root wheel 91, 2 Apr 13 16:46 s64 /dev/vinum/sd: total 0 brwxr-xr-- 1 root wheel 25, 0x20000002 Apr 13 16:46 s64.p0.s0 brwxr-xr-- 1 root wheel 25, 0x20100002 Apr 13 16:46 s64.p0.s1 brwxr-xr-- 1 root wheel 25, 0x20200002 Apr 13 16:46 s64.p0.s2 brwxr-xr-- 1 root wheel 25, 0x20300002 Apr 13 16:46 s64.p0.s3 /dev/vinum/vol: total 1 brwxr-xr-- 1 root wheel 25, 2 Apr 13 16:46 s64 drwxr-xr-x 3 root wheel 512 Apr 13 16:46 s64.plex /dev/vinum/vol/s64.plex: total 1 brwxr-xr-- 1 root wheel 25, 0x10000002 Apr 13 16:46 s64.p0 drwxr-xr-x 2 root wheel 512 Apr 13 16:46 s64.p0.sd /dev/vinum/vol/s64.plex/s64.p0.sd: total 0 brwxr-xr-- 1 root wheel 25, 0x20000002 Apr 13 16:46 s64.p0.s0 brwxr-xr-- 1 root wheel 25, 0x20100002 Apr 13 16:46 s64.p0.s1 brwxr-xr-- 1 root wheel 25, 0x20200002 Apr 13 16:46 s64.p0.s2 brwxr-xr-- 1 root wheel 25, 0x20300002 Apr 13 16:46 s64.p0.s3 .... Benché sia raccomandato non allocare nomi specifici a plex e sottodischi, i drive di Vinum devono avere un nome. Questo permette di spostare un disco in una differente locazione e continuare a riconoscerlo automaticamente. I nomi di drive possono essere lunghi fino a 32 caratteri. === Creare i File System I volumi appaiono al sistema identici ai dischi, con un'eccezione. Differentemente dai dischi UNIX(R), Vinum non partiziona i volumi, che quindi non contengono una tabella delle partizioni. Questo ha reso necessario modificare alcuni programmi di utilità del disco, tra cui man:newfs[8], che precedentemente cercava di interpretare l'ultima lettera di un volume Vinum come un identificatore di partizione. Ad esempio un disco potrebbe avere un nome come [.filename]#/dev/ad0a# o [.filename]#/dev/da2h#. Questi nomi rappresentano la prima partizione ([.filename]#a#) del primo (0) disco IDE ([.filename]#ad#) e l'ottava partizione ([.filename]#h#) del terzo (2) disco SCSI ([.filename]#da#), rispettivamente. Al contrario un volume Vinum potrebbe essere chiamato [.filename]#/dev/vinum/concat#, un nome che non ha alcuna relazione con nomi di partizione. Normalmente man:newfs[8] interpreta il nome del disco e si lamenta se non riesce a comprenderlo. Per esempio: [source,shell] .... # newfs /dev/vinum/concat newfs: /dev/vinum/concat: can't figure out file system partition .... [NOTE] ==== Queste informazioni sono valide solo per versioni di FreeBSD precedenti alla 5.0: ==== Per poter creare un file system su questo volume usa man:newfs[8] con l'opzione `-v`: [source,shell] .... # newfs -v /dev/vinum/concat .... [[vinum-config]] == Configurare Vinum Il kernel [.filename]#GENERIC# non contiene Vinum. È possibile creare un kernel speciale che lo contenga, ma questo non è raccomandato: il metodo standard per lanciare Vinum è come modulo del kernel (kld). Non è neanche necessario usare man:kldload[8] per Vinum: quando lanci man:vinum[8] il programma controlla se il modulo è stato caricato e, in caso non sia caricato, lo carica automaticamente. === Avvio Vinum immagazzina le informazioni sulla configurazione dei dischi essenzialmente nella stessa forma dei file di configurazione. Quando legge il database di configurazione Vinum riconosce un numero di parole chiave che non sono permesse nei file di configurazione, ad esempio un file di configurazione del disco potrebbe contenere il seguente testo: [.programlisting] .... volume myvol state up volume bigraid state down plex name myvol.p0 state up org concat vol myvol plex name myvol.p1 state up org concat vol myvol plex name myvol.p2 state init org striped 512b vol myvol plex name bigraid.p0 state initializing org raid5 512b vol bigraid sd name myvol.p0.s0 drive a plex myvol.p0 state up len 1048576b driveoffset 265b plexoffset 0b sd name myvol.p0.s1 drive b plex myvol.p0 state up len 1048576b driveoffset 265b plexoffset 1048576b sd name myvol.p1.s0 drive c plex myvol.p1 state up len 1048576b driveoffset 265b plexoffset 0b sd name myvol.p1.s1 drive d plex myvol.p1 state up len 1048576b driveoffset 265b plexoffset 1048576b sd name myvol.p2.s0 drive a plex myvol.p2 state init len 524288b driveoffset 1048841b plexoffset 0b sd name myvol.p2.s1 drive b plex myvol.p2 state init len 524288b driveoffset 1048841b plexoffset 524288b sd name myvol.p2.s2 drive c plex myvol.p2 state init len 524288b driveoffset 1048841b plexoffset 1048576b sd name myvol.p2.s3 drive d plex myvol.p2 state init len 524288b driveoffset 1048841b plexoffset 1572864b sd name bigraid.p0.s0 drive a plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 0b sd name bigraid.p0.s1 drive b plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 4194304b sd name bigraid.p0.s2 drive c plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 8388608b sd name bigraid.p0.s3 drive d plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 12582912b sd name bigraid.p0.s4 drive e plex bigraid.p0 state initializing len 4194304b driveoff set 1573129b plexoffset 16777216b .... Le ovvie differenze sono qua la presenza di informazioni esplicite sulle locazioni e sulla nomenclatura (entrambe permesse, ma scoraggiate, all'utente) e le informazioni sugli stati (che non sono disponibili all'utente). Vinum non immagazzina informazioni sui drive tra le informazioni della configurazione: trova i drive scandendo le partizioni dei dischi configurati alla ricerca di un'etichetta Vinum. Questo permette a Vinum di identificare i drive correttamente anche se gli è stato assegnato un differente codice identificativo di drive UNIX(R). [[vinum-rc-startup]] ==== Avvio Automatico Per poter lanciare Vinum automaticamente all'avvio del sistema assicuratevi che le seguenti linee siano nel vostro [.filename]#/etc/rc.conf#: [.programlisting] .... start_vinum="YES" # set to YES to start vinum .... Se non hai un file [.filename]#/etc/rc.conf#, creane uno con questo contenuto. Questo ordinerà al sistema di caricare il Vinum kld all'avvio, inizializzando ogni oggetto menzionato nella configurazione. Questo viene fatto prima del mount dei file system quindi è possibile fare automaticamente man:fsck[8] e mount dei file system su volumi Vinum. Quando esegui Vinum con il comando `vinum start` Vinum legge il database di configurazione da uno dei drive Vinum. In circostanze normali ogni drive contiene una copia identica del database di configurazione quindi non conta da quale disco viene letto. Dopo un crash, tuttavia, Vinum deve determinare quale drive è stato aggiornato più recentemente e leggere la configurazione da questo drive. Quindi aggiorna la configurazione, se necessario, sui drive progressivamente più vecchi. [[vinum-root]] == Usare Vinum nel Filesystem Root Per una macchina con filesystem completamente in mirroring con Vinum è desiderabile mettere in mirroring anche il filesystem di root; fare questo è meno semplice che non per un filesystem arbitrario, dato che: * Il filesystem root deve essere disponibile nella parte iniziale del processo di boot, quindi l'infrastruttura di Vinum deve essere già disponibile in quel momento. * Il volume contenente il filesystem root contiene anche il sistema di avvio e il kernel, che devono essere letti usando le funzioni native del sistema (ovvero il BIOS, sui PC) che spesso non conoscono i dettagli di Vinum. Nelle sezioni seguenti, il termine "volume root" è usato generalmente per descrivere il volume Vinum che contiene il filesystem root. È probabilmente una buona idea usare il nome `"root"` per questo volume, ma non è necessario. Tutti gli esempi nelle prossime sezioni usano questo nome. === Lanciare Vinum abbastanza presto per il Filesystem Root Ci sono varie misure da prendere per fare in modo che questo accada: * Vinum deve essere disponibile nel kernel già all'avvio, quindi il metodo per lanciare Vinum automaticamente descritto in <> non può essere utilizzato e il parametro `start_vinum` in realtà _non_ va impostato in questo tipo di configurazione. La prima possibilità è di compilare Vinum staticamente dentro al kernel, in modo che sia sempre disponibile, ma questo non è normalmente desiderabile. Un'altra opportunità à di fare in modo che [.filename]#/boot/loader# (crossref:boot[boot-loader,Stadio Tre, /boot/loader]) carichi il modulo kernel di Vinum prima di lanciare il kernel. Questo può essere fatto utilizzando la riga: + [.programlisting] .... vinum_load="YES" .... + nel file [.filename]#/boot/loader.conf#. * Vinum deve essere inizializzato subito in modo da poter fornire il volume per il filesystem root. Per default la porzione kernel di Vinum non cerca dischi che contengano informazioni sui volumi Vinum fino a quando un amministratore (o uno degli script di partenza) non esegue un comando di `vinum start`. + [NOTE] ==== I seguenti paragrafi spiegano i passi necessari per FreeBSD 5.X e superiori. L'impostazione richiesta da FreeBSD 4.X è diversa ed è descritta dopo, in <>. ==== + Inserendo la linea: + [.programlisting] .... vinum.autostart="YES" .... + dentro a [.filename]#/boot/loader.conf#, Vinum viene istruito, alla partenza del kernel, di controllare automaticamente tutti i dischi alla ricerca di informazioni sui volumi Vinum. + Da notare il fatto che non è necessario istruire il kernel sulla locazione del filesystem root. [.filename]#/boot/loader# cerca il nome del device di root in [.filename]#/etc/fstab# e passa l'informazione al kernel. Quando è necessario montare il filesystem root, il kernel, tramite il nome di device fornitogli, capisce a quale driver deve chiedere la conversione di tale nome in ID interno di device (numero maggiore/minore). === Rendere un volume di root basato su Vinum accessibile dall'avvio Dato che il codice di avvio di FreeBSD è attualmente di soli 7.5 KB ed è già appesantito dalla lettura di file (come [.filename]#/boot/loader#) da un filesystem UFS, è semplicemente impossibile insegnargli anche a leggere le strutture interne di Vinum in modo da fargli leggere i dati della configurazione di Vinum per ricavarne gli elementi del volume di boot stesso. Sono quindi necessari alcuni trucchi per dare al codice di avvio l'illusione di una partizione `"a"` standard contenente il filesystem di root. Perché questo sia anche solo possibile, il volume di root deve rispondere ai seguenti requisiti: * Il volume di root non deve essere in striping o in RAID-5. * Il volume di root non deve contenere la concatenazione di più di un sottodisco per ogni plex. Da notare che è desiderabile e possibile avere plex multipli, contenente ognuno una replica del filesystem root. Il processo di avvio, però, usa solo una di queste repliche per trovare i file necessario all'avvio, fino a quando il kernel monta il filesystem di root stesso. Ogni singolo sottodisco in questi plex avrà quindi bisogno di una propria partizione `"a"` illusoria, affinché la periferica relativa possa essere avviabile. Non è strettamente necessario che ognuna di queste finte partizioni `"a"` sia locato con lo stesso spiazzamento all'interno della propria periferica, rispetto alle periferiche contenenti gli altri plex del volume. È comunque probabilmente una buona idea creare i volumi Vinum in modo che le periferiche in mirror siano simmetriche, per evitare confusione. Per poter configurare queste partizioni `"a"`, in ogni periferica contenente parte del volume di root, bisogna fare le seguenti cose: [.procedure] ==== . La locazione (spiazzamento dall'inizio della periferica) e la dimensione del sottodisco che è parte del volume di root deve essere esaminato, usando il comando: + [source,shell] .... # vinum l -rv root .... + Da notare che gli spiazzamenti e le dimensioni in Vinum sono misurati in byte. Devono essere divisi per 512 per ottenere il numero di blocchi necessari nel comando `disklabel`. + . Esegui il comando: + [source,shell] .... # disklabel -e devname .... + per ogni periferica che partecipa al volume di root. _devname_ deve essere o il nome della slice (ad esempio [.filename]#ad0s1#) o il nome del disco (ad esempio [.filename]#da0#) per dischi senza tabella delle slice (ovvero i nomi che si usano anche con fdisk). + Se c'è già una partizione `"a"` sulla periferica (presumibilmente contenente un filesystem root precedente all'uso di Vinum), dovrebbe essere rinominata in altro modo, in modo da restare accessibile (non si sa mai), ma non essere usata più per default per avviare il sistema. Da notare che le partizioni attive (ad esempio un filesystem root attualmente montato) non possono essere rinominati, quindi questo deve essere eseguito o avviando da un disco "Fixit" o (in caso di mirror) in un processo a due passi dove il disco non di avvio viene modificato per primo. + Infine, lo spiazzamento della partizione Vinum sulla periferica va aggiunto allo spiazzamento del rispettivo sottodisco di root rispetto alla periferica stessa. Il risultato sarà il valore di `"offset"` (spiazzamento) per la nuova partizione `"a"`. Il valore `"size"` (dimensione) per questa partizione può essere copiato pari pari dal calcolo fatto sopra. Il valore `"fstype"` deve essere `4.2BSD`. I valori `"fsize"`, `"bsize"` e `"cpg"` devono preferibilmente essere scelti per corrispondere al vero e proprio filesystem, anche se in questo contesto non sono molto importanti. + In questo modo una nuova partizione `"a"` sarà creata ricoprendo le partizioni Vinum su questa periferica. Da notare che `disklabel` permetterà questa ricopertura solo se la partizione Vinum è stata appropriatamente marcata usando un `"fstype"` pari a `"vinum"` fstype. . È tutto! Ora una falsa partizione `"a"` esiste su ogni periferica che abbia una replica del volume di root. È altamente raccomandabile verificare nuovamente i risultati, usando un comando come: + [source,shell] .... # fsck -n /dev/devnamea .... ==== Bisogna ricordarsi che tutte le informazioni contenute nei file di controllo devono essere relative al filesystem di root nel volume Vinum che, quando si imposta un nuovo volume di root Vinum, potrebbe non coincidere con quello del filesystem root attualmente attivo. In particolare bisogna stare attenti ai file [.filename]#/etc/fstab# e [.filename]#/boot/loader.conf#. Al seguente riavvio il sistema dovrebbe ricavare le informazioni di controllo appropriate dal filesystem di root Vinum e agire di consequenza. Alla fine del processo di inizializzazione del kernel, dopo che tutte le periferiche sono state annunciate, l'avvertimento principale che conferma il successo dell'avvio è un messaggio simile a questo: [source,shell] .... Mounting root from ufs:/dev/vinum/root .... === Esempi di configurazioni con root basata su Vinum Dopo aver creato il volume di root Vinum, `vinum l -rv root` dovrebbe produrre qualcosa di simile a: [source,shell] .... ... Subdisk root.p0.s0: Size: 125829120 bytes (120 MB) State: up Plex root.p0 at offset 0 (0 B) Drive disk0 (/dev/da0h) at offset 135680 (132 kB) Subdisk root.p1.s0: Size: 125829120 bytes (120 MB) State: up Plex root.p1 at offset 0 (0 B) Drive disk1 (/dev/da1h) at offset 135680 (132 kB) .... I valori su cui fare caso sono il `135680` dello spiazzamento (relativo alla partizione [.filename]#/dev/da0h#). Questo si traduce in 265 blocchi da 512 byte nei termini di `disklabel`. [.filename]#/dev/da1h#, contenente la seconda replica del volume di root, ha una configurazione simmetrica. La `disklabel` per queste periferiche dovrebbe essere simile a questa: [source,shell] .... ... 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 245760 281 4.2BSD 2048 16384 0 # (Cyl. 0*- 15*) c: 71771688 0 unused 0 0 # (Cyl. 0 - 4467*) h: 71771672 16 vinum # (Cyl. 0*- 4467*) .... Si può notare che il parametro `"size"` per la finta partizione `"a"` corrisponde al valore di cui abbiamo parlato prima, mentre il parametro `"offset"` è la somma dello spiazzamento all'interno della partizione Vinum `"h"` e lo spiazzamento all'interno della periferica (o slice). Questa è una tipica configurazione necessaria per evitare il problema descritto in <>. Si può anche notare che l'intera partizione `"a"` è completamente contenuta dalla partizione `"h"`, che contiene tutti i dati Vinum per questa periferica. Notate che in questo esempio l'intera periferica è dedicata a Vinum e non c'è spazio per partizioni pre-Vinum, dato che questo disco è stato configurato da nuovo per il solo scopo di far parte di una configurazione Vinum. === Risoluzione Problemi Se qualcosa va storto, serve un modo per tornare in una situazione di normalità. Segue una lista di alcuni tipici problemi, completi di soluzione. ==== Il Codice di Avvio si Carica, ma il Sistema non si Avvia Se per qualsiasi ragione in sistema non prosegue nell'avvio, si può interrompere il processo di avvio premendo il tasto kbd:[spazio] all'avviso dei 10 secondi. Le variabili di avvio potranno quindi essere esaminate usando il comando `show` e manipolate con `set` e `unset`. Se l'unico problema è che il modulo kernel di Vinum non è ancora presente nella lista dei moduli da caricare automaticamente, un semplice `load vinum` aiuterà. Quando pronti, il processo di avvio può continuare con un `boot -as`. Le opzioni `-as` comunicano al kernel di montare il filesystem di root (`-a`) e di interrompere il processo di avvio in modalità singolo utente (`-s`), quando il filesystem di root è montato a sola lettura. In questo modo benché uno solo dei plex do un volume multi-plex sia stato montato, non si rischia inconsistenza dei dati tra i vari plex. Alla richiesta di un filesystem di root da montare, ogni periferica che contenga un filesystem di root valido può essere usata. Se [.filename]#/etc/fstab# è stato impostato correttamente, il default dovrebbe essere `ufs:/dev/vinum/root` o simile. Una tipica alternativa potrebbe essere `ufs:da0d`, una ipotetica partizione contenente un filesystem root pre-Vinum. Bisogna fare attenzione quando si sceglie una delle partizioni `"a"` alias di un sottodisco della periferica root di Vinum, perché in una configurazione in mirror questo monterebbe solo un pezzo della root in mirror. Se questo filesystem deve poi essere montato in lettura/scrittura è necessario rimuovere gli altri plex del volume root di Vinum dato che conterrebbero comunque dati non consistenti. ==== Si Carica Solo l'Avvio Primario -Se [.filename]#/boot/loader# non si carica, ma l'avvio primario si carica comunque (si capisce dal singolo trattino nella colonna di sinistra dello schermo subito dopo l'inizio dell'avvio), si può tentare di fermare l'avvio primario in questo punto, premendo il tasto kbd:[spazio]. Questo fermerà l'avvio nella seconda fase, vedi crossref:boot[boot-boot1,Stadio Uno, /boot/boot1, e Stadio Due, /boot/boot2]. Qua si può fare un tentativo di caricare una differente partizione, ad esempio la partizione contenente il precedente filesystem di root `"a"`, prima di sostituirlo come sopra indicato. +Se [.filename]#/boot/loader# non si carica, ma l'avvio primario si carica comunque (si capisce dal singolo trattino nella colonna di sinistra dello schermo subito dopo l'inizio dell'avvio), si può tentare di fermare l'avvio primario in questo punto, premendo il tasto kbd:[spazio]. Questo fermerà l'avvio nella seconda fase, vedi crossref:boot[boot-boot1,"Stadio Uno, /boot/boot1, e Stadio Due, /boot/boot2"]. Qua si può fare un tentativo di caricare una differente partizione, ad esempio la partizione contenente il precedente filesystem di root `"a"`, prima di sostituirlo come sopra indicato. [[vinum-root-panic]] ==== Non si Carica Niente, l'Avvio va in Panico Questa situazione accade quando l'installazione di Vinum ha distrutto il codice di avvio. Sfortunatamente Vinum lascia solo 4 KB liberi all'inizio della sua partizione prima di scrivere il proprio header. Purtroppo le due fasi dell'avvio e la disklabel compresa tra le due attualmente occupa 8 KB, quindi se la partizione Vinum è creata con spiazzamento 0 in una slice o disco che dovrebbe essere avviabile, la configurazione di Vinum distruggerà il codice di avvio. -Similmente, se la situazione sopra descritta è stata risolta avviando da un disco di "Fixit", e il codice di avvio re-installato usando `disklabel -B` come descritto in crossref:boot[boot-boot1,Stadio Uno, /boot/boot1, e Stadio Due, /boot/boot2], il codice di avvio distruggerà l'header di Vinum, che non saprà più trovare i propri dischi. Benché nessun dato, di configurazione o contenuto, sia distrutto da questo processo, che risulta quindi recuperabile reinserendo la stessa configurazione di Vinum, la situazione è in realtà di difficile risoluzione: sarebbe necessario spostare l'intera partizione Vinum di almeno 4 KB, in modo da lasciare abbastanza spazio sia per il codice di avvio che per l'header Vinum. +Similmente, se la situazione sopra descritta è stata risolta avviando da un disco di "Fixit", e il codice di avvio re-installato usando `disklabel -B` come descritto in crossref:boot[boot-boot1,"Stadio Uno, /boot/boot1, e Stadio Due, /boot/boot2"], il codice di avvio distruggerà l'header di Vinum, che non saprà più trovare i propri dischi. Benché nessun dato, di configurazione o contenuto, sia distrutto da questo processo, che risulta quindi recuperabile reinserendo la stessa configurazione di Vinum, la situazione è in realtà di difficile risoluzione: sarebbe necessario spostare l'intera partizione Vinum di almeno 4 KB, in modo da lasciare abbastanza spazio sia per il codice di avvio che per l'header Vinum. [[vinum-root-4x]] === Differenze per FreeBSD 4.X In FreeBSD 4.X alcune funzioni interne necessarie a Vinum per poter trovare automaticamente tutti i dischi non sono presenti e il codice che ricava l'ID interno della periferica di root non è abbastanza intelligente da gestire automaticamente nomi come [.filename]#/dev/vinum/root#, quindi le cose vengono fatte in modo un po' diverso. Bisogna dire esplicitamente a Vinum quali dischi controllare, usando una riga di [.filename]#/boot/loader.conf# come la seguente: [.programlisting] .... vinum.drives="/dev/da0 /dev/da1" .... È importante indicare tutti i drive che possono contenere dati Vinum. Non è un problema indicare drive _di troppo_, non è neanche necessario aggiungere esplicitamente ogni slice e/o partizione, dato che Vinum cercherà header Vinum in tutte le slice e partizioni dei drive nominati. Dato che le procedure utilizzate per interpretare il nome del filesystem di root e derivarne l'ID di periferica (numeri maggiore e minore) sono adatte per gestire solo nomi "classici" di periferica come [.filename]#/dev/ad0s1a#, non riescono a capire nomi di volumi root come [.filename]#/dev/vinum/root#. Per questo motivo Vinum ha bisogno di pre-impostare durante la propria inzializzazione il parametro kernel interno che contiene l'ID della periferica di root. Questo viene fatto indicando il nome del volume di root nella variabile di avvio `vinum.root`. La riga di [.filename]#/boot/loader.conf# adatta per fare questo è simile alla seguente: [.programlisting] .... vinum.root="root" .... Quando l'inizializzazione del kernel cerca di trovare la periferica root da montare controlla se qualche modulo del kernel ha già pre-inizializzato il parametro kernel apposito; se questo è il caso _e_ la periferica che dice di essere la periferica di root ha il numero maggiore e minore corrispondenti al driver come trovato dal nome della periferica di root passata (ovvero `"vinum"`, nel nostro caso), userà l'ID di periferica pre-allocato, anziché cercar di trovarne uno da solo. In tal modo durante l'usuale avvio automatico può continuare a montare il volume di root Vinum per il filesystem di root. Bisogna comunque notare che anche quando `boot -a` richiede di inserire manualmente il nome della periferica di root tale nome non può essere intepretato nel caso sia un nome riferito a un volume Vinum. Se è inserito un nome di periferica non riferito a una periferica Vinum, la non corrispondenza col numero maggiore pre-allocato del parametro di root e del driver ricavato dal nome farà sì che la procedura utilizzi il normale interprete, quindi una stringa come `ufs:da0d` funzionerà come previsto. Da notare che se questo fallisce non si può più provare una stringa come `ufs:vinum/root`, dato che non verrebbe interpretata; l'unica via di uscita è riavviare e ripartire da zero. (Alla richiesta di "askroot", la parte iniziale [.filename]#/dev/# può sempre essere omessa.)