diff --git a/documentation/content/en/articles/linux-users/_index.adoc b/documentation/content/en/articles/linux-users/_index.adoc index 37f9a4126d..9db8a0eeea 100644 --- a/documentation/content/en/articles/linux-users/_index.adoc +++ b/documentation/content/en/articles/linux-users/_index.adoc @@ -1,377 +1,377 @@ --- title: FreeBSD Quickstart Guide for Linux® Users authors: - author: John Ferrell copyright: 2008 The FreeBSD Documentation Project description: This document is intended to quickly familiarize intermediate to advanced Linux® users with the basics of FreeBSD. trademarks: ["freebsd", "intel", "redhat", "linux", "unix", "general"] tags: ["Quickstart", "guide", "Linux", "FreeBSD"] --- = FreeBSD Quickstart Guide for Linux(R) Users :doctype: article :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :source-highlighter: rouge :experimental: :images-path: articles/linux-users/ ifdef::env-beastie[] ifdef::backend-html5[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] :imagesdir: ../../../images/{images-path} endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [.abstract-title] Abstract This document is intended to quickly familiarize intermediate to advanced Linux(R) users with the basics of FreeBSD. ''' toc::[] [[intro]] == Introduction This document highlights some of the technical differences between FreeBSD and Linux(R) so that intermediate to advanced Linux(R) users can quickly familiarize themselves with the basics of FreeBSD. This document assumes that FreeBSD is already installed. Refer to the extref:{handbook}[Installing FreeBSD, bsdinstall] chapter of the FreeBSD Handbook for help with the installation process. [[shells]] == Default Shell Linux(R) users are often surprised to find that Bash is not the default shell in FreeBSD. In fact, Bash is not included in the default installation. Instead, FreeBSD uses man:tcsh[1] as the default root shell, and the Bourne shell-compatible man:sh[1] as the default user shell. man:sh[1] is very similar to Bash but with a much smaller feature-set. Generally shell scripts written for man:sh[1] will run in Bash, but the reverse is not always true. However, Bash and other shells are available for installation using the FreeBSD extref:{handbook}[Packages and Ports Collection, ports]. After installing another shell, use man:chsh[1] to change a user's default shell. It is recommended that the `root` user's default shell remain unchanged since shells which are not included in the base distribution are installed to [.filename]#/usr/local/bin#. In the event of a problem, the file system where [.filename]#/usr/local/bin# is located may not be mounted. In this case, `root` would not have access to its default shell, preventing `root` from logging in and fixing the problem. [[software]] == Packages and Ports: Adding Software in FreeBSD FreeBSD provides two methods for installing applications: binary packages and compiled ports. Each method has its own benefits: .Binary Packages * Faster installation as compared to compiling large applications. * Does not require an understanding of how to compile software. * No need to install a compiler. .Ports * Ability to customize installation options. * Custom patches can be applied. If an application installation does not require any customization, installing the package is sufficient. Compile the port instead whenever an application requires customization of the default options. If needed, a custom package can be compiled from ports using `make package`. A complete list of all available ports and packages can be found https://www.freebsd.org/ports/[here]. [[packages]] === Packages Packages are pre-compiled applications, the FreeBSD equivalents of [.filename]#.deb# files on Debian/Ubuntu based systems and [.filename]#.rpm# files on Red Hat/Fedora based systems. Packages are installed using `pkg`. For example, the following command installs Apache 2.4: [source,shell] .... # pkg install apache24 .... For more information on packages refer to section 5.4 of the FreeBSD Handbook: extref:{handbook}[Using pkgng for Binary Package Management, pkgng-intro]. [[ports]] === Ports The FreeBSD Ports Collection is a framework of [.filename]#Makefiles# and patches specifically customized for installing applications from source on FreeBSD. When installing a port, the system will fetch the source code, apply any required patches, compile the code, and install the application and any required dependencies. -The Ports Collection, sometimes referred to as the ports tree, can be installed to [.filename]#/usr/ports# using man:portsnap[8]. -Detailed instructions for installing the Ports Collection can be found in extref:{handbook}[section 5.5, ports-using] of the FreeBSD Handbook. +The Ports Collection, sometimes referred to as the ports tree, can be installed to [.filename]#/usr/ports# using link:{handbook}mirrors/#git[Git]. +Detailed instructions for installing the Ports Collection can be found in extref:{handbook}[section 4.5.1, ports-using-installation-methods] of the FreeBSD Handbook. To compile a port, change to the port's directory and start the build process. The following example installs Apache 2.4 from the Ports Collection: [source,shell] .... # cd /usr/ports/www/apache24 # make install clean .... A benefit of using ports to install software is the ability to customize the installation options. This example specifies that the mod_ldap module should also be installed: [source,shell] .... # cd /usr/ports/www/apache24 # make WITH_LDAP="YES" install clean .... Refer to extref:{handbook}[Using the Ports Collection, ports-using] for more information. [[startup]] == System Startup Many Linux(R) distributions use the SysV init system, whereas FreeBSD uses the traditional BSD-style man:init[8]. Under the BSD-style man:init[8], there are no run-levels and [.filename]#/etc/inittab# does not exist. Instead, startup is controlled by man:rc[8] scripts. -At system boot, [.filename]#/etc/rc# reads [.filename]#/etc/rc.conf# and [.filename]#/etc/defaults/rc.conf# to determine which services are to be started. +At system boot, [.filename]#/etc/rc# reads [.filename]#/etc/rc.conf# and [.filename]#/etc/defaults/rc.conf# to determine which services are to be started. The specified services are then started by running the corresponding service initialization scripts located in [.filename]#/etc/rc.d/# and [.filename]#/usr/local/etc/rc.d/#. These scripts are similar to the scripts located in [.filename]#/etc/init.d/# on Linux(R) systems. -The scripts found in [.filename]#/etc/rc.d/# are for applications that are part of the "base" system, such as man:cron[8], man:sshd[8], and man:syslog[3]. +The scripts found in [.filename]#/etc/rc.d/# are for applications that are part of the "base" system, such as man:cron[8], man:sshd[8], and man:syslog[3]. The scripts in [.filename]#/usr/local/etc/rc.d/# are for user-installed applications such as Apache and Squid. Since FreeBSD is developed as a complete operating system, user-installed applications are not considered to be part of the "base" system. User-installed applications are generally installed using extref:{handbook}[Packages or Ports, ports-using]. In order to keep them separate from the base system, user-installed applications are installed under [.filename]#/usr/local/#. Therefore, user-installed binaries reside in [.filename]#/usr/local/bin/#, configuration files are in [.filename]#/usr/local/etc/#, and so on. Services are enabled by adding an entry for the service in [.filename]#/etc/rc.conf#. The system defaults are found in [.filename]#/etc/defaults/rc.conf# and these default settings are overridden by settings in [.filename]#/etc/rc.conf#. Refer to man:rc.conf[5] for more information about the available entries. When installing additional applications, review the application's install message to determine how to enable any associated services. The following entries in [.filename]#/etc/rc.conf# enable man:sshd[8], enable Apache 2.4, and specify that Apache should be started with SSL. [.programlisting] .... # enable SSHD sshd_enable="YES" # enable Apache with SSL apache24_enable="YES" apache24_flags="-DSSL" .... Once a service has been enabled in [.filename]#/etc/rc.conf#, it can be started without rebooting the system: [source,shell] .... # service sshd start # service apache24 start .... If a service has not been enabled, it can be started from the command line using `onestart`: [source,shell] .... # service sshd onestart .... [[network]] == Network Configuration Instead of a generic _ethX_ identifier that Linux(R) uses to identify a network interface, FreeBSD uses the driver name followed by a number. The following output from man:ifconfig[8] shows two Intel(R) Pro 1000 network interfaces ([.filename]#em0# and [.filename]#em1#): [source,shell] .... % ifconfig em0: flags=8843 mtu 1500 options=b inet 10.10.10.100 netmask 0xffffff00 broadcast 10.10.10.255 ether 00:50:56:a7:70:b2 media: Ethernet autoselect (1000baseTX ) status: active em1: flags=8843 mtu 1500 options=b inet 192.168.10.222 netmask 0xffffff00 broadcast 192.168.10.255 ether 00:50:56:a7:03:2b media: Ethernet autoselect (1000baseTX ) status: active .... An IP address can be assigned to an interface using man:ifconfig[8]. To remain persistent across reboots, the IP configuration must be included in [.filename]#/etc/rc.conf#. The following [.filename]#/etc/rc.conf# entries specify the hostname, IP address, and default gateway: [.programlisting] .... hostname="server1.example.com" ifconfig_em0="inet 10.10.10.100 netmask 255.255.255.0" defaultrouter="10.10.10.1" .... Use the following entries to instead configure an interface for DHCP: [.programlisting] .... hostname="server1.example.com" ifconfig_em0="DHCP" .... [[firewall]] == Firewall FreeBSD does not use Linux(R) IPTABLES for its firewall. Instead, FreeBSD offers a choice of three kernel level firewalls: * extref:{handbook}[PF, firewalls-pf] * extref:{handbook}[IPFILTER, firewalls-ipf] * extref:{handbook}[IPFW, firewalls-ipfw] PF is developed by the OpenBSD project and ported to FreeBSD. PF was created as a replacement for IPFILTER and its syntax is similar to that of IPFILTER. PF can be paired with man:altq[4] to provide QoS features. This sample PF entry allows inbound SSH: [.programlisting] .... pass in on $ext_if inet proto tcp from any to ($ext_if) port 22 .... IPFILTER is the firewall application developed by Darren Reed. It is not specific to FreeBSD and has been ported to several operating systems including NetBSD, OpenBSD, SunOS, HP/UX, and Solaris. The IPFILTER syntax to allow inbound SSH is: [.programlisting] .... pass in on $ext_if proto tcp from any to any port = 22 .... IPFW is the firewall developed and maintained by FreeBSD. It can be paired with man:dummynet[4] to provide traffic shaping capabilities and simulate different types of network connections. The IPFW syntax to allow inbound SSH would be: [.programlisting] .... ipfw add allow tcp from any to me 22 in via $ext_if .... [[updates]] == Updating FreeBSD There are two methods for updating a FreeBSD system: from source or binary updates. Updating from source is the most involved update method, but offers the greatest amount of flexibility. The process involves synchronizing a local copy of the FreeBSD source code with the FreeBSD Subversion servers. Once the local source code is up-to-date, a new version of the kernel and userland can be compiled. Binary updates are similar to using `yum` or `apt-get` to update a Linux(R) system. In FreeBSD, man:freebsd-update[8] can be used fetch new binary updates and install them. These updates can be scheduled using man:cron[8]. [NOTE] ==== When using man:cron[8] to schedule updates, use `freebsd-update cron` in the man:crontab[1] to reduce the possibility of a large number of machines all pulling updates at the same time: [.programlisting] .... 0 3 * * * root /usr/sbin/freebsd-update cron .... ==== For more information on source and binary updates, refer to extref:{handbook}[the chapter on updating, updating-upgrading] in the FreeBSD Handbook. [[procfs]] == procfs: Gone But Not Forgotten In some Linux(R) distributions, one could look at [.filename]#/proc/sys/net/ipv4/ip_forward# to determine if IP forwarding is enabled. In FreeBSD, man:sysctl[8] is instead used to view this and other system settings. For example, use the following to determine if IP forwarding is enabled on a FreeBSD system: [source,shell] .... % sysctl net.inet.ip.forwarding net.inet.ip.forwarding: 0 .... Use `-a` to list all the system settings: [source,shell] .... % sysctl -a | more .... If an application requires procfs, add the following entry to [.filename]#/etc/fstab#: [source,shell] .... proc /proc procfs rw,noauto 0 0 .... Including `noauto` will prevent [.filename]#/proc# from being automatically mounted at boot. To mount the file system without rebooting: [source,shell] .... # mount /proc .... [[commands]] == Common Commands Some common command equivalents are as follows: [.informaltable] [cols="1,1,1", frame="none", options="header"] |=== | Linux(R) command (Red Hat/Debian) | FreeBSD equivalent | Purpose |`yum install _package_` / `apt-get install _package_` |`pkg install _package_` |Install package from remote repository |`rpm -ivh _package_` / `dpkg -i _package_` |`pkg add _package_` |Install local package |`rpm -qa` / `dpkg -l` |`pkg info` |List installed packages |`lspci` |`pciconf` |List PCI devices |`lsmod` |`kldstat` |List loaded kernel modules |`modprobe` |`kldload` / `kldunload` |Load/Unload kernel modules |`strace` |`truss` |Trace system calls |=== [[conclusion]] == Conclusion This document has provided an overview of FreeBSD. Refer to the extref:{handbook}[FreeBSD Handbook] for more in-depth coverage of these topics as well as the many topics not covered by this document. diff --git a/documentation/content/en/books/handbook/jails/_index.adoc b/documentation/content/en/books/handbook/jails/_index.adoc index f59416450b..5b2d469e0b 100644 --- a/documentation/content/en/books/handbook/jails/_index.adoc +++ b/documentation/content/en/books/handbook/jails/_index.adoc @@ -1,1231 +1,1231 @@ --- title: Chapter 15. Jails part: Part III. System Administration prev: books/handbook/security next: books/handbook/mac description: Jails improve on the concept of the traditional chroot environment in several ways tags: ["jails", "creating", "managing", "updating", "ezjail"] showBookMenu: true weight: 19 path: "/books/handbook/" aliases: ["/en/books/handbook/jails-terms/","/en/books/handbook/jails-build/","/en/books/handbook/jails-tuning/","/en/books/handbook/jails-application/","/en/books/handbook/jails-ezjail/"] --- [[jails]] = Jails :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :sectnumoffset: 15 :partnums: :source-highlighter: rouge :experimental: :images-path: books/handbook/jails/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[jails-synopsis]] == Synopsis Since system administration is a difficult task, many tools have been developed to make life easier for the administrator. These tools often enhance the way systems are installed, configured, and maintained. One of the tools which can be used to enhance the security of a FreeBSD system is _jails_. Jails have been available since FreeBSD 4.X and continue to be enhanced in their usefulness, performance, reliability, and security. Jails build upon the man:chroot[2] concept, which is used to change the root directory of a set of processes. This creates a safe environment, separate from the rest of the system. Processes created in the chrooted environment can not access files or resources outside of it. For that reason, compromising a service running in a chrooted environment should not allow the attacker to compromise the entire system. However, a chroot has several limitations. It is suited to easy tasks which do not require much flexibility or complex, advanced features. Over time, many ways have been found to escape from a chrooted environment, making it a less than ideal solution for securing services. Jails improve on the concept of the traditional chroot environment in several ways. In a traditional chroot environment, processes are only limited in the part of the file system they can access. The rest of the system resources, system users, running processes, and the networking subsystem are shared by the chrooted processes and the processes of the host system. Jails expand this model by virtualizing access to the file system, the set of users, and the networking subsystem. More fine-grained controls are available for tuning the access of a jailed environment. Jails can be considered as a type of operating system-level virtualization. A jail is characterized by four elements: * A directory subtree: the starting point from which a jail is entered. Once inside the jail, a process is not permitted to escape outside of this subtree. * A hostname: which will be used by the jail. * An IP address: which is assigned to the jail. The IP address of a jail is often an alias address for an existing network interface. * A command: the path name of an executable to run inside the jail. The path is relative to the root directory of the jail environment. Jails have their own set of users and their own `root` account which are limited to the jail environment. The `root` account of a jail is not allowed to perform operations to the system outside of the associated jail environment. This chapter provides an overview of the terminology and commands for managing FreeBSD jails. Jails are a powerful tool for both system administrators, and advanced users. After reading this chapter, you will know: * What a jail is and what purpose it may serve in FreeBSD installations. * How to build, start, and stop a jail. * The basics of jail administration, both from inside and outside the jail. [IMPORTANT] ==== Jails are a powerful tool, but they are not a security panacea. While it is not possible for a jailed process to break out on its own, there are several ways in which an unprivileged user outside the jail can cooperate with a privileged user inside the jail to obtain elevated privileges in the host environment. Most of these attacks can be mitigated by ensuring that the jail root is not accessible to unprivileged users in the host environment. As a general rule, untrusted users with privileged access to a jail should not be given access to the host environment. ==== [[jails-terms]] == Terms Related to Jails To facilitate better understanding of parts of the FreeBSD system related to jails, their internals and the way they interact with the rest of FreeBSD, the following terms are used further in this chapter: man:chroot[8] (command):: Utility, which uses man:chroot[2] FreeBSD system call to change the root directory of a process and all its descendants. man:chroot[2] (environment):: The environment of processes running in a "chroot". This includes resources such as the part of the file system which is visible, user and group IDs which are available, network interfaces and other IPC mechanisms, etc. man:jail[8] (command):: The system administration utility which allows launching of processes within a jail environment. host (system, process, user, etc.):: The controlling system of a jail environment. The host system has access to all the hardware resources available, and can control processes both outside of and inside a jail environment. One of the important differences of the host system from a jail is that the limitations which apply to superuser processes inside a jail are not enforced for processes of the host system. hosted (system, process, user, etc.):: A process, user or other entity, whose access to resources is restricted by a FreeBSD jail. [[jails-build]] == Creating and Controlling Jails Some administrators divide jails into the following two types: "complete" jails, which resemble a real FreeBSD system, and "service" jails, dedicated to one application or service, possibly running with privileges. This is only a conceptual division and the process of building a jail is not affected by it. When creating a "complete" jail there are two options for the source of the userland: use prebuilt binaries (such as those supplied on an install media) or build from source. === Installing a Jail [[jails-install-internet]] ==== To install a Jail from the Internet The man:bsdinstall[8] tool can be used to fetch and install the binaries needed for a jail. This will walk through the picking of a mirror, which distributions will be installed into the destination directory, and some basic configuration of the jail: [source,shell] .... # bsdinstall jail /here/is/the/jail .... Once the command is complete, the next step is configuring the host to run the jail. [[jails-install-iso]] ==== To install a Jail from an ISO To install the userland from installation media, first create the root directory for the jail. This can be done by setting the `DESTDIR` variable to the proper location. Start a shell and define `DESTDIR`: [source,shell] .... # sh # export DESTDIR=/here/is/the/jail .... Mount the install media as covered in man:mdconfig[8] when using the install ISO: [source,shell] .... # mount -t cd9660 /dev/`mdconfig -f cdimage.iso` /mnt # cd /mnt/usr/freebsd-dist/ .... Extract the binaries from the tarballs on the install media into the declared destination. Minimally, only the base set needs to be extracted, but a complete install can be performed when preferred. To install just the base system: [source,shell] .... # tar -xf base.txz -C $DESTDIR .... To install everything except the kernel: [source,shell] .... # for set in base ports; do tar -xf $set.txz -C $DESTDIR ; done .... [[jails-install-source]] ==== To build and install a Jail from source The man:jail[8] manual page explains the procedure for building a jail: [source,shell] .... # setenv D /here/is/the/jail # mkdir -p $D <.> # cd /usr/src # make buildworld <.> # make installworld DESTDIR=$D <.> # make distribution DESTDIR=$D <.> # mount -t devfs devfs $D/dev <.> .... <.> Selecting a location for a jail is the best starting point. This is where the jail will physically reside within the file system of the jail's host. A good choice can be [.filename]#/usr/jail/jailname#, where _jailname_ is the hostname identifying the jail. Usually, [.filename]#/usr/# has enough space for the jail file system, which for "complete" jails is, essentially, a replication of every file present in a default installation of the FreeBSD base system. <.> If you have already rebuilt your userland using `make world` or `make buildworld`, you can skip this step and install your existing userland into the new jail. <.> This command will populate the directory subtree chosen as jail's physical location on the file system with the necessary binaries, libraries, manual pages and so on. <.> The `distribution` target for make installs every needed configuration file. In simple words, it installs every installable file of [.filename]#/usr/src/etc/# to the [.filename]#/etc# directory of the jail environment: [.filename]#$D/etc/#. <.> Mounting the man:devfs[8] file system inside a jail is not required. On the other hand, any, or almost any application requires access to at least one device, depending on the purpose of the given application. It is very important to control access to devices from inside a jail, as improper settings could permit an attacker to do nasty things in the jail. Control over man:devfs[8] is managed through rulesets which are described in the man:devfs[8] and man:devfs.conf[5] manual pages. === Configuring the Host Once a jail is installed, it can be started by using the man:jail[8] utility. The man:jail[8] utility takes four mandatory arguments which are described in the <>. Other arguments may be specified too, e.g., to run the jailed process with the credentials of a specific user. The `_command_` argument depends on the type of the jail; for a _virtual system_, [.filename]#/etc/rc# is a good choice, since it will replicate the startup sequence of a real FreeBSD system. For a _service_ jail, it depends on the service or application that will run within the jail. Jails are often started at boot time and the FreeBSD [.filename]#rc# mechanism provides an easy way to do this. [.procedure] * Configure jail parameters in [.filename]#jail.conf#: + [.programlisting] .... www { host.hostname = www.example.org; # Hostname ip4.addr = 192.168.0.10; # IP address of the jail path = "/usr/jail/www"; # Path to the jail mount.devfs; # Mount devfs inside the jail exec.start = "/bin/sh /etc/rc"; # Start command exec.stop = "/bin/sh /etc/rc.shutdown"; # Stop command } .... + Configure jails to start at boot time in [.filename]#rc.conf#: + [.programlisting] .... jail_enable="YES" # Set to NO to disable starting of any jails .... + The default startup of jails configured in man:jail.conf[5], will run the [.filename]#/etc/rc# script of the jail, which assumes the jail is a complete virtual system. For service jails, the default startup command of the jail should be changed, by setting the `exec.start` option appropriately. + [NOTE] ==== For a full list of available options, please see the man:jail.conf[5] manual page. ==== man:service[8] can be used to start or stop a jail by hand, if an entry for it exists in [.filename]#jail.conf#: [source,shell] .... # service jail start www # service jail stop www .... Jails can be shut down with man:jexec[8]. Use man:jls[8] to identify the jail's `JID`, then use man:jexec[8] to run the shutdown script in that jail. [source,shell] .... # jls JID IP Address Hostname Path 3 192.168.0.10 www /usr/jail/www # jexec 3 /etc/rc.shutdown .... More information about this can be found in the man:jail[8] manual page. [[jails-tuning]] == Fine Tuning and Administration There are several options which can be set for any jail, and various ways of combining a host FreeBSD system with jails, to produce higher level applications. This section presents: * Some of the options available for tuning the behavior and security restrictions implemented by a jail installation. * Some of the high-level applications for jail management, which are available through the FreeBSD Ports Collection, and can be used to implement overall jail-based solutions. [[jails-tuning-utilities]] === System Tools for Jail Tuning in FreeBSD Fine tuning of a jail's configuration is mostly done by setting man:sysctl[8] variables. A special subtree of sysctl exists as a basis for organizing all the relevant options: the `security.jail.*` hierarchy of FreeBSD kernel options. Here is a list of the main jail-related sysctls, complete with their default value. Names should be self-explanatory, but for more information about them, please refer to the man:jail[8] and man:sysctl[8] manual pages. * `security.jail.set_hostname_allowed: 1` * `security.jail.socket_unixiproute_only: 1` * `security.jail.sysvipc_allowed: 0` * `security.jail.enforce_statfs: 2` * `security.jail.allow_raw_sockets: 0` * `security.jail.chflags_allowed: 0` * `security.jail.jailed: 0` These variables can be used by the system administrator of the _host system_ to add or remove some of the limitations imposed by default on the `root` user. Note that there are some limitations which cannot be removed. The `root` user is not allowed to mount or unmount file systems from within a man:jail[8]. The `root` inside a jail may not load or unload man:devfs[8] rulesets, set firewall rules, or do many other administrative tasks which require modifications of in-kernel data, such as setting the `securelevel` of the kernel. The base system of FreeBSD contains a basic set of tools for viewing information about the active jails, and attaching to a jail to run administrative commands. The man:jls[8] and man:jexec[8] commands are part of the base FreeBSD system, and can be used to perform the following simple tasks: * Print a list of active jails and their corresponding jail identifier (JID), IP address, hostname and path. * Attach to a running jail, from its host system, and run a command inside the jail or perform administrative tasks inside the jail itself. This is especially useful when the `root` user wants to cleanly shut down a jail. The man:jexec[8] utility can also be used to start a shell in a jail to do administration in it; for example: + [source,shell] .... # jexec 1 tcsh .... [[jails-tuning-admintools]] === High-Level Administrative Tools in the FreeBSD Ports Collection Among the many third-party utilities for jail administration, one of the most complete and useful is package:sysutils/ezjail[]. It is a set of scripts that contribute to man:jail[8] management. Please refer to <> for more information. [[jails-updating]] === Keeping Jails Patched and up to Date Jails should be kept up to date from the host operating system as attempting to patch userland from within the jail may likely fail as the default behavior in FreeBSD is to disallow the use of man:chflags[1] in a jail which prevents the replacement of some files. It is possible to change this behavior but it is recommended to use man:freebsd-update[8] to maintain jails instead. Use `-b` to specify the path of the jail to be updated. To update the jail to the latest patch release of the version of FreeBSD it is already running, then execute the following commands on the host: [source,shell] .... # freebsd-update -b /here/is/the/jail fetch # freebsd-update -b /here/is/the/jail install .... To upgrade the jail to a new major or minor version, first upgrade the host system as described in crossref:cutting-edge[freebsdupdate-upgrade,“Performing Major and Minor Version Upgrades”]. Once the host has been upgraded and rebooted, the jail can then be upgraded. For example to upgrade from 12.0-RELEASE to 12.1-RELEASE, on the host run: [source,shell] .... # freebsd-update -b /here/is/the/jail --currently-running 12.0-RELEASE -r 12.1-RELEASE upgrade # freebsd-update -b /here/is/the/jail install # service jail restart myjail # freebsd-update -b /here/is/the/jail install .... Then, if it was a major version upgrade, reinstall all installed packages and restart the jail again. This is required because the ABI version changes when upgrading between major versions of FreeBSD. From the host: [source,shell] .... # pkg -j myjail upgrade -f # service jail restart myjail .... [[jails-application]] == Updating Multiple Jails The management of multiple jails can become problematic because every jail has to be rebuilt from scratch whenever it is upgraded. This can be time consuming and tedious if a lot of jails are created and manually updated. This section demonstrates one method to resolve this issue by safely sharing as much as is possible between jails using read-only man:mount_nullfs[8] mounts, so that updating is simpler. This makes it more attractive to put single services, such as HTTP, DNS, and SMTP, into individual jails. Additionally, it provides a simple way to add, remove, and upgrade jails. [NOTE] ==== Simpler solutions exist, such as ezjail, which provides an easier method of administering FreeBSD jails but is less versatile than this setup. ezjail is covered in more detail in <>. ==== The goals of the setup described in this section are: * Create a simple and easy to understand jail structure that does not require running a full installworld on each and every jail. * Make it easy to add new jails or remove existing ones. * Make it easy to update or upgrade existing jails. * Make it possible to run a customized FreeBSD branch. * Be paranoid about security, reducing as much as possible the possibility of compromise. * Save space and inodes, as much as possible. This design relies on a single, read-only master template which is mounted into each jail and one read-write device per jail. A device can be a separate physical disc, a partition, or a vnode backed memory device. This example uses read-write nullfs mounts. The file system layout is as follows: * The jails are based under the [.filename]#/home# partition. * Each jail will be mounted under the [.filename]#/home/j# directory. * The template for each jail and the read-only partition for all of the jails is [.filename]#/home/j/mroot#. * A blank directory will be created for each jail under the [.filename]#/home/j# directory. * Each jail will have a [.filename]#/s# directory that will be linked to the read-write portion of the system. * Each jail will have its own read-write system that is based upon [.filename]#/home/j/skel#. * The read-write portion of each jail will be created in [.filename]#/home/js#. [[jails-service-jails-template]] === Creating the Template This section describes the steps needed to create the master template. It is recommended to first update the host FreeBSD system to the latest -RELEASE branch using the instructions in crossref:cutting-edge[makeworld,“Updating FreeBSD from Source”]. -Additionally, this template uses the package:sysutils/cpdup[] package or port and portsnap will be used to download the FreeBSD Ports Collection. +Additionally, this template uses the package:sysutils/cpdup[] package or port and link:{handbook}mirrors/#git[Git] will be used to download the FreeBSD Ports Collection. [.procedure] . First, create a directory structure for the read-only file system which will contain the FreeBSD binaries for the jails. Then, change directory to the FreeBSD source tree and install the read-only file system to the jail template: + [source,shell] .... # mkdir /home/j /home/j/mroot # cd /usr/src # make installworld DESTDIR=/home/j/mroot .... . Next, prepare a FreeBSD Ports Collection for the jails as well as a FreeBSD source tree, which is required for mergemaster: + [source,shell] .... # cd /home/j/mroot # mkdir usr/ports -# portsnap -p /home/j/mroot/usr/ports fetch extract +# git clone -o freebsd https://git.FreeBSD.org/ports.git /home/j/mroot/usr/ports # cpdup /usr/src /home/j/mroot/usr/src .... . Create a skeleton for the read-write portion of the system: + [source,shell] .... # mkdir /home/j/skel /home/j/skel/home /home/j/skel/usr-X11R6 /home/j/skel/distfiles # mv etc /home/j/skel # mv usr/local /home/j/skel/usr-local # mv tmp /home/j/skel # mv var /home/j/skel # mv root /home/j/skel .... . Use mergemaster to install missing configuration files. Then, remove the extra directories that mergemaster creates: + [source,shell] .... # mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i # cd /home/j/skel # rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev .... . Now, symlink the read-write file system to the read-only file system. Ensure that the symlinks are created in the correct [.filename]#s/# locations as the creation of directories in the wrong locations will cause the installation to fail. + [source,shell] .... # cd /home/j/mroot # mkdir s # ln -s s/etc etc # ln -s s/home home # ln -s s/root root # ln -s ../s/usr-local usr/local # ln -s ../s/usr-X11R6 usr/X11R6 # ln -s ../../s/distfiles usr/ports/distfiles # ln -s s/tmp tmp # ln -s s/var var .... . As a last step, create a generic [.filename]#/home/j/skel/etc/make.conf# containing this line: + [.programlisting] .... WRKDIRPREFIX?= /s/portbuild .... + This makes it possible to compile FreeBSD ports inside each jail. Remember that the ports directory is part of the read-only system. The custom path for `WRKDIRPREFIX` allows builds to be done in the read-write portion of every jail. [[jails-service-jails-creating]] === Creating Jails The jail template can now be used to setup and configure the jails in [.filename]#/etc/rc.conf#. This example demonstrates the creation of 3 jails: `NS`, `MAIL` and `WWW`. [.procedure] . Add the following lines to [.filename]#/etc/fstab#, so that the read-only template for the jails and the read-write space will be available in the respective jails: + [.programlisting] .... /home/j/mroot /home/j/ns nullfs ro 0 0 /home/j/mroot /home/j/mail nullfs ro 0 0 /home/j/mroot /home/j/www nullfs ro 0 0 /home/js/ns /home/j/ns/s nullfs rw 0 0 /home/js/mail /home/j/mail/s nullfs rw 0 0 /home/js/www /home/j/www/s nullfs rw 0 0 .... + To prevent fsck from checking nullfs mounts during boot and dump from backing up the read-only nullfs mounts of the jails, the last two columns are both set to `0`. . Configure the jails in [.filename]#/etc/rc.conf#: + [.programlisting] .... jail_enable="YES" jail_set_hostname_allow="NO" jail_list="ns mail www" jail_ns_hostname="ns.example.org" jail_ns_ip="192.168.3.17" jail_ns_rootdir="/usr/home/j/ns" jail_ns_devfs_enable="YES" jail_mail_hostname="mail.example.org" jail_mail_ip="192.168.3.18" jail_mail_rootdir="/usr/home/j/mail" jail_mail_devfs_enable="YES" jail_www_hostname="www.example.org" jail_www_ip="62.123.43.14" jail_www_rootdir="/usr/home/j/www" jail_www_devfs_enable="YES" .... + The `jail__name__rootdir` variable is set to [.filename]#/usr/home# instead of [.filename]#/home# because the physical path of [.filename]#/home# on a default FreeBSD installation is [.filename]#/usr/home#. The `jail__name__rootdir` variable must _not_ be set to a path which includes a symbolic link, otherwise the jails will refuse to start. . Create the required mount points for the read-only file system of each jail: + [source,shell] .... # mkdir /home/j/ns /home/j/mail /home/j/www .... . Install the read-write template into each jail using package:sysutils/cpdup[]: + [source,shell] .... # mkdir /home/js # cpdup /home/j/skel /home/js/ns # cpdup /home/j/skel /home/js/mail # cpdup /home/j/skel /home/js/www .... . In this phase, the jails are built and prepared to run. First, mount the required file systems for each jail, and then start them: + [source,shell] .... # mount -a # service jail start .... The jails should be running now. To check if they have started correctly, use `jls`. Its output should be similar to the following: [source,shell] .... # jls JID IP Address Hostname Path 3 192.168.3.17 ns.example.org /home/j/ns 2 192.168.3.18 mail.example.org /home/j/mail 1 62.123.43.14 www.example.org /home/j/www .... At this point, it should be possible to log onto each jail, add new users, or configure daemons. The `JID` column indicates the jail identification number of each running jail. Use the following command to perform administrative tasks in the jail whose JID is `3`: [source,shell] .... # jexec 3 tcsh .... [[jails-service-jails-upgrading]] === Upgrading The design of this setup provides an easy way to upgrade existing jails while minimizing their downtime. Also, it provides a way to roll back to the older version should a problem occur. [.procedure] . The first step is to upgrade the host system. Then, create a new temporary read-only template in [.filename]#/home/j/mroot2#. + [source,shell] .... # mkdir /home/j/mroot2 # cd /usr/src # make installworld DESTDIR=/home/j/mroot2 # cd /home/j/mroot2 # cpdup /usr/src usr/src # mkdir s .... + The `installworld` creates a few unnecessary directories, which should be removed: + [source,shell] .... # chflags -R 0 var # rm -R etc var root usr/local tmp .... . Recreate the read-write symlinks for the master file system: + [source,shell] .... # ln -s s/etc etc # ln -s s/root root # ln -s s/home home # ln -s ../s/usr-local usr/local # ln -s ../s/usr-X11R6 usr/X11R6 # ln -s s/tmp tmp # ln -s s/var var .... . Next, stop the jails: + [source,shell] .... # service jail stop .... . Unmount the original file systems as the read-write systems are attached to the read-only system ([.filename]#/s#): + [source,shell] .... # umount /home/j/ns/s # umount /home/j/ns # umount /home/j/mail/s # umount /home/j/mail # umount /home/j/www/s # umount /home/j/www .... . Move the old read-only file system and replace it with the new one. This will serve as a backup and archive of the old read-only file system should something go wrong. The naming convention used here corresponds to when a new read-only file system has been created. Move the original FreeBSD Ports Collection over to the new file system to save some space and inodes: + [source,shell] .... # cd /home/j # mv mroot mroot.20060601 # mv mroot2 mroot # mv mroot.20060601/usr/ports mroot/usr .... . At this point the new read-only template is ready, so the only remaining task is to remount the file systems and start the jails: + [source,shell] .... # mount -a # service jail start .... Use `jls` to check if the jails started correctly. Run `mergemaster` in each jail to update the configuration files. [[jails-ezjail]] == Managing Jails with ezjail Creating and managing multiple jails can quickly become tedious and error-prone. Dirk Engling's ezjail automates and greatly simplifies many jail tasks. A _basejail_ is created as a template. Additional jails use man:mount_nullfs[8] to share many of the basejail directories without using additional disk space. Each additional jail takes only a few megabytes of disk space before applications are installed. Upgrading the copy of the userland in the basejail automatically upgrades all of the other jails. Additional benefits and features are described in detail on the ezjail web site, https://erdgeist.org/arts/software/ezjail/[]. [[jails-ezjail-install]] === Installing ezjail Installing ezjail consists of adding a loopback interface for use in jails, installing the port or package, and enabling the service. [[jails-ezjail-install-procedure]] [.procedure] . To keep jail loopback traffic off the host's loopback network interface `lo0`, a second loopback interface is created by adding an entry to [.filename]#/etc/rc.conf#: + [.programlisting] .... cloned_interfaces="lo1" .... + The second loopback interface `lo1` will be created when the system starts. It can also be created manually without a restart: + [source,shell] .... # service netif cloneup Created clone interfaces: lo1. .... + Jails can be allowed to use aliases of this secondary loopback interface without interfering with the host. + Inside a jail, access to the loopback address `127.0.0.1` is redirected to the first IP address assigned to the jail. To make the jail loopback correspond with the new `lo1` interface, that interface must be specified first in the list of interfaces and IP addresses given when creating a new jail. + Give each jail a unique loopback address in the `127.0.0.0/8` netblock. . Install package:sysutils/ezjail[]: + [source,shell] .... # cd /usr/ports/sysutils/ezjail # make install clean .... . Enable ezjail by adding this line to [.filename]#/etc/rc.conf#: + [.programlisting] .... ezjail_enable="YES" .... . The service will automatically start on system boot. It can be started immediately for the current session: + [source,shell] .... # service ezjail start .... [[jails-ezjail-initialsetup]] === Initial Setup With ezjail installed, the basejail directory structure can be created and populated. This step is only needed once on the jail host computer. In both of these examples, `-p` causes the ports tree to be retrieved with man:portsnap[8] into the basejail. That single copy of the ports directory will be shared by all the jails. Using a separate copy of the ports directory for jails isolates them from the host. The ezjailFAQ explains in more detail: http://erdgeist.org/arts/software/ezjail/#FAQ[]. [[jails-ezjail-initialsetup-procedure]] [.procedure] . To Populate the Jail with FreeBSD-RELEASE + For a basejail based on the FreeBSD RELEASE matching that of the host computer, use `install`. For example, on a host computer running FreeBSD 10-STABLE, the latest RELEASE version of FreeBSD -10 will be installed in the jail): + [source,shell] .... # ezjail-admin install -p .... . To Populate the Jail with `installworld` + The basejail can be installed from binaries created by `buildworld` on the host with `ezjail-admin update`. + In this example, FreeBSD 10-STABLE has been built from source. The jail directories are created. Then `installworld` is executed, installing the host's [.filename]#/usr/obj# into the basejail. + [source,shell] .... # ezjail-admin update -i -p .... + The host's [.filename]#/usr/src# is used by default. A different source directory on the host can be specified with `-s` and a path, or set with `ezjail_sourcetree` in [.filename]#/usr/local/etc/ezjail.conf#. [TIP] ==== The basejail's ports tree is shared by other jails. However, downloaded distfiles are stored in the jail that downloaded them. By default, these files are stored in [.filename]#/var/ports/distfiles# within each jail. [.filename]#/var/ports# inside each jail is also used as a work directory when building ports. ==== [TIP] ==== The FTP protocol is used by default to download packages for the installation of the basejail. Firewall or proxy configurations can prevent or interfere with FTP transfers. The HTTP protocol works differently and avoids these problems. It can be chosen by specifying a full URL for a particular download mirror in [.filename]#/usr/local/etc/ezjail.conf#: [.programlisting] .... ezjail_ftphost=http://ftp.FreeBSD.org .... See the crossref:mirrors[mirrors,mirrors] section for a list of sites. ==== [[jails-ezjail-create]] === Creating and Starting a New Jail New jails are created with `ezjail-admin create`. In these examples, the `lo1` loopback interface is used as described above. [[jails-ezjail-create-steps]] [.procedure] .Procedure: Create and Start a New Jail . Create the jail, specifying a name and the loopback and network interfaces to use, along with their IP addresses. In this example, the jail is named `dnsjail`. + [source,shell] .... # ezjail-admin create dnsjail 'lo1|127.0.1.1,em0|192.168.1.50' .... + [TIP] ==== Most network services run in jails without problems. A few network services, most notably man:ping[8], use _raw network sockets_. In jails, raw network sockets are disabled by default for security. Services that require them will not work. Occasionally, a jail genuinely needs raw sockets. For example, network monitoring applications often use man:ping[8] to check the availability of other computers. When raw network sockets are actually needed in a jail, they can be enabled by editing the ezjail configuration file for the individual jail, [.filename]#/usr/local/etc/ezjail/jailname#. Modify the `parameters` entry: [.programlisting] .... export jail_jailname_parameters="allow.raw_sockets=1" .... Do not enable raw network sockets unless services in the jail actually require them. ==== . Start the jail: + [source,shell] .... # ezjail-admin start dnsjail .... . Use a console on the jail: + [source,shell] .... # ezjail-admin console dnsjail .... The jail is operating and additional configuration can be completed. Typical settings added at this point include: [.procedure] . Set the `root` Password + Connect to the jail and set the `root` user's password: + [source,shell] .... # ezjail-admin console dnsjail # passwd Changing local password for root New Password: Retype New Password: .... . Time Zone Configuration + The jail's time zone can be set with man:tzsetup[8]. To avoid spurious error messages, the man:adjkerntz[8] entry in [.filename]#/etc/crontab# can be commented or removed. This job attempts to update the computer's hardware clock with time zone changes, but jails are not allowed to access that hardware. . DNS Servers + Enter domain name server lines in [.filename]#/etc/resolv.conf# so DNS works in the jail. . Edit [.filename]#/etc/hosts# + Change the address and add the jail name to the `localhost` entries in [.filename]#/etc/hosts#. . Configure [.filename]#/etc/rc.conf# + Enter configuration settings in [.filename]#/etc/rc.conf#. This is much like configuring a full computer. The host name and IP address are not set here. Those values are already provided by the jail configuration. With the jail configured, the applications for which the jail was created can be installed. [TIP] ==== Some ports must be built with special options to be used in a jail. For example, both of the network monitoring plugin packages package:net-mgmt/nagios-plugins[] and package:net-mgmt/monitoring-plugins[] have a `JAIL` option which must be enabled for them to work correctly inside a jail. ==== [[jails-ezjail-update]] === Updating Jails [[jails-ezjail-update-os]] ==== Updating the Operating System Because the basejail's copy of the userland is shared by the other jails, updating the basejail automatically updates all of the other jails. Either source or binary updates can be used. To build the world from source on the host, then install it in the basejail, use: [source,shell] .... # ezjail-admin update -b .... If the world has already been compiled on the host, install it in the basejail with: [source,shell] .... # ezjail-admin update -i .... Binary updates use man:freebsd-update[8]. These updates have the same limitations as if man:freebsd-update[8] were being run directly. The most important one is that only -RELEASE versions of FreeBSD are available with this method. Update the basejail to the latest patched release of the version of FreeBSD on the host. For example, updating from RELEASE-p1 to RELEASE-p2. [source,shell] .... # ezjail-admin update -u .... To upgrade the basejail to a new version, first upgrade the host system as described in crossref:cutting-edge[freebsdupdate-upgrade,“Performing Major and Minor Version Upgrades”]. Once the host has been upgraded and rebooted, the basejail can then be upgraded. man:freebsd-update[8] has no way of determining which version is currently installed in the basejail, so the original version must be specified. Use man:file[1] to determine the original version in the basejail: [source,shell] .... # file /usr/jails/basejail/bin/sh /usr/jails/basejail/bin/sh: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 9.3, stripped .... Now use this information to perform the upgrade from `9.3-RELEASE` to the current version of the host system: [source,shell] .... # ezjail-admin update -U -s 9.3-RELEASE .... After updating the basejail, man:mergemaster[8] must be run to update each jail's configuration files. How to use man:mergemaster[8] depends on the purpose and trustworthiness of a jail. If a jail's services or users are not trusted, then man:mergemaster[8] should only be run from within that jail: [[jails-ezjail-update-mergemaster-untrusted]] .man:mergemaster[8] on Untrusted Jail [example] ==== Delete the link from the jail's [.filename]#/usr/src# into the basejail and create a new [.filename]#/usr/src# in the jail as a mountpoint. Mount the host computer's [.filename]#/usr/src# read-only on the jail's new [.filename]#/usr/src# mountpoint: [source,shell] .... # rm /usr/jails/jailname/usr/src # mkdir /usr/jails/jailname/usr/src # mount -t nullfs -o ro /usr/src /usr/jails/jailname/usr/src .... Get a console in the jail: [source,shell] .... # ezjail-admin console jailname .... Inside the jail, run `mergemaster`. Then exit the jail console: [source,shell] .... # cd /usr/src # mergemaster -U # exit .... Finally, unmount the jail's [.filename]#/usr/src#: [source,shell] .... # umount /usr/jails/jailname/usr/src .... ==== [[jails-ezjail-update-mergemaster-trusted]] .man:mergemaster[8] on Trusted Jail [example] ==== If the users and services in a jail are trusted, man:mergemaster[8] can be run from the host: [source,shell] .... # mergemaster -U -D /usr/jails/jailname .... ==== [TIP] ==== After a major version update it is recommended by package:sysutils/ezjail[] to make sure your `pkg` is of the correct version. Therefore enter: [source,shell] .... # pkg-static upgrade -f pkg .... to upgrade or downgrade to the appropriate version. ==== [[jails-ezjail-update-ports]] ==== Updating Ports The ports tree in the basejail is shared by the other jails. Updating that copy of the ports tree gives the other jails the updated version also. The basejail ports tree is updated with man:portsnap[8]: [source,shell] .... # ezjail-admin update -P .... [[jails-ezjail-control]] === Controlling Jails [[jails-ezjail-control-stop-start]] ==== Stopping and Starting Jails ezjail automatically starts jails when the computer is started. Jails can be manually stopped and restarted with `stop` and `start`: [source,shell] .... # ezjail-admin stop sambajail Stopping jails: sambajail. .... By default, jails are started automatically when the host computer starts. Autostarting can be disabled with `config`: [source,shell] .... # ezjail-admin config -r norun seldomjail .... This takes effect the next time the host computer is started. A jail that is already running will not be stopped. Enabling autostart is very similar: [source,shell] .... # ezjail-admin config -r run oftenjail .... [[jails-ezjail-control-backup]] ==== Archiving and Restoring Jails Use `archive` to create a [.filename]#.tar.gz# archive of a jail. The file name is composed from the name of the jail and the current date. Archive files are written to the archive directory, [.filename]#/usr/jails/ezjail_archives#. A different archive directory can be chosen by setting `ezjail_archivedir` in the configuration file. The archive file can be copied elsewhere as a backup, or an existing jail can be restored from it with `restore`. A new jail can be created from the archive, providing a convenient way to clone existing jails. Stop and archive a jail named `wwwserver`: [source,shell] .... # ezjail-admin stop wwwserver Stopping jails: wwwserver. # ezjail-admin archive wwwserver # ls /usr/jails/ezjail-archives/ wwwserver-201407271153.13.tar.gz .... Create a new jail named `wwwserver-clone` from the archive created in the previous step. Use the [.filename]#em1# interface and assign a new IP address to avoid conflict with the original: [source,shell] .... # ezjail-admin create -a /usr/jails/ezjail_archives/wwwserver-201407271153.13.tar.gz wwwserver-clone 'lo1|127.0.3.1,em1|192.168.1.51' .... [[jails-ezjail-example-bind]] === Full Example: BIND in a Jail Putting the BINDDNS server in a jail improves security by isolating it. This example creates a simple caching-only name server. * The jail will be called `dns1`. * The jail will use IP address `192.168.1.240` on the host's `re0` interface. * The upstream ISP's DNS servers are at `10.0.0.62` and `10.0.0.61`. * The basejail has already been created and a ports tree installed as shown in <>. [[jails-ezjail-example-bind-steps]] .Running BIND in a Jail [example] ==== Create a cloned loopback interface by adding a line to [.filename]#/etc/rc.conf#: [.programlisting] .... cloned_interfaces="lo1" .... Immediately create the new loopback interface: [source,shell] .... # service netif cloneup Created clone interfaces: lo1. .... Create the jail: [source,shell] .... # ezjail-admin create dns1 'lo1|127.0.2.1,re0|192.168.1.240' .... Start the jail, connect to a console running on it, and perform some basic configuration: [source,shell] .... # ezjail-admin start dns1 # ezjail-admin console dns1 # passwd Changing local password for root New Password: Retype New Password: # tzsetup # sed -i .bak -e '/adjkerntz/ s/^/#/' /etc/crontab # sed -i .bak -e 's/127.0.0.1/127.0.2.1/g; s/localhost.my.domain/dns1.my.domain dns1/' /etc/hosts .... Temporarily set the upstream DNS servers in [.filename]#/etc/resolv.conf# so ports can be downloaded: [.programlisting] .... nameserver 10.0.0.62 nameserver 10.0.0.61 .... Still using the jail console, install package:dns/bind99[]. [source,shell] .... # make -C /usr/ports/dns/bind99 install clean .... Configure the name server by editing [.filename]#/usr/local/etc/namedb/named.conf#. Create an Access Control List (ACL) of addresses and networks that are permitted to send DNS queries to this name server. This section is added just before the `options` section already in the file: [.programlisting] .... ... // or cause huge amounts of useless Internet traffic. acl "trusted" { 192.168.1.0/24; localhost; localnets; }; options { ... .... Use the jail IP address in the `listen-on` setting to accept DNS queries from other computers on the network: [.programlisting] .... listen-on { 192.168.1.240; }; .... A simple caching-only DNS name server is created by changing the `forwarders` section. The original file contains: [.programlisting] .... /* forwarders { 127.0.0.1; }; */ .... Uncomment the section by removing the `/\*` and `*/` lines. Enter the IP addresses of the upstream DNS servers. Immediately after the `forwarders` section, add references to the `trusted` ACL defined earlier: [.programlisting] .... forwarders { 10.0.0.62; 10.0.0.61; }; allow-query { any; }; allow-recursion { trusted; }; allow-query-cache { trusted; }; .... Enable the service in [.filename]#/etc/rc.conf#: [.programlisting] .... named_enable="YES" .... Start and test the name server: [source,shell] .... # service named start wrote key file "/usr/local/etc/namedb/rndc.key" Starting named. # /usr/local/bin/dig @192.168.1.240 freebsd.org .... A response that includes [source,shell] .... ;; Got answer; .... shows that the new DNS server is working. A long delay followed by a response including [source,shell] .... ;; connection timed out; no servers could be reached .... shows a problem. Check the configuration settings and make sure any local firewalls allow the new DNS access to the upstream DNS servers. The new DNS server can use itself for local name resolution, just like other local computers. Set the address of the DNS server in the client computer's [.filename]#/etc/resolv.conf#: [.programlisting] .... nameserver 192.168.1.240 .... A local DHCP server can be configured to provide this address for a local DNS server, providing automatic configuration on DHCP clients. ==== diff --git a/documentation/content/en/books/handbook/mirrors/_index.adoc b/documentation/content/en/books/handbook/mirrors/_index.adoc index b97c43d334..34a866c3b1 100644 --- a/documentation/content/en/books/handbook/mirrors/_index.adoc +++ b/documentation/content/en/books/handbook/mirrors/_index.adoc @@ -1,577 +1,577 @@ --- title: Appendix A. Obtaining FreeBSD part: Part V. Appendices prev: books/handbook/partv next: books/handbook/bibliography description: "How to get FreeBSD: CD and DVD sets, FTP sites and how to install and use Git" tags: ["Obtaining", "CD", "DVD", "FTP", "Git"] showBookMenu: true weight: 39 path: "/books/handbook/" aliases: ["/en/books/handbook/mirrors-ftp/","/en/books/handbook/svn/","/en/books/handbook/mirrors-rsync/"] --- [appendix] = Obtaining FreeBSD :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :sectnumoffset: A :partnums: :source-highlighter: rouge :experimental: :images-path: books/handbook/mirrors/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[mirrors]] == Mirrors The official mirrors of the FreeBSD project are made up of many machines operated by the project cluster administrators and behind GeoDNS to direct users to the closest available mirror. Current locations are Australia, Brazil, Japan (two sites), Malaysia, Netherlands, South Africa, Taiwan, United Kingdom, United States of America (California, New Jersey, and Washington). Official mirrors service: [cols="1,1,3"] |=== | Service Name | Protocols | More information | **download.FreeBSD.org** | link:https://download.FreeBSD.org/[https] link:ftp://download.FreeBSD.org/pub/FreeBSD/[ftp] | Same content as `ftp.FreeBSD.org`, `ftp` is a legacy name; `download.FreeBSD.org` is recommended. | **git.FreeBSD.org** | git over `https` and `ssh` | More details on link:https://docs.freebsd.org/en/books/handbook/mirrors/#git[using git] section. | **pkg.FreeBSD.org** | man:pkg[8] over `http` and `https` | Official FreeBSD package repositories used by the man:pkg[8] program. | **vuxml.FreeBSD.org** / **www.VuXML.org** | link:https://www.vuxml.org/[https] | FreeBSD Project VuXML web page. `pkg audit` fetches the list of vulnerabilities from this service. |=== All official mirrors support IPv4 and IPv6. The FreeBSD website (https://www.FreeBSD.org and https://docs.FreeBSD.org) are not hosted in the GeoDNS Infrastructure; there are ongoing studies of its implementation. http://ftp-archive.FreeBSD.org is not in the GeoDNS Infrastructure, hosted in only one location (US). The project is looking for new locations; those willing to sponsor, please reach out to the Cluster Administrators team for more information. Mirror list maintained by the community and other companies: [cols="1,1,3"] |=== |Country | Hostname | Protocols | Australia icon:envelope[link=mailto:{mirrors-australia-email}, title="mirror contact"] | ftp.au.FreeBSD.org | link:http://ftp.au.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.au.FreeBSD.org/pub/FreeBSD[http_v6] link:rsync://ftp.au.FreeBSD.org[rsync] link:rsync://ftp.au.FreeBSD.org[rsync_v6] | | ftp3.au.FreeBSD.org | link:http://ftp3.au.FreeBSD.org/pub/FreeBSD[http] link:ftp://ftp3.au.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp3.au.FreeBSD.org[rsync] | Austria icon:envelope[link=mailto:{mirrors-austria-email}, title="mirror contact"] | ftp.at.FreeBSD.org | link:http://ftp.at.FreeBSD.org/pub/FreeBSD/[http] link:http://ftp.at.FreeBSD.org/pub/FreeBSD/[http_v6] link:ftp://ftp.at.FreeBSD.org/pub/FreeBSD/[ftp] link:ftp://ftp.at.FreeBSD.org/pub/FreeBSD/[ftp_v6] link:rsync://ftp.at.FreeBSD.org/pub/FreeBSD/[rsync] link:rsync://ftp.at.FreeBSD.org/pub/FreeBSD/[rsync_v6] | Brazil icon:envelope[link=mailto:{mirrors-brazil-email}, title="mirror contact"] | ftp2.br.FreeBSD.org | link:http://ftp2.br.FreeBSD.org/FreeBSD[http] link:rsync://ftp2.br.FreeBSD.org[rsync] link:rsync://ftp2.br.FreeBSD.org[rsync_v6] | | ftp3.br.FreeBSD.org | link:http://ftp3.br.FreeBSD.org/pub/FreeBSD[http] link:ftp://ftp3.br.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp3.br.FreeBSD.org[rsync] | Bulgaria icon:envelope[link=mailto:{mirrors-bulgaria-email}, title="mirror contact"] | ftp.bg.FreeBSD.org | link:ftp://ftp.bg.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.bg.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.bg.FreeBSD.org[rsync] link:rsync://ftp.bg.FreeBSD.org[rsync_v6] | Czech Republic icon:envelope[link=mailto:{mirrors-czech-email}, title="mirror contact"] | ftp.cz.FreeBSD.org | link:http://ftp.cz.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.cz.FreeBSD.org/pub/FreeBSD[http_v6] link:rsync://ftp.cz.FreeBSD.org[rsync] link:rsync://ftp.cz.FreeBSD.org[rsync_v6] | Denmark icon:envelope[link=mailto:{mirrors-denmark-email}, title="mirror contact"] | ftp.dk.FreeBSD.org | link:http://ftp.dk.FreeBSD.org/FreeBSD/[http] link:http://ftp.dk.FreeBSD.org/FreeBSD/[http_v6] link:ftp://ftp.dk.FreeBSD.org/FreeBSD/[ftp] link:ftp://ftp.dk.FreeBSD.org/FreeBSD/[ftp_v6] link:rsync://ftp.dk.FreeBSD.org/FreeBSD/[rsync] link:rsync://ftp.dk.FreeBSD.org/FreeBSD/[rsync_v6] | Finland icon:envelope[link=mailto:{mirrors-finland-email}, title="mirror contact"] | ftp.fi.FreeBSD.org | link:ftp://ftp.fi.FreeBSD.org/pub/FreeBSD[ftp] | France icon:envelope[link=mailto:{mirrors-france-email}, title="mirror contact"] | ftp.fr.FreeBSD.org | link:http://ftp.fr.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.fr.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.fr.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.fr.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.fr.FreeBSD.org[rsync] link:rsync://ftp.fr.FreeBSD.org[rsync_v6] | | ftp3.fr.FreeBSD.org | link:ftp://ftp3.fr.FreeBSD.org/pub/FreeBSD[ftp] | | ftp6.fr.FreeBSD.org | link:http://ftp6.fr.FreeBSD.org/pub/FreeBSD[http] link:ftp://ftp6.fr.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp6.fr.FreeBSD.org[rsync] | Germany icon:envelope[link=mailto:{mirrors-germany-email}, title="mirror contact"] | ftp.de.FreeBSD.org | link:ftp://ftp.de.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.de.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.de.FreeBSD.org[rsync] link:rsync://ftp.de.FreeBSD.org[rsync_v6] | | ftp1.de.FreeBSD.org | link:http://ftp1.de.FreeBSD.org/pub/FreeBSD[http] link:http://ftp1.de.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp1.de.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp1.de.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp1.de.FreeBSD.org[rsync] link:rsync://ftp1.de.FreeBSD.org[rsync_v6] | | ftp2.de.FreeBSD.org | link:http://ftp2.de.FreeBSD.org/pub/FreeBSD[http] link:http://ftp2.de.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp2.de.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp2.de.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp2.de.FreeBSD.org[rsync] link:rsync://ftp2.de.FreeBSD.org[rsync_v6] | | ftp5.de.FreeBSD.org | link:ftp://ftp5.de.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp5.de.FreeBSD.org/pub/FreeBSD[ftp_v6] | | ftp7.de.FreeBSD.org | link:http://ftp7.de.FreeBSD.org/pub/FreeBSD[http] link:http://ftp7.de.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp7.de.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp7.de.FreeBSD.org/pub/FreeBSD[ftp_v6] | Greece icon:envelope[link=mailto:{mirrors-greece-email}, title="mirror contact"] | ftp.gr.FreeBSD.org | link:http://ftp.gr.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.gr.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.gr.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.gr.FreeBSD.org/pub/FreeBSD[ftp_v6] | | ftp2.gr.FreeBSD.org | link:http://ftp2.gr.FreeBSD.org/pub/FreeBSD[http] link:http://ftp2.gr.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp2.gr.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp2.gr.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp2.gr.FreeBSD.org[rsync] | Japan icon:envelope[link=mailto:{mirrors-japan-email}, title="mirror contact"] | ftp.jp.FreeBSD.org | link:http://ftp.jp.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.jp.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.jp.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.jp.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.jp.FreeBSD.org[rsync] link:rsync://ftp.jp.FreeBSD.org[rsync_v6] | | ftp2.jp.FreeBSD.org | link:ftp://ftp2.jp.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp2.jp.FreeBSD.org[rsync] link:rsync://ftp2.jp.FreeBSD.org[rsync_v6] | | ftp3.jp.FreeBSD.org | link:http://ftp3.jp.FreeBSD.org/pub/FreeBSD[http] link:rsync://ftp3.jp.FreeBSD.org[rsync] | | ftp4.jp.FreeBSD.org | link:ftp://ftp4.jp.FreeBSD.org/pub/FreeBSD[ftp] | | ftp6.jp.FreeBSD.org | link:http://ftp6.jp.FreeBSD.org/pub/FreeBSD[http] link:http://ftp6.jp.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp6.jp.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp6.jp.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp6.jp.FreeBSD.org[rsync] link:rsync://ftp6.jp.FreeBSD.org[rsync_v6] | Korea icon:envelope[link=mailto:{mirrors-korea-email}, title="mirror contact"] | ftp.kr.FreeBSD.org | link:http://ftp.kr.FreeBSD.org/pub/FreeBSD[http] link:https://ftp.kr.FreeBSD.org/pub/FreeBSD[https] link:ftp://ftp.kr.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp.kr.FreeBSD.org[rsync] | | ftp2.kr.FreeBSD.org | link:rsync://ftp2.kr.FreeBSD.org[rsync] | Latvia icon:envelope[link=mailto:{mirrors-latvia-email}, title="mirror contact"] | ftp.lv.FreeBSD.org | link:http://ftp.lv.FreeBSD.org/freebsd[http] link:ftp://ftp.lv.FreeBSD.org/freebsd[ftp] | Netherlands icon:envelope[link=mailto:{mirrors-netherlands-email}, title="mirror contact"] | ftp.nl.FreeBSD.org | link:http://ftp.nl.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.nl.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.nl.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.nl.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.nl.FreeBSD.org[rsync] link:rsync://ftp.nl.FreeBSD.org[rsync_v6] | | ftp2.nl.FreeBSD.org | link:http://ftp2.nl.FreeBSD.org/pub/FreeBSD[http] link:ftp://ftp2.nl.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp2.nl.FreeBSD.org[rsync] | New Zealand icon:envelope[link=mailto:{mirrors-new-zealand-email}, title="mirror contact"] | ftp.nz.FreeBSD.org | link:http://ftp.nz.FreeBSD.org/pub/FreeBSD[http] link:ftp://ftp.nz.FreeBSD.org/pub/FreeBSD[ftp] | Norway icon:envelope[link=mailto:{mirrors-norway-email}, title="mirror contact"] | ftp.no.FreeBSD.org | link:ftp://ftp.no.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.no.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.no.FreeBSD.org[rsync] link:rsync://ftp.no.FreeBSD.org[rsync_v6] | Poland icon:envelope[link=mailto:{mirrors-poland-email}, title="mirror contact"] | ftp.pl.FreeBSD.org | link:http://ftp.pl.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.pl.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.pl.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp.pl.FreeBSD.org[rsync] link:rsync://ftp.pl.FreeBSD.org[rsync_v6] | Russia icon:envelope[link=mailto:{mirrors-russia-email}, title="mirror contact"] | ftp.ru.FreeBSD.org | link:http://ftp.ru.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.ru.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.ru.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.ru.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.ru.FreeBSD.org[rsync] link:rsync://ftp.ru.FreeBSD.org[rsync_v6] | | ftp2.ru.FreeBSD.org | link:https://ftp2.ru.FreeBSD.org/pub/FreeBSD[https] link:ftp://ftp2.ru.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp2.ru.FreeBSD.org[rsync] | Slovenia icon:envelope[link=mailto:{mirrors-slovenia-email}, title="mirror contact"] | ftp.si.FreeBSD.org | link:http://ftp.si.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.si.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.si.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.si.FreeBSD.org/pub/FreeBSD[ftp_v6] | South Africa icon:envelope[link=mailto:{mirrors-south-africa-email}, title="mirror contact"] | ftp.za.FreeBSD.org | link:https://ftp.za.FreeBSD.org/pub/FreeBSD[https] link:https://ftp.za.FreeBSD.org/pub/FreeBSD[https_v6] link:rsync://ftp.za.FreeBSD.org[rsync] link:rsync://ftp.za.FreeBSD.org[rsync_v6] | | ftp2.za.FreeBSD.org | link:http://ftp2.za.FreeBSD.org/pub/FreeBSD[http] link:http://ftp2.za.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp2.za.FreeBSD.org/pub/FreeBSD[ftp_v6] | | ftp4.za.FreeBSD.org | link:http://ftp4.za.FreeBSD.org/pub/FreeBSD[http] link:ftp://ftp4.za.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp4.za.FreeBSD.org[rsync] | Sweden icon:envelope[link=mailto:{mirrors-sweden-email}, title="mirror contact"] | ftp.se.FreeBSD.org | link:http://ftp.se.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.se.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.se.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.se.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.se.FreeBSD.org[rsync] link:rsync://ftp.se.FreeBSD.org[rsync_v6] | Taiwan icon:envelope[link=mailto:{mirrors-taiwan-email}, title="mirror contact"] | ftp4.tw.FreeBSD.org | link:https://ftp4.tw.FreeBSD.org/pub/FreeBSD[https] link:ftp://ftp4.tw.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp4.tw.FreeBSD.org[rsync] | | ftp5.tw.FreeBSD.org | link:http://ftp5.tw.FreeBSD.org/pub/FreeBSD[http] link:ftp://ftp5.tw.FreeBSD.org/pub/FreeBSD[ftp] | Ukraine icon:envelope[link=mailto:{mirrors-ukraine-email}, title="mirror contact"] | ftp.ua.FreeBSD.org | link:http://ftp.ua.FreeBSD.org/pub/FreeBSD[http] link:ftp://ftp.ua.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.ua.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.ua.FreeBSD.org[rsync] link:rsync://ftp.ua.FreeBSD.org[rsync_v6] | United Kingdom icon:envelope[link=mailto:{mirrors-uk-email}, title="mirror contact"] | ftp.uk.FreeBSD.org | link:http://ftp.uk.FreeBSD.org/pub/FreeBSD[http] link:http://ftp.uk.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp.uk.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp.uk.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp.uk.FreeBSD.org[rsync] link:rsync://ftp.uk.FreeBSD.org[rsync_v6] | | ftp2.uk.FreeBSD.org | link:http://ftp2.uk.FreeBSD.org/pub/FreeBSD[http] link:http://ftp2.uk.FreeBSD.org/pub/FreeBSD[http_v6] link:https://ftp2.uk.FreeBSD.org/pub/FreeBSD[https] link:https://ftp2.uk.FreeBSD.org/pub/FreeBSD[https_v6] link:ftp://ftp2.uk.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp2.uk.FreeBSD.org/pub/FreeBSD[ftp_v6] | United States of America icon:envelope[link=mailto:{mirrors-us-email}, title="mirror contact"] | ftp11.FreeBSD.org | link:http://ftp11.FreeBSD.org/pub/FreeBSD[http] link:http://ftp11.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp11.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp11.FreeBSD.org/pub/FreeBSD[ftp_v6] link:rsync://ftp11.FreeBSD.org[rsync] link:rsync://ftp11.FreeBSD.org[rsync_v6] | | ftp14.FreeBSD.org | link:ftp://ftp14.FreeBSD.org/pub/FreeBSD[ftp] link:rsync://ftp14.FreeBSD.org[rsync] (Former official tier 1) | | ftp5.FreeBSD.org | link:http://ftp5.FreeBSD.org/pub/FreeBSD[http] link:http://ftp5.FreeBSD.org/pub/FreeBSD[http_v6] link:ftp://ftp5.FreeBSD.org/pub/FreeBSD[ftp] link:ftp://ftp5.FreeBSD.org/pub/FreeBSD[ftp_v6] |=== The current list of protocols supported by the community mirrors was last updated on 2022-01-31, and it's not guaranteed. [[git]] == Using Git [[git-intro]] === Introduction As of December 2020, FreeBSD uses git as the primary version control system for storing all of FreeBSD's base source code and documentation. As of April 2021, FreeBSD uses git as the only version control system for storing all of FreeBSD's Ports Collection. [NOTE] ==== Git is generally a developer tool. -Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `portsnap` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection. +Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `git` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection. ==== This section demonstrates how to install Git on a FreeBSD system and use it to create a local copy of a FreeBSD source code repository. [[git-install]] === Installation Git can be installed from the Ports Collection, or as a package: [source,shell] .... # pkg install git .... [[git-usage]] === Running Git To fetch a clean copy of the sources into a local directory, use `git clone`. This directory of files is called the _working tree_. Git uses URLs to designate a repository. There are three different repositories, `src` for the FreeBSD system source code, `doc` for documentation, and `ports` for the FreeBSD Ports Collection. All three are reachable over two different protocols: HTTPS and SSH. For example, the URL `https://git.FreeBSD.org/src.git` specifies the main branch of the `src` repository, using the `https` protocol. [[git-url-table]] .FreeBSD Git Repository URL Table [options="header,footer"] |======================================================= |Item | Git URL | Read-only src repo via HTTPS | `https://git.FreeBSD.org/src.git` | Read-only src repo via anon-ssh | `ssh://anongit@git.FreeBSD.org/src.git` | Read-only doc repo via HTTPS | `https://git.FreeBSD.org/doc.git` | Read-only doc repo via anon-ssh | `ssh://anongit@git.FreeBSD.org/doc.git` | Read-only ports repo via HTTPS | `https://git.FreeBSD.org/ports.git` | Read-only ports repo via anon-ssh | `ssh://anongit@git.FreeBSD.org/ports.git` |======================================================= External mirrors maintained by project members are also available; please refer to the <> section. To clone a copy of the FreeBSD system source code repository: [source,shell] .... # git clone -o freebsd https://git.FreeBSD.org/src.git /usr/src .... The `-o freebsd` option specifies the origin; by convention in the FreeBSD documentation, the origin is assumed to be `freebsd`. Because the initial checkout must download the full branch of the remote repository, it can take a while. Please be patient. Initially, the working tree contains source code for the `main` branch, which corresponds to CURRENT. To switch to 13-STABLE instead: [source,shell] .... # cd /usr/src # git checkout stable/13 .... The working tree can be updated with `git pull`. To update [.filename]#/usr/src# created in the example above, use: [source,shell] .... # cd /usr/src # git pull --rebase .... The update is much quicker than a checkout, only transferring files that have changed. === Web-based repository browser The FreeBSD project uses cgit as the web-based repository browser: link:https://cgit.FreeBSD.org/[https://cgit.FreeBSD.org/]. === For Developers For information about write access to repositories see the extref:{committers-guide}[Committer's Guide, git-mini-primer]. [[external-mirrors]] === External mirrors Those mirrors are not hosted in FreeBSD.org but still maintained by the project members. Users and developers are welcome to pull or browse repositories on those mirrors. Pull requests for the `doc` GitHub repository are being accepted; otherwise, the project workflow with those mirrors is still under discussion. Codeberg:: - doc: https://codeberg.org/FreeBSD/freebsd-doc - ports: https://codeberg.org/FreeBSD/freebsd-ports - src: https://codeberg.org/FreeBSD/freebsd-src GitHub:: - doc: https://github.com/freebsd/freebsd-doc - ports: https://github.com/freebsd/freebsd-ports - src: https://github.com/freebsd/freebsd-src GitLab:: - doc: https://gitlab.com/FreeBSD/freebsd-doc - ports: https://gitlab.com/FreeBSD/freebsd-ports - src: https://gitlab.com/FreeBSD/freebsd-src === Mailing lists The main mailing list for general usage and questions about git in the FreeBSD project is https://lists.freebsd.org/subscription/freebsd-git[freebsd-git]. For more details, including commit messages lists, see the crossref:handbook/eresources[eresources-mail, Mailing Lists] chapter. === SSH host keys * gitrepo.FreeBSD.org host key fingerprints: ** ECDSA key fingerprint is `SHA256:seWO5D27ySURcx4bknTNKlC1mgai0whP443PAKEvvZA` ** ED25519 key fingerprint is `SHA256:lNR6i4BEOaaUhmDHBA1WJsO7H3KtvjE2r5q4sOxtIWo` ** RSA key fingerprint is `SHA256:f453CUEFXEJAXlKeEHV+ajJfeEfx9MdKQUD7lIscnQI` * git.FreeBSD.org host key fingerprints: ** ECDSA key fingerprint is `SHA256:/UlirUAsGiitupxmtsn7f9b7zCWd0vCs4Yo/tpVWP9w` ** ED25519 key fingerprint is `SHA256:y1ljKrKMD3lDObRUG3xJ9gXwEIuqnh306tSyFd1tuZE` ** RSA key fingerprint is `SHA256:jBe6FQGoH4HjvrIVM23dcnLZk9kmpdezR/CvQzm7rJM` These are also published as SSHFP records in DNS. [[svn]] == Using Subversion [[svn-intro]] === Introduction As of December 2020, FreeBSD uses git as the primary version control system for storing all of FreeBSD's source code and documentation. Changes from the git repo on the `stable/11`, `stable/12` and related releng branches are exported to the subversion repository. This export will continue through the life of these branches. From July 2012 to March 2021, FreeBSD used Subversion as the only version control system for storing all of FreeBSD's Ports Collection. As of April 2021, FreeBSD uses git as the only version control system for storing all of FreeBSD's Ports Collection. [NOTE] ==== Subversion is generally a developer tool. -Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `portsnap` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection. +Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `git` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection. After March 2021, subversion use is only for legacy branches (`stable/11` and `stable/12`). ==== This section demonstrates how to install Subversion on a FreeBSD system and use it to create a local copy of a FreeBSD repository. Additional information on the use of Subversion is included. [[svn-svnlite]] === Svnlite A lightweight version of Subversion is already installed on FreeBSD as `svnlite`. The port or package version of Subversion is only needed if the Python or Perl API is needed, or if a later version of Subversion is desired. The only difference from normal Subversion use is that the command name is `svnlite`. [[svn-install]] === Installation If `svnlite` is unavailable or the full version of Subversion is needed, then it must be installed. Subversion can be installed from the Ports Collection: [source,shell] .... # cd /usr/ports/devel/subversion # make install clean .... Subversion can also be installed as a package: [source,shell] .... # pkg install subversion .... [[svn-usage]] === Running Subversion To fetch a clean copy of the sources into a local directory, use `svn`. The files in this directory are called a _local working copy_. [WARNING] ==== Move or delete an existing destination directory before using `checkout` for the first time. Checkout over an existing non-`svn` directory can cause conflicts between the existing files and those brought in from the repository. ==== Subversion uses URLs to designate a repository, taking the form of _protocol://hostname/path_. The first component of the path is the FreeBSD repository to access. There are three different repositories, `base` for the FreeBSD base system source code, `ports` for the Ports Collection, and `doc` for documentation. For example, the URL `https://svn.FreeBSD.org/base/head/` specifies the main branch of the src repository, using the `https` protocol. A checkout from a given repository is performed with a command like this: [source,shell] .... # svn checkout https://svn.FreeBSD.org/repository/branch lwcdir .... where: * _repository_ is one of the Project repositories: `base`, `ports`, or `doc`. * _branch_ depends on the repository used. `ports` and `doc` are mostly updated in the `head` branch, while `base` maintains the latest version of -CURRENT under `head` and the respective latest versions of the -STABLE branches under `stable/11` (11._x_) and `stable/12` (12._x_). * _lwcdir_ is the target directory where the contents of the specified branch should be placed. This is usually [.filename]#/usr/ports# for `ports`, [.filename]#/usr/src# for `base`, and [.filename]#/usr/doc# for `doc`. This example checks out the Source Tree from the FreeBSD repository using the HTTPS protocol, placing the local working copy in [.filename]#/usr/src#. If [.filename]#/usr/src# is already present but was not created by `svn`, remember to rename or delete it before the checkout. [source,shell] .... # svn checkout https://svn.FreeBSD.org/base/head /usr/src .... Because the initial checkout must download the full branch of the remote repository, it can take a while. Please be patient. After the initial checkout, the local working copy can be updated by running: [source,shell] .... # svn update lwcdir .... To update [.filename]#/usr/src# created in the example above, use: [source,shell] .... # svn update /usr/src .... The update is much quicker than a checkout, only transferring files that have changed. An alternate way of updating the local working copy after checkout is provided by the [.filename]#Makefile# in the [.filename]#/usr/ports#, [.filename]#/usr/src#, and [.filename]#/usr/doc# directories. Set `SVN_UPDATE` and use the `update` target. For example, to update [.filename]#/usr/src#: [source,shell] .... # cd /usr/src # make update SVN_UPDATE=yes .... [[svn-mirrors]] === Subversion Mirror Sites The FreeBSD Subversion repository is: [.programlisting] .... svn.FreeBSD.org .... This is a publicly accessible mirror network that uses GeoDNS to select an appropriate back end server. To view the FreeBSD Subversion repositories through a browser, use https://svnweb.FreeBSD.org/[https://svnweb.FreeBSD.org/]. HTTPS is the preferred protocol, but the [.filename]#security/ca_root_nss# package will need to be installed in order to automatically validate certificates. === For More Information For other information about using Subversion, please see the "Subversion Book", titled http://svnbook.red-bean.com/[Version Control with Subversion], or the http://subversion.apache.org/docs/[Subversion Documentation]. [[mirrors-cdrom]] == CD and DVD Sets FreeBSD CD and DVD sets are available from several online retailers: * FreeBSD Mall, Inc. + 1164 Claremont Dr + Brentwood, CA + 94513 + USA + Phone: +1 925 240-6652 + Fax: +1 925 674-0821 + Email: info@freebsdmall.com + WWW: https://www.freebsdmall.com * Getlinux + WWW: https://www.getlinux.fr/ * Dr. Hinner EDV + Schäftlarnstr. 10 // 4. Stock + D-81371 München + Germany + Phone: +49 171 417 544 6 + Email: infow@hinner.de + WWW: http://www.hinner.de/linux/freebsd.html diff --git a/documentation/content/en/books/handbook/ports/_index.adoc b/documentation/content/en/books/handbook/ports/_index.adoc index 9ef5fc049d..b7c29721c5 100644 --- a/documentation/content/en/books/handbook/ports/_index.adoc +++ b/documentation/content/en/books/handbook/ports/_index.adoc @@ -1,1344 +1,1297 @@ --- title: "Chapter 4. Installing Applications: Packages and Ports" part: Part I. Getting Started prev: books/handbook/basics next: books/handbook/x11 description: "FreeBSD provides two complementary technologies for installing third-party software: the FreeBSD Ports Collection, for installing from source, and packages, for installing from pre-built binaries" tags: ["ports", "collection", "pkg", "poudriere", "management"] showBookMenu: true weight: 6 path: "/books/handbook/" aliases: ["/en/books/handbook/ports-overview/","/en/books/handbook/ports-finding-applications/","/en/books/handbook/pkgng-intro/","/en/books/handbook/ports-using/","/en/books/handbook/ports-poudriere/","/en/books/handbook/ports-nextsteps/","/en/books/handbook/ports-broken/"] --- [[ports]] = Installing Applications: Packages and Ports :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :sectnumoffset: 4 :partnums: :source-highlighter: rouge :experimental: :images-path: books/handbook/ports/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[ports-synopsis]] == Synopsis FreeBSD is bundled with a rich collection of system tools as part of the base system. In addition, FreeBSD provides two complementary technologies for installing third-party software: the FreeBSD Ports Collection, for installing from source, and packages, for installing from pre-built binaries. Either method may be used to install software from local media or from the network. After reading this chapter, you will know: * The difference between binary packages and ports. * How to find third-party software that has been ported to FreeBSD. * How to manage binary packages using pkg. * How to build third-party software from source using the Ports Collection. * How to find the files installed with the application for post-installation configuration. * What to do if a software installation fails. [[ports-overview]] == Overview of Software Installation The typical steps for installing third-party software on a UNIX(R) system include: [.procedure] . Find and download the software, which might be distributed in source code format or as a binary. . Unpack the software from its distribution format. This is typically a tarball compressed with a program such as man:compress[1], man:gzip[1], man:bzip2[1] or man:xz[1]. . Locate the documentation in [.filename]#INSTALL#, [.filename]#README# or some file in a [.filename]#doc/# subdirectory and read up on how to install the software. . If the software was distributed in source format, compile it. This may involve editing a [.filename]#Makefile# or running a `configure` script. . Test and install the software. A FreeBSD _port_ is a collection of files designed to automate the process of compiling an application from source code. The files that comprise a port contain all the necessary information to automatically download, extract, patch, compile, and install the application. If the software has not already been adapted and tested on FreeBSD, the source code might need editing in order for it to install and run properly. However, over link:https://www.FreeBSD.org/ports/[{numports}] third-party applications have already been ported to FreeBSD. When feasible, these applications are made available for download as pre-compiled _packages_. Packages can be manipulated with the FreeBSD package management commands. Both packages and ports understand dependencies. If a package or port is used to install an application and a dependent library is not already installed, the library will automatically be installed first. A FreeBSD package contains pre-compiled copies of all the commands for an application, as well as any configuration files and documentation. A package can be manipulated with the man:pkg[8] commands, such as `pkg install`. While the two technologies are similar, packages and ports each have their own strengths. Select the technology that meets your requirements for installing a particular application. .Package Benefits * A compressed package tarball is typically smaller than the compressed tarball containing the source code for the application. * Packages do not require compilation time. For large applications, such as Mozilla, KDE, or GNOME, this can be important on a slow system. * Packages do not require any understanding of the process involved in compiling software on FreeBSD. .Port Benefits * Packages are normally compiled with conservative options because they have to run on the maximum number of systems. By compiling from the port, one can change the compilation options. * Some applications have compile-time options relating to which features are installed. For example, Apache can be configured with a wide variety of different built-in options. + In some cases, multiple packages will exist for the same application to specify certain settings. For example, Ghostscript is available as a [.filename]#ghostscript# package and a [.filename]#ghostscript-nox11# package, depending on whether or not Xorg is installed. Creating multiple packages rapidly becomes impossible if an application has more than one or two different compile-time options. * The licensing conditions of some software forbid binary distribution. Such software must be distributed as source code which must be compiled by the end-user. * Some people do not trust binary distributions or prefer to read through source code in order to look for potential problems. * Source code is needed in order to apply custom patches. To keep track of updated ports, subscribe to the {freebsd-ports} and the {freebsd-ports-bugs}. [WARNING] ==== Before installing any application, check https://vuxml.freebsd.org/[] for security issues related to the application or type `pkg audit -F` to check all installed applications for known vulnerabilities. ==== The remainder of this chapter explains how to use packages and ports to install and manage third-party software on FreeBSD. [[ports-finding-applications]] == Finding Software FreeBSD's list of available applications is growing all the time. There are a number of ways to find software to install: * The FreeBSD web site maintains an up-to-date searchable list of all the available applications, at link:https://www.FreeBSD.org/ports/[https://www.FreeBSD.org/ports/]. The ports can be searched by application name or by software category. * Dan Langille maintains http://www.FreshPorts.org/[FreshPorts.org] which provides a comprehensive search utility and also tracks changes to the applications in the Ports Collection. Registered users can create a customized watch list in order to receive an automated email when their watched ports are updated. * If finding a particular application becomes challenging, try searching a site like http://www.sourceforge.net/[SourceForge.net] or http://www.github.com/[GitHub.com] then check back at the link:https://www.FreeBSD.org/ports/[FreeBSD site] to see if the application has been ported. * To search the binary package repository for an application: + [source,shell] .... # pkg search subversion git-subversion-1.9.2 java-subversion-1.8.8_2 p5-subversion-1.8.8_2 py27-hgsubversion-1.6 py27-subversion-1.8.8_2 ruby-subversion-1.8.8_2 subversion-1.8.8_2 subversion-book-4515 subversion-static-1.8.8_2 subversion16-1.6.23_4 subversion17-1.7.16_2 .... + Package names include the version number and, in the case of ports based on python, the version number of the version of python the package was built with. Some ports also have multiple versions available. In the case of Subversion, there are different versions available, as well as different compile options. In this case, the statically linked version of Subversion. When indicating which package to install, it is best to specify the application by the port origin, which is the path in the ports tree. Repeat the `pkg search` with `-o` to list the origin of each package: + [source,shell] .... # pkg search -o subversion devel/git-subversion java/java-subversion devel/p5-subversion devel/py-hgsubversion devel/py-subversion devel/ruby-subversion devel/subversion16 devel/subversion17 devel/subversion devel/subversion-book devel/subversion-static .... + Searching by shell globs, regular expressions, exact match, by description, or any other field in the repository database is also supported by `pkg search`. After installing package:ports-mgmt/pkg[] or package:ports-mgmt/pkg-devel[], see man:pkg-search[8] for more details. * If the Ports Collection is already installed, there are several methods to query the local version of the ports tree. To find out which category a port is in, type `whereis _file_`, where _file_ is the program to be installed: + [source,shell] .... # whereis lsof lsof: /usr/ports/sysutils/lsof .... + Alternately, an man:echo[1] statement can be used: + [source,shell] .... # echo /usr/ports/*/*lsof* /usr/ports/sysutils/lsof .... + Note that this will also return any matched files downloaded into the [.filename]#/usr/ports/distfiles# directory. * Another way to find software is by using the Ports Collection's built-in search mechanism. To use the search feature, cd to [.filename]#/usr/ports# then run `make search name=program-name` where _program-name_ is the name of the software. For example, to search for `lsof`: + [source,shell] .... # cd /usr/ports # make search name=lsof Port: lsof-4.88.d,8 Path: /usr/ports/sysutils/lsof Info: Lists information about open files (similar to fstat(1)) Maint: ler@lerctr.org Index: sysutils B-deps: R-deps: .... + [TIP] ==== The built-in search mechanism uses a file of index information. If a message indicates that the [.filename]#INDEX# is required, run `make fetchindex` to download the current index file. With the [.filename]#INDEX# present, `make search` will be able to perform the requested search. ==== + The "Path:" line indicates where to find the port. + To receive less information, use the `quicksearch` feature: + [source,shell] .... # cd /usr/ports # make quicksearch name=lsof Port: lsof-4.88.d,8 Path: /usr/ports/sysutils/lsof Info: Lists information about open files (similar to fstat(1)) .... + For more in-depth searching, use `make search key=_string_` or `make quicksearch key=_string_`, where _string_ is some text to search for. The text can be in comments, descriptions, or dependencies in order to find ports which relate to a particular subject when the name of the program is unknown. + When using `search` or `quicksearch`, the search string is case-insensitive. Searching for "LSOF" will yield the same results as searching for "lsof". [[pkgng-intro]] == Using pkg for Binary Package Management pkg is the next generation replacement for the traditional FreeBSD package management tools, offering many features that make dealing with binary packages faster and easier. For sites wishing to only use prebuilt binary packages from the FreeBSD mirrors, managing packages with pkg can be sufficient. However, for those sites building from source or using their own repositories, a separate <> will be needed. Since pkg only works with binary packages, it is not a replacement for such tools. Those tools can be used to install software from both binary packages and the Ports Collection, while pkg installs only binary packages. [[pkgng-initial-setup]] === Getting Started with pkg FreeBSD includes a bootstrap utility which can be used to download and install pkg and its manual pages. This utility is designed to work with versions of FreeBSD starting with 10._X_. [NOTE] ==== Not all FreeBSD versions and architectures support this bootstrap process. The current list is at https://pkg.freebsd.org/[]. For other cases, pkg must instead be installed from the Ports Collection or as a binary package. ==== To bootstrap the system, run: [source,shell] .... # /usr/sbin/pkg .... You must have a working Internet connection for the bootstrap process to succeed. Otherwise, to install the port, run: [source,shell] .... # cd /usr/ports/ports-mgmt/pkg # make # make install clean .... When upgrading an existing system that originally used the older pkg_* tools, the database must be converted to the new format, so that the new tools are aware of the already installed packages. Once pkg has been installed, the package database must be converted from the traditional format to the new format by running this command: [source,shell] .... # pkg2ng .... [NOTE] ==== This step is not required for new installations that do not yet have any third-party software installed. ==== [IMPORTANT] ==== This step is not reversible. Once the package database has been converted to the pkg format, the traditional `pkg_*` tools should no longer be used. ==== [NOTE] ==== The package database conversion may emit errors as the contents are converted to the new version. Generally, these errors can be safely ignored. However, a list of software that was not successfully converted is shown after `pkg2ng` finishes. These applications must be manually reinstalled. ==== To ensure that the Ports Collection registers new software with pkg instead of the traditional packages database, FreeBSD versions earlier than 10._X_ require this line in [.filename]#/etc/make.conf#: [.programlisting] .... WITH_PKGNG= yes .... By default, pkg uses the binary packages from the FreeBSD package mirrors (the _repository_). For information about building a custom package repository, see <>. Additional pkg configuration options are described in man:pkg.conf[5]. Usage information for pkg is available in the man:pkg[8] manual page or by running `pkg` without additional arguments. Each pkg command argument is documented in a command-specific manual page. To read the manual page for `pkg install`, for example, run either of these commands: [source,shell] .... # pkg help install .... [source,shell] .... # man pkg-install .... The rest of this section demonstrates common binary package management tasks which can be performed using pkg. Each demonstrated command provides many switches to customize its use. Refer to a command's help or man page for details and more examples. [[quarterly-latest-branch]] === Quarterly and Latest Ports Branches The `Quarterly` branch provides users with a more predictable and stable experience for port and package installation and upgrades. This is done essentially by only allowing non-feature updates. Quarterly branches aim to receive security fixes (that may be version updates, or backports of commits), bug fixes and ports compliance or framework changes. The Quarterly branch is cut from HEAD at the beginning of every (yearly) quarter in January, April, July, and October. Branches are named according to the year (YYYY) and quarter (Q1-4) they are created in. For example, the quarterly branch created in January 2016, is named 2016Q1. And the `Latest` branch provides the latest versions of the packages to the users. To switch from quarterly to latest run the following commands: [source,shell] .... # mkdir -p /usr/local/etc/pkg/repos # cp /etc/pkg/FreeBSD.conf /usr/local/etc/pkg/repos/FreeBSD.conf .... Edit the file [.filename]#/usr/local/etc/pkg/repos/FreeBSD.conf# and change the string _quarterly_ to _latest_ in the `url:` line. The result should be similar to the following: [.programlisting] .... FreeBSD: { url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest", mirror_type: "srv", signature_type: "fingerprints", fingerprints: "/usr/share/keys/pkg", enabled: yes } .... And finally run this command to update from the new (latest) repository metadata. [source,shell] .... # pkg update -f .... [[pkgng-pkg-info]] === Obtaining Information About Installed Packages Information about the packages installed on a system can be viewed by running `pkg info` which, when run without any switches, will list the package version for either all installed packages or the specified package. For example, to see which version of pkg is installed, run: [source,shell] .... # pkg info pkg pkg-1.1.4_1 .... [[pkgng-installing-deinstalling]] === Installing and Removing Packages To install a binary package use the following command, where _packagename_ is the name of the package to install: [source,shell] .... # pkg install packagename .... This command uses repository data to determine which version of the software to install and if it has any uninstalled dependencies. For example, to install curl: [source,shell] .... # pkg install curl Updating repository catalogue /usr/local/tmp/All/curl-7.31.0_1.txz 100% of 1181 kB 1380 kBps 00m01s /usr/local/tmp/All/ca_root_nss-3.15.1_1.txz 100% of 288 kB 1700 kBps 00m00s Updating repository catalogue The following 2 packages will be installed: Installing ca_root_nss: 3.15.1_1 Installing curl: 7.31.0_1 The installation will require 3 MB more space 0 B to be downloaded Proceed with installing packages [y/N]: y Checking integrity... done [1/2] Installing ca_root_nss-3.15.1_1... done [2/2] Installing curl-7.31.0_1... done Cleaning up cache files...Done .... The new package and any additional packages that were installed as dependencies can be seen in the installed packages list: [source,shell] .... # pkg info ca_root_nss-3.15.1_1 The root certificate bundle from the Mozilla Project curl-7.31.0_1 Non-interactive tool to get files from FTP, GOPHER, HTTP(S) servers pkg-1.1.4_6 New generation package manager .... Packages that are no longer needed can be removed with `pkg delete`. For example: [source,shell] .... # pkg delete curl The following packages will be deleted: curl-7.31.0_1 The deletion will free 3 MB Proceed with deleting packages [y/N]: y [1/1] Deleting curl-7.31.0_1... done .... [[pkgng-upgrading]] === Upgrading Installed Packages Installed packages can be upgraded to their latest versions by running: [source,shell] .... # pkg upgrade .... This command will compare the installed versions with those available in the repository catalogue and upgrade them from the repository. [[pkgng-auditing]] === Auditing Installed Packages Software vulnerabilities are regularly discovered in third-party applications. To address this, pkg includes a built-in auditing mechanism. To determine if there are any known vulnerabilities for the software installed on the system, run: [source,shell] .... # pkg audit -F .... [[pkgng-autoremove]] === Automatically Removing Unused Packages Removing a package may leave behind dependencies which are no longer required. Unneeded packages that were installed as dependencies (leaf packages) can be automatically detected and removed using: [source,shell] .... # pkg autoremove Packages to be autoremoved: ca_root_nss-3.15.1_1 The autoremoval will free 723 kB Proceed with autoremoval of packages [y/N]: y Deinstalling ca_root_nss-3.15.1_1... done .... Packages installed as dependencies are called _automatic_ packages. Non-automatic packages, i.e the packages that were explicity installed not as a dependency to another package, can be listed using: [source,shell] .... # pkg prime-list nginx openvpn sudo .... `pkg prime-list` is an alias command declared in [.filename]#/usr/local/etc/pkg.conf#. There are many others that can be used to query the package database of the system. For instance, command `pkg prime-origins` can be used to get the origin port directory of the list mentioned above: [source,shell] .... # pkg prime-origins www/nginx security/openvpn security/sudo .... This list can be used to rebuild all packages installed on a system using build tools such as package:ports-mgmt/poudriere[] or package:ports-mgmt/synth[]. Marking an installed package as automatic can be done using: [source,shell] .... # pkg set -A 1 devel/cmake .... Once a package is a leaf package and is marked as automatic, it gets selected by `pkg autoremove`. Marking an installed package as _not_ automatic can be done using: [source,shell] .... # pkg set -A 0 devel/cmake .... [[pkgng-backup]] === Restoring the Package Database Unlike the traditional package management system, pkg includes its own package database backup mechanism. This functionality is enabled by default. [TIP] ==== To disable the periodic script from backing up the package database, set `daily_backup_pkgdb_enable="NO"` in man:periodic.conf[5]. ==== To restore the contents of a previous package database backup, run the following command replacing _/path/to/pkg.sql_ with the location of the backup: [source,shell] .... # pkg backup -r /path/to/pkg.sql .... [NOTE] ==== If restoring a backup taken by the periodic script, it must be decompressed prior to being restored. ==== To run a manual backup of the pkg database, run the following command, replacing _/path/to/pkg.sql_ with a suitable file name and location: [source,shell] .... # pkg backup -d /path/to/pkg.sql .... [[pkgng-clean]] === Removing Stale Packages By default, pkg stores binary packages in a cache directory defined by `PKG_CACHEDIR` in man:pkg.conf[5]. Only copies of the latest installed packages are kept. Older versions of pkg kept all previous packages. To remove these outdated binary packages, run: [source,shell] .... # pkg clean .... The entire cache may be cleared by running: [source,shell] .... # pkg clean -a .... [[pkgng-set]] === Modifying Package Metadata Software within the FreeBSD Ports Collection can undergo major version number changes. To address this, pkg has a built-in command to update package origins. This can be useful, for example, if package:lang/php5[] is renamed to package:lang/php53[] so that package:lang/php5[] can now represent version `5.4`. To change the package origin for the above example, run: [source,shell] .... # pkg set -o lang/php5:lang/php53 .... As another example, to update package:lang/ruby18[] to package:lang/ruby19[], run: [source,shell] .... # pkg set -o lang/ruby18:lang/ruby19 .... As a final example, to change the origin of the [.filename]#libglut# shared libraries from package:graphics/libglut[] to package:graphics/freeglut[], run: [source,shell] .... # pkg set -o graphics/libglut:graphics/freeglut .... [NOTE] ==== When changing package origins, it is important to reinstall packages that are dependent on the package with the modified origin. To force a reinstallation of dependent packages, run: [source,shell] .... # pkg install -Rf graphics/freeglut .... ==== [[ports-using]] == Using the Ports Collection The Ports Collection is a set of [.filename]##Makefile##s, patches, and description files. Each set of these files is used to compile and install an individual application on FreeBSD, and is called a _port_. By default, the Ports Collection itself is stored as a subdirectory of [.filename]#/usr/ports#. [WARNING] ==== Before installing and using the Ports Collection, please be aware that it is generally ill-advised to use the Ports Collection in conjunction with the binary packages provided via pkg to install software. pkg, by default, tracks quarterly branch-releases of the ports tree and not HEAD. Dependencies could be different for a port in HEAD compared to its counterpart in a quarterly branch release and this could result in conflicts between dependencies installed by pkg and those from the Ports Collection. If the Ports Collection and pkg must be used in conjunction, then be sure that your Ports Collection and pkg are on the same branch release of the ports tree. ==== The Ports Collection contains directories for software categories. Inside each category are subdirectories for individual applications. Each application subdirectory contains a set of files that tells FreeBSD how to compile and install that program, called a _ports skeleton_. Each port skeleton includes these files and directories: * [.filename]#Makefile#: contains statements that specify how the application should be compiled and where its components should be installed. * [.filename]#distinfo#: contains the names and checksums of the files that must be downloaded to build the port. * [.filename]#files/#: this directory contains any patches needed for the program to compile and install on FreeBSD. This directory may also contain other files used to build the port. * [.filename]#pkg-descr#: provides a more detailed description of the program. * [.filename]#pkg-plist#: a list of all the files that will be installed by the port. It also tells the ports system which files to remove upon deinstallation. Some ports include [.filename]#pkg-message# or other files to handle special situations. For more details on these files, and on ports in general, refer to the extref:{porters-handbook}[FreeBSD Porter's Handbook]. The port does not include the actual source code, also known as a [.filename]#distfile#. The extract portion of building a port will automatically save the downloaded source to [.filename]#/usr/ports/distfiles#. [[ports-using-installation-methods]] === Installing the Ports Collection Before an application can be compiled using a port, the Ports Collection must first be installed. If it was not installed during the installation of FreeBSD, use one of the following methods to install it: -[[ports-using-portsnap-method]] -[.procedure] -**** -*Procedure: Portsnap Method* - -The base system of FreeBSD includes Portsnap. -This is a fast and user-friendly tool for retrieving the Ports Collection and is the recommended choice for most users not running FreeBSD-CURRENT. -This utility connects to a FreeBSD site, verifies the secure key, and downloads a new copy of the Ports Collection. -The key is used to verify the integrity of all downloaded files. - -Note that Portsnap updates are generated from a snapshot of the main -branch of the Ports Collection and cannot be configured to use a -different branch (for example, `quarterly`). -If it is necessary to use a different branch of the Ports Collection -(for instance as referenced earlier in conjunction with binary -packages), then the <> must be used. - -. To download a compressed snapshot of the Ports Collection into [.filename]#/var/db/portsnap#: -+ -[source,shell] -.... -# portsnap fetch -.... -+ -. When running Portsnap for the first time, extract the snapshot into [.filename]#/usr/ports#: -+ -[source,shell] -.... -# portsnap extract -.... -+ -. After the first use of Portsnap has been completed as shown above, [.filename]#/usr/ports# can be updated as needed by running: -+ -[source,shell] -.... -# portsnap fetch -# portsnap update -.... -+ -When using `fetch`, the `extract` or the `update` operation may be run consecutively, like so: -+ -[source,shell] -.... -# portsnap fetch update -.... -**** - [[ports-using-git-method]] [.procedure] **** *Procedure: Git Method* If more control over the ports tree is needed or if local changes need to be maintained, or if running FreeBSD-CURRENT, Git can be used to obtain the Ports Collection. Refer to extref:{committers-guide}[the Git Primer, git-primer] for a detailed description of Git. . Git must be installed before it can be used to check out the ports tree. If a copy of the ports tree is already present, install Git like this: + [source,shell] .... # cd /usr/ports/devel/git # make install clean .... + If the ports tree is not available, or pkg is being used to manage packages, Git can be installed as a package: + [source,shell] .... # pkg install git .... + . Check out a copy of the HEAD branch of the ports tree: + [source,shell] .... # git clone https://git.FreeBSD.org/ports.git /usr/ports .... + . Or, check out a copy of a quarterly branch: + [source,shell] .... # git clone https://git.FreeBSD.org/ports.git -b 2020Q3 /usr/ports .... + . As needed, update [.filename]#/usr/ports# after the initial Git checkout: + [source,shell] .... # git -C /usr/ports pull .... + . As needed, switch [.filename]#/usr/ports# to a different quarterly branch: + [source,shell] .... # git -C /usr/ports switch 2020Q4 .... **** === Installing Ports This section provides basic instructions on using the Ports Collection to install or remove software. The detailed description of available `make` targets and environment variables is available in man:ports[7]. [WARNING] ==== Before compiling any port, be sure to update the Ports Collection as described in the previous section. Since the installation of any third-party software can introduce security vulnerabilities, it is recommended to first check https://vuxml.freebsd.org/[] for known security issues related to the port. Alternately, run `pkg audit -F` before installing a new port. This command can be configured to automatically perform a security audit and an update of the vulnerability database during the daily security system check. For more information, refer to man:pkg-audit[8] and man:periodic[8]. ==== Using the Ports Collection assumes a working Internet connection. It also requires superuser privilege. To compile and install the port, change to the directory of the port to be installed, then type `make install` at the prompt. Messages will indicate the progress: [source,shell] .... # cd /usr/ports/sysutils/lsof # make install >> lsof_4.88D.freebsd.tar.gz doesn't seem to exist in /usr/ports/distfiles/. >> Attempting to fetch from ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/. ===> Extracting for lsof-4.88 ... [extraction output snipped] ... >> Checksum OK for lsof_4.88D.freebsd.tar.gz. ===> Patching for lsof-4.88.d,8 ===> Applying FreeBSD patches for lsof-4.88.d,8 ===> Configuring for lsof-4.88.d,8 ... [configure output snipped] ... ===> Building for lsof-4.88.d,8 ... [compilation output snipped] ... ===> Installing for lsof-4.88.d,8 ... [installation output snipped] ... ===> Generating temporary packing list ===> Compressing manual pages for lsof-4.88.d,8 ===> Registering installation for lsof-4.88.d,8 ===> SECURITY NOTE: This port has installed the following binaries which execute with increased privileges. /usr/local/sbin/lsof # .... Since `lsof` is a program that runs with increased privileges, a security warning is displayed as it is installed. Once the installation is complete, the prompt will be returned. Some shells keep a cache of the commands that are available in the directories listed in the `PATH` environment variable, to speed up lookup operations for the executable file of these commands. Users of the `tcsh` shell should type `rehash` so that a newly installed command can be used without specifying its full path. Use `hash -r` instead for the `sh` shell. Refer to the documentation for the shell for more information. During installation, a working subdirectory is created which contains all the temporary files used during compilation. Removing this directory saves disk space and minimizes the chance of problems later when upgrading to the newer version of the port: [source,shell] .... # make clean ===> Cleaning for lsof-88.d,8 # .... [NOTE] ==== To save this extra step, instead use `make install clean` when compiling the port. ==== ==== Customizing Ports Installation Some ports provide build options which can be used to enable or disable application components, provide security options, or allow for other customizations. Examples include package:www/firefox[], package:security/gpgme[], and package:mail/sylpheed-claws[]. If the port depends upon other ports which have configurable options, it may pause several times for user interaction as the default behavior is to prompt the user to select options from a menu. To avoid this and do all of the configuration in one batch, run `make config-recursive` within the port skeleton. Then, run `make install [clean]` to compile and install the port. [TIP] ==== When using `config-recursive`, the list of ports to configure are gathered by the `all-depends-list` target. It is recommended to run `make config-recursive` until all dependent ports options have been defined, and ports options screens no longer appear, to be certain that all dependency options have been configured. ==== There are several ways to revisit a port's build options menu in order to add, remove, or change these options after a port has been built. One method is to `cd` into the directory containing the port and type `make config`. Another option is to use `make showconfig`. Another option is to execute `make rmconfig` which will remove all selected options and allow you to start over. All of these options, and others, are explained in great detail in man:ports[7]. The ports system uses man:fetch[1] to download the source files, which supports various environment variables. The `FTP_PASSIVE_MODE`, `FTP_PROXY`, and `FTP_PASSWORD` variables may need to be set if the FreeBSD system is behind a firewall or FTP/HTTP proxy. See man:fetch[3] for the complete list of supported variables. For users who cannot be connected to the Internet all the time, `make fetch` can be run within [.filename]#/usr/ports#, to fetch all distfiles, or within a category, such as [.filename]#/usr/ports/net#, or within the specific port skeleton. Note that if a port has any dependencies, running this command in a category or ports skeleton will _not_ fetch the distfiles of ports from another category. Instead, use `make fetch-recursive` to also fetch the distfiles for all the dependencies of a port. In rare cases, such as when an organization has a local distfiles repository, the `MASTER_SITES` variable can be used to override the download locations specified in the [.filename]#Makefile#. When using, specify the alternate location: [source,shell] .... # cd /usr/ports/directory # make MASTER_SITE_OVERRIDE= \ ftp://ftp.organization.org/pub/FreeBSD/ports/distfiles/ fetch .... The `WRKDIRPREFIX` and `PREFIX` variables can override the default working and target directories. For example: [source,shell] .... # make WRKDIRPREFIX=/usr/home/example/ports install .... will compile the port in [.filename]#/usr/home/example/ports# and install everything under [.filename]#/usr/local#. [source,shell] .... # make PREFIX=/usr/home/example/local install .... will compile the port in [.filename]#/usr/ports# and install it in [.filename]#/usr/home/example/local#. And: [source,shell] .... # make WRKDIRPREFIX=../ports PREFIX=../local install .... will combine the two. These can also be set as environmental variables. Refer to the manual page for your shell for instructions on how to set an environmental variable. [[ports-removing]] === Removing Installed Ports Installed ports can be uninstalled using `pkg delete`. Examples for using this command can be found in the man:pkg-delete[8] manual page. Alternately, `make deinstall` can be run in the port's directory: [source,shell] .... # cd /usr/ports/sysutils/lsof # make deinstall ===> Deinstalling for sysutils/lsof ===> Deinstalling Deinstallation has been requested for the following 1 packages: lsof-4.88.d,8 The deinstallation will free 229 kB [1/1] Deleting lsof-4.88.d,8... done .... It is recommended to read the messages as the port is uninstalled. If the port has any applications that depend upon it, this information will be displayed but the uninstallation will proceed. In such cases, it may be better to reinstall the application in order to prevent broken dependencies. [[ports-upgrading]] === Upgrading Ports Over time, newer versions of software become available in the Ports Collection. This section describes how to determine which software can be upgraded and how to perform the upgrade. To determine if newer versions of installed ports are available, ensure that the latest version of the ports tree is installed, -using the updating command described in either <> or <>. +using the updating command described in <>. On FreeBSD 10 and later, or if the system has been converted to pkg, the following command will list the installed ports which are out of date: [source,shell] .... # pkg version -l "<" .... For FreeBSD 9._X_ and lower, the following command will list the installed ports that are out of date: [source,shell] .... # pkg_version -l "<" .... [IMPORTANT] ==== Before attempting an upgrade, read [.filename]#/usr/ports/UPDATING# from the top of the file to the date closest to the last time ports were upgraded or the system was installed. This file describes various issues and additional steps users may encounter and need to perform when updating a port, including such things as file format changes, changes in locations of configuration files, or any incompatibilities with previous versions. Make note of any instructions which match any of the ports that need upgrading and follow these instructions when performing the upgrade. ==== [[ports-upgrading-tools]] ==== Tools to Upgrade and Manage Ports The Ports Collection contains several utilities to perform the actual upgrade. Each has its strengths and weaknesses. Historically, most installations used either Portmaster or Portupgrade. Synth is a newer alternative. [NOTE] ==== The choice of which tool is best for a particular system is up to the system administrator. It is recommended practice to back up your data before using any of these tools. ==== [[portmaster]] ==== Upgrading Ports Using Portmaster package:ports-mgmt/portmaster[] is a very small utility for upgrading installed ports. It is designed to use the tools installed with the FreeBSD base system without depending on other ports or databases. To install this utility as a port: [source,shell] .... # cd /usr/ports/ports-mgmt/portmaster # make install clean .... Portmaster defines four categories of ports: * Root port: has no dependencies and is not a dependency of any other ports. * Trunk port: has no dependencies, but other ports depend upon it. * Branch port: has dependencies and other ports depend upon it. * Leaf port: has dependencies but no other ports depend upon it. To list these categories and search for updates: [source,shell] .... # portmaster -L ===>>> Root ports (No dependencies, not depended on) ===>>> ispell-3.2.06_18 ===>>> screen-4.0.3 ===>>> New version available: screen-4.0.3_1 ===>>> tcpflow-0.21_1 ===>>> 7 root ports ... ===>>> Branch ports (Have dependencies, are depended on) ===>>> apache22-2.2.3 ===>>> New version available: apache22-2.2.8 ... ===>>> Leaf ports (Have dependencies, not depended on) ===>>> automake-1.9.6_2 ===>>> bash-3.1.17 ===>>> New version available: bash-3.2.33 ... ===>>> 32 leaf ports ===>>> 137 total installed ports ===>>> 83 have new versions available .... This command is used to upgrade all outdated ports: [source,shell] .... # portmaster -a .... [NOTE] ==== By default, Portmaster makes a backup package before deleting the existing port. If the installation of the new version is successful, Portmaster deletes the backup. Using `-b` instructs Portmaster not to automatically delete the backup. Adding `-i` starts Portmaster in interactive mode, prompting for confirmation before upgrading each port. Many other options are available. Read through the manual page for man:portmaster[8] for details regarding their usage. ==== If errors are encountered during the upgrade process, add `-f` to upgrade and rebuild all ports: [source,shell] .... # portmaster -af .... Portmaster can also be used to install new ports on the system, upgrading all dependencies before building and installing the new port. To use this function, specify the location of the port in the Ports Collection: [source,shell] .... # portmaster shells/bash .... More information about package:ports-mgmt/portmaster[] may be found in its [.filename]#pkg-descr#. [[portupgrade]] ==== Upgrading Ports Using Portupgrade package:ports-mgmt/portupgrade[] is another utility that can be used to upgrade ports. It installs a suite of applications which can be used to manage ports. However, it is dependent upon Ruby. To install the port: [source,shell] .... # cd /usr/ports/ports-mgmt/portupgrade # make install clean .... Before performing an upgrade using this utility, it is recommended to scan the list of installed ports using `pkgdb -F` and to fix all the inconsistencies it reports. To upgrade all the outdated ports installed on the system, use `portupgrade -a`. Alternately, include `-i` to be asked for confirmation of every individual upgrade: [source,shell] .... # portupgrade -ai .... To upgrade only a specified application instead of all available ports, use `portupgrade _pkgname_`. It is very important to include `-R` to first upgrade all the ports required by the given application: [source,shell] .... # portupgrade -R firefox .... If `-P` is included, Portupgrade searches for available packages in the local directories listed in `PKG_PATH`. If none are available locally, it then fetches packages from a remote site. If packages can not be found locally or fetched remotely, Portupgrade will use ports. To avoid using ports entirely, specify `-PP`. This last set of options tells Portupgrade to abort if no packages are available: [source,shell] .... # portupgrade -PP gnome3 .... To just fetch the port distfiles, or packages, if `-P` is specified, without building or installing anything, use `-F`. For further information on all of the available switches, refer to the manual page for `portupgrade`. More information about package:ports-mgmt/portupgrade[] may be found in its [.filename]#pkg-descr#. [[ports-disk-space]] === Ports and Disk Space Using the Ports Collection will use up disk space over time. After building and installing a port, running `make clean` within the ports skeleton will clean up the temporary [.filename]#work# directory. If Portmaster is used to install a port, it will automatically remove this directory unless `-K` is specified. If Portupgrade is installed, this command will remove all [.filename]#work# directories found within the local copy of the Ports Collection: [source,shell] .... # portsclean -C .... In addition, outdated source distribution files accumulate in [.filename]#/usr/ports/distfiles# over time. To use Portupgrade to delete all the distfiles that are no longer referenced by any ports: [source,shell] .... # portsclean -D .... Portupgrade can remove all distfiles not referenced by any port currently installed on the system: [source,shell] .... # portsclean -DD .... If Portmaster is installed, use: [source,shell] .... # portmaster --clean-distfiles .... By default, this command is interactive and prompts the user to confirm if a distfile should be deleted. In addition to these commands, package:ports-mgmt/pkg_cutleaves[] automates the task of removing installed ports that are no longer needed. [[ports-poudriere]] == Building Packages with Poudriere Poudriere is a `BSD`-licensed utility for creating and testing FreeBSD packages. It uses FreeBSD jails to set up isolated compilation environments. These jails can be used to build packages for versions of FreeBSD that are different from the system on which it is installed, and also to build packages for i386 if the host is an amd64 system. Once the packages are built, they are in a layout identical to the official mirrors. These packages are usable by man:pkg[8] and other package management tools. Poudriere is installed using the package:ports-mgmt/poudriere[] package or port. The installation includes a sample configuration file [.filename]#/usr/local/etc/poudriere.conf.sample#. Copy this file to [.filename]#/usr/local/etc/poudriere.conf#. Edit the copied file to suit the local configuration. While `ZFS` is not required on the system running poudriere, it is beneficial. When `ZFS` is used, `ZPOOL` must be specified in [.filename]#/usr/local/etc/poudriere.conf# and `FREEBSD_HOST` should be set to a nearby mirror. Defining `CCACHE_DIR` enables the use of package:devel/ccache[] to cache compilation and reduce build times for frequently-compiled code. It may be convenient to put poudriere datasets in an isolated tree mounted at [.filename]#/poudriere#. Defaults for the other configuration values are adequate. The number of processor cores detected is used to define how many builds will run in parallel. Supply enough virtual memory, either with `RAM` or swap space. If virtual memory runs out, the compilation jails will stop and be torn down, resulting in weird error messages. [[poudriere-initialization]] === Initialize Jails and Port Trees After configuration, initialize poudriere so that it installs a jail with the required FreeBSD tree and a ports tree. Specify a name for the jail using `-j` and the FreeBSD version with `-v`. On systems running FreeBSD/amd64, the architecture can be set with `-a` to either `i386` or `amd64`. The default is the architecture shown by `uname`. [source,shell] .... # poudriere jail -c -j 13amd64 -v 13.1-RELEASE [00:00:00] Creating 13amd64 fs at /poudriere/jails/13amd64... done [00:00:00] Using pre-distributed MANIFEST for FreeBSD 13.1-RELEASE amd64 [00:00:00] Fetching base for FreeBSD 13.1-RELEASE amd64 /poudriere/jails/13amd64/fromftp/base.txz 125 MB 4110 kBps 31s [00:00:33] Extracting base... done [00:00:54] Fetching src for FreeBSD 13.1-RELEASE amd64 /poudriere/jails/13amd64/fromftp/src.txz 154 MB 4178 kBps 38s [00:01:33] Extracting src... done [00:02:31] Fetching lib32 for FreeBSD 13.1-RELEASE amd64 /poudriere/jails/13amd64/fromftp/lib32.txz 24 MB 3969 kBps 06s [00:02:38] Extracting lib32... done [00:02:42] Cleaning up... done [00:02:42] Recording filesystem state for clean... done [00:02:42] Upgrading using ftp /etc/resolv.conf -> /poudriere/jails/13amd64/etc/resolv.conf Looking up update.FreeBSD.org mirrors... 3 mirrors found. Fetching public key from update4.freebsd.org... done. Fetching metadata signature for 13.1-RELEASE from update4.freebsd.org... done. Fetching metadata index... done. Fetching 2 metadata files... done. Inspecting system... done. Preparing to download files... done. Fetching 124 patches.....10....20....30....40....50....60....70....80....90....100....110....120.. done. Applying patches... done. Fetching 6 files... done. The following files will be added as part of updating to 13.1-RELEASE-p1: /usr/src/contrib/unbound/.github /usr/src/contrib/unbound/.github/FUNDING.yml /usr/src/contrib/unbound/contrib/drop2rpz /usr/src/contrib/unbound/contrib/unbound_portable.service.in /usr/src/contrib/unbound/services/rpz.c /usr/src/contrib/unbound/services/rpz.h /usr/src/lib/libc/tests/gen/spawnp_enoexec.sh The following files will be updated as part of updating to 13.1-RELEASE-p1: […] Installing updates...Scanning //usr/share/certs/blacklisted for certificates... Scanning //usr/share/certs/trusted for certificates... done. 13.1-RELEASE-p1 [00:04:06] Recording filesystem state for clean... done [00:04:07] Jail 13amd64 13.1-RELEASE-p1 amd64 is ready to be used .... [source,shell] .... # poudriere ports -c -p local -m git+https [00:00:00] Creating local fs at /poudriere/ports/local... done [00:00:00] Checking out the ports tree... done .... On a single computer, poudriere can build ports with multiple configurations, in multiple jails, and from different port trees. Custom configurations for these combinations are called _sets_. See the CUSTOMIZATION section of man:poudriere[8] for details after package:ports-mgmt/poudriere[] or package:ports-mgmt/poudriere-devel[] is installed. The basic configuration shown here puts a single jail-, port-, and set-specific [.filename]#make.conf# in [.filename]#/usr/local/etc/poudriere.d#. The filename in this example is created by combining the jail name, port name, and set name: [.filename]#13amd64-local-workstation-make.conf#. The system [.filename]#make.conf# and this new file are combined at build time to create the [.filename]#make.conf# used by the build jail. Packages to be built are entered in [.filename]#13amd64-local-workstation-pkglist#: [.programlisting] .... editors/emacs devel/git ports-mgmt/pkg ... .... Options and dependencies for the specified ports are configured: [source,shell] .... # poudriere options -j 13amd64 -p local -z workstation -f 13amd64-local-workstation-pkglist .... Finally, packages are built and a package repository is created: [source,shell] .... # poudriere bulk -j 13amd64 -p local -z workstation -f 13amd64-local-workstation-pkglist .... While running, pressing kbd:[Ctrl+t] displays the current state of the build. Poudriere also builds files in [.filename]#/poudriere/logs/bulk/jailname# that can be used with a web server to display build information. After completion, the new packages are now available for installation from the poudriere repository. For more information on using poudriere, see man:poudriere[8] and the main web site, https://github.com/freebsd/poudriere/wiki[]. === Configuring pkg Clients to Use a Poudriere Repository While it is possible to use both a custom repository along side of the official repository, sometimes it is useful to disable the official repository. This is done by creating a configuration file that overrides and disables the official configuration file. Create [.filename]#/usr/local/etc/pkg/repos/FreeBSD.conf# that contains the following: [.programlisting] .... FreeBSD: { enabled: no } .... Usually it is easiest to serve a poudriere repository to the client machines via HTTP. Set up a webserver to serve up the package directory, for instance: [.filename]#/usr/local/poudriere/data/packages/13amd64#, where [.filename]#13amd64# is the name of the build. If the URL to the package repository is: `http://pkg.example.com/13amd64`, then the repository configuration file in [.filename]#/usr/local/etc/pkg/repos/custom.conf# would look like: [.programlisting] .... custom: { url: "http://pkg.example.com/13amd64", enabled: yes, } .... [[ports-nextsteps]] == Post-Installation Considerations Regardless of whether the software was installed from a binary package or port, most third-party applications require some level of configuration after installation. The following commands and locations can be used to help determine what was installed with the application. * Most applications install at least one default configuration file in [.filename]#/usr/local/etc#. In cases where an application has a large number of configuration files, a subdirectory will be created to hold them. Often, sample configuration files are installed which end with a suffix such as [.filename]#.sample#. The configuration files should be reviewed and possibly edited to meet the system's needs. To edit a sample file, first copy it without the [.filename]#.sample# extension. * Applications which provide documentation will install it into [.filename]#/usr/local/share/doc# and many applications also install manual pages. This documentation should be consulted before continuing. * Some applications run services which must be added to [.filename]#/etc/rc.conf# before starting the application. These applications usually install a startup script in [.filename]#/usr/local/etc/rc.d#. See crossref:config[configtuning-starting-services,Starting Services] for more information. + [NOTE] ==== By design, applications do not run their startup script upon installation, nor do they run their stop script upon deinstallation or upgrade. This decision is left to the individual system administrator. ==== * Users of man:csh[1] should run `rehash` to rebuild the known binary list in the shells `PATH`. * Use `pkg info` to determine which files, man pages, and binaries were installed with the application. [[ports-broken]] == Dealing with Broken Ports When a port does not build or install, try the following: . Search to see if there is a fix pending for the port in the link:https://www.FreeBSD.org/support/[Problem Report database]. If so, implementing the proposed fix may fix the issue. . Ask the maintainer of the port for help. Type `make maintainer` in the ports skeleton or read the port's [.filename]#Makefile# to find the maintainer's email address. Remember to include the output leading up to the error in the email to the maintainer. + [NOTE] ==== Some ports are not maintained by an individual but instead by a group maintainer represented by a extref:{mailing-list-faq}[mailing list]. Many, but not all, of these addresses look like mailto:freebsd-listname@FreeBSD.org[freebsd-listname@FreeBSD.org]. Please take this into account when sending an email. In particular, ports maintained by mailto:ports@FreeBSD.org[ports@FreeBSD.org] are not maintained by a specific individual. Instead, any fixes and support come from the general community who subscribe to that mailing list. More volunteers are always needed! ==== + If there is no response to the email, use Bugzilla to submit a bug report using the instructions in extref:{problem-reports}[Writing FreeBSD Problem Reports]. . Fix it! The extref:{porters-handbook}[Porter's Handbook] includes detailed information on the ports infrastructure so that you can fix the occasional broken port or even submit your own! . Install the package instead of the port using the instructions in <>. diff --git a/documentation/content/en/books/porters-handbook/testing/_index.adoc b/documentation/content/en/books/porters-handbook/testing/_index.adoc index 9491c07530..c23c29766c 100644 --- a/documentation/content/en/books/porters-handbook/testing/_index.adoc +++ b/documentation/content/en/books/porters-handbook/testing/_index.adoc @@ -1,662 +1,654 @@ --- title: Chapter 10. Testing the Port prev: books/porters-handbook/pkg-files next: books/porters-handbook/upgrading description: Testing a FreeBSD Port tags: ["testing", "port", "Portclippy", "Portfmt", "Portlint", "Poudriere", "sets"] showBookMenu: true weight: 10 path: "/books/porters-handbook/" --- [[testing]] = Testing the Port :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :sectnumoffset: 10 :partnums: :source-highlighter: rouge :experimental: :images-path: books/porters-handbook/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[make-describe]] == Running `make describe` Several of the FreeBSD port maintenance tools, such as man:portupgrade[1], rely on a database called [.filename]#/usr/ports/INDEX# which keeps track of such items as port dependencies. [.filename]#INDEX# is created by the top-level [.filename]#ports/Makefile# via `make index`, which descends into each port subdirectory and executes `make describe` there. Thus, if `make describe` fails in any port, no one can generate [.filename]#INDEX#, and many people will quickly become unhappy. [NOTE] ==== It is important to be able to generate this file no matter what options are present in [.filename]#make.conf#, so please avoid doing things such as using `.error` statements when (for instance) a dependency is not satisfied. (See crossref:porting-dads[dads-dot-error,Avoid Use of the `.error` Construct].) ==== If `make describe` produces a string rather than an error message, everything is probably safe. See [.filename]#bsd.port.mk# for the meaning of the string produced. Also note that running a recent version of `portlint` (as specified in the next section) will cause `make describe` to be run automatically. [[testing-portclippy]] == Portclippy / Portfmt Those tools come from package:ports-mgmt/portfmt[]. Portclippy is a linter that checks if variables in the [.filename]#Makefile# are in the correct order according to crossref:order[porting-order,Order of Variables in Port Makefiles]. Portfmt is a tool for automatically formatting [.filename]#Makefile#. [[testing-portlint]] == Portlint Do check the port with crossref:quick-porting[porting-portlint,`portlint`] before submitting or committing it. `portlint` warns about many common errors, both functional and stylistic. For a new (or repocopied) port, `portlint -A` is the most thorough; for an existing port, `portlint -C` is sufficient. Since `portlint` uses heuristics to try to figure out errors, it can produce false positive warnings. In addition, occasionally something that is flagged as a problem really cannot be done in any other way due to limitations in the ports framework. When in doubt, the best thing to do is ask on {freebsd-ports}. [[testing-porttools]] == Port Tools The package:ports-mgmt/porttools[] program is part of the Ports Collection. `port` is the front-end script, which can help simplify the testing job. Whenever a new port or an update to an existing one needs testing, use `port test` to test the port, including the <> checking. This command also detects and lists any files that are not listed in [.filename]#pkg-plist#. For example: [source,shell] .... # port test /usr/ports/net/csup .... [[porting-prefix]] == `PREFIX` and `DESTDIR` `PREFIX` determines where the port will be installed. It defaults to [.filename]#/usr/local#, but can be set by the user to a custom path like [.filename]#/opt#. The port must respect the value of this variable. `DESTDIR`, if set by the user, determines the complete alternative environment, usually a jail or an installed system mounted somewhere other than [.filename]#/#. A port will actually install into [.filename]#DESTDIR/PREFIX#, and register with the package database in [.filename]#DESTDIR/var/db/pkg#. `DESTDIR` is handled automatically by the ports infrastructure with man:chroot[8]. There is no need for modifications or any extra care to write `DESTDIR`-compliant ports. The value of `PREFIX` will be set to `LOCALBASE` (defaulting to [.filename]#/usr/local#). If `USE_LINUX_PREFIX` is set, `PREFIX` will be `LINUXBASE` (defaulting to [.filename]#/compat/linux#). Avoiding hard-coded [.filename]#/usr/local# paths in the source makes the port much more flexible and able to cater to the needs of other sites. Often, this can be accomplished by replacing occurrences of [.filename]#/usr/local# in the port's various [.filename]##Makefile##s with `${PREFIX}`. This variable is automatically passed down to every stage of the build and install processes. Make sure the application is not installing things in [.filename]#/usr/local# instead of `PREFIX`. A quick test for such hard-coded paths is: [source,shell] .... % make clean; make package PREFIX=/var/tmp/`make -V PORTNAME` .... If anything is installed outside of `PREFIX`, the package creation process will complain that it cannot find the files. In addition, it is worth checking the same with the stage directory support (see crossref:special[staging,Staging]): [source,shell] .... % make stage && make check-plist && make stage-qa && make package .... * `check-plist` checks for files missing from the plist, and files in the plist that are not installed by the port. * `stage-qa` checks for common problems like bad shebang, symlinks pointing outside the stage directory, setuid files, and non-stripped libraries... These tests will not find hard-coded paths inside the port's files, nor will it verify that `LOCALBASE` is being used to correctly refer to files from other ports. The temporarily-installed port in [.filename]#/var/tmp/`make -V PORTNAME`# must be tested for proper operation to make sure there are no problems with paths. `PREFIX` must not be set explicitly in a port's [.filename]#Makefile#. Users installing the port may have set `PREFIX` to a custom location, and the port must respect that setting. Refer to programs and files from other ports with the variables mentioned above, not explicit pathnames. For instance, if the port requires a macro `PAGER` to have the full pathname of `less`, do not use a literal path of [.filename]#/usr/local/bin/less#. Instead, use `${LOCALBASE}`: [.programlisting] .... -DPAGER=\"${LOCALBASE}/bin/less\" .... The path with `LOCALBASE` is more likely to still work if the system administrator has moved the whole [.filename]#/usr/local# tree somewhere else. [TIP] ==== All these tests are done automatically when running `poudriere testport` or `poudriere bulk -t`. It is highly recommended that every ports contributor install and test their ports with it. See <> for more information. ==== [[testing-poudriere]] == Poudriere For a ports contributor, Poudriere is one of the most important and helpful testing and build tools. Its main features include: * Bulk building of the entire ports tree, specific subsets of the ports tree, or a single port including its dependencies * Automatic packaging of build results * Generation of build log files per port * Providing a signed man:pkg[8] repository * Testing of port builds before submitting a patch to the FreeBSD bug tracker or committing to the ports tree * Testing for successful ports builds using different options Because Poudriere performs its building in a clean man:jail[8] environment and uses man:zfs[8] features, it has several advantages over traditional testing on the host system: * No pollution of the host environment: No leftover files, no accidental removals, no changes of existing configuration files. * Verify [.filename]#pkg-plist# for missing or superfluous entries * Ports committers sometimes ask for a Poudriere log alongside a patch submission to assess whether the patch is ready for integration into the ports tree It is also quite straightforward to set up and use, has no dependencies, and will run on any supported FreeBSD release. This section shows how to install, configure, and run Poudriere as part of the normal workflow of a ports contributor. The examples in this section show a default file layout, as standard in FreeBSD. Substitute any local changes accordingly. The ports tree, represented by `${PORTSDIR}`, is located in [.filename]#/usr/ports#. Both `${LOCALBASE}` and `${PREFIX}` are [.filename]#/usr/local# by default. [[testing-poudriere-installing]] === Installing Poudriere Poudriere is available in the ports tree in package:ports-mgmt/poudriere[]. It can be installed using man:pkg[8] or from ports: [source,shell] .... # pkg install poudriere .... or [source,shell] .... # make -C /usr/ports/ports-mgmt/poudriere install clean .... There is also a work-in-progress version of Poudriere which will eventually become the next release. It is available in package:ports-mgmt/poudriere-devel[]. This development version is used for the official FreeBSD package builds, so it is well tested. It often has newer interesting features. A ports committer will want to use the development version because it is what is used in production, and has all the new features that will make sure everything is exactly right. A contributor will not necessarily need those as the most important fixes are backported to released version. The main reason for the use of the development version to build the official package is because it is faster, in a way that will shorten a full build from 18 hours to 17 hours when using a high end 32 CPU server with 128GB of RAM. Those optimizations will not matter a lot when building ports on a desktop machine. [[testing-poudriere-setup]] === Setting Up Poudriere The port installs a default configuration file, [.filename]#/usr/local/etc/poudriere.conf#. Each parameter is documented in the configuration file and in man:poudriere[8]. Here is a minimal example config file: [.programlisting] .... ZPOOL=tank ZROOTFS=/poudriere BASEFS=/poudriere DISTFILES_CACHE=/usr/ports/distfiles RESOLV_CONF=/etc/resolv.conf FREEBSD_HOST=ftp://ftp.freebsd.org SVN_HOST=svn.FreeBSD.org .... `ZPOOL`:: The name of the ZFS storage pool which Poudriere shall use. Must be listed in the output of `zpool status`. `ZROOTFS`:: The root of Poudriere-managed file systems. This entry will cause Poudriere to create man:zfs[8] file systems under `tank/poudriere`. `BASEFS`:: The root mount point for Poudriere file systems. This entry will cause Poudriere to mount `tank/poudriere` to `/poudriere`. `DISTFILES_CACHE`:: Defines where distfiles are stored. In this example, Poudriere and the host share the distfiles storage directory. This avoids downloading tarballs which are already present on the system. Please create this directory if it does not already exist so that Poudriere can find it. `RESOLV_CONF`:: Use the host [.filename]#/etc/resolv.conf# inside jails for DNS. This is needed so jails can resolve the URLs of distfiles when downloading. It is not needed when using a proxy. Refer to the default configuration file for proxy configuration. `FREEBSD_HOST`:: The FTP/HTTP server to use when the jails are installed from FreeBSD releases and updated with man:freebsd-update[8]. Choose a server location which is close, for example if the machine is located in Australia, use `ftp.au.freebsd.org`. `SVN_HOST`:: The server from where jails are installed and updated when using Subversion. -Also used for ports tree when not using man:portsnap[8]. Again, choose a nearby location. A list of official Subversion mirrors can be found in the extref:{handbook}[FreeBSD Handbook Subversion section, svn-mirrors]. [[testing-poudriere-create-jails]] === Creating Poudriere Jails Create the base jails which Poudriere will use for building: [source,shell] .... # poudriere jail -c -j 131Ramd64 -v 13.1-RELEASE -a amd64 .... Fetch a `13.1-RELEASE` for `amd64` from the FTP server given by `FREEBSD_HOST` in [.filename]#poudriere.conf#, create the zfs file system `tank/poudriere/jails/131Ramd64`, mount it on [.filename]#/poudriere/jails/131Ramd64# and extract the `13.1-RELEASE` tarballs into this file system. [source,shell] .... # poudriere jail -c -j 12i386 -v stable/12 -a i386 -m git+https .... Create `tank/poudriere/jails/12i386`, mount it on [.filename]#/poudriere/jails/12i386#, then check out the tip of the Subversion branch of `FreeBSD-12-STABLE` from `SVN_HOST` in [.filename]#poudriere.conf# into [.filename]#/poudriere/jails/12i386/usr/src#, then complete a `buildworld` and install it into [.filename]#/poudriere/jails/12i386#. [TIP] ==== If a specific Subversion revision is needed, append it to the version string. For example: [source,shell] .... # poudriere jail -c -j 12i386 -v stable/12@123456 -a i386 -m git+https .... ==== [NOTE] ==== While it is possible to build a newer version of FreeBSD on an older version, most of the time it will not run. For example, if a `stable/13` jail is needed, the host will have to run `stable/13` too. Running `13.1-RELEASE` is not enough. ==== [NOTE] ==== To create a Poudriere jail for `14.0-CURRENT`: [source,shell] .... # poudriere jail -c -j 14amd64 -v main -a amd64 -m git+https .... In order to run a `14.0-CURRENT` Poudriere jail you must be running `14.0-CURRENT`. In general, newer kernels can build and run older jails. For instance, a `14.0-CURRENT` kernel can build and run a `12.3-STABLE`. Poudriere jail if the `COMPAT_FREEBSD12` kernel option was compiled in (on by default in `14.0-CURRENT`[.filename]#GENERIC# kernel config). ==== [CAUTION] ==== The default `svn` protocol works but is not very secure. Using `svn+https` along with verifying the remote server's SSL fingerprint is advised. It will ensure that the files used for building the jail are from a trusted source. ==== A list of jails currently known to Poudriere can be shown with `poudriere jail -l`: [source,shell] .... # poudriere jail -l JAILNAME VERSION ARCH METHOD 131Ramd64 13.1-RELEASE amd64 ftp 12i386 12.3-STABLE i386 git+https .... [[testing-poudriere-maintaining-jails]] === Keeping Poudriere Jails Updated Managing updates is very straightforward. The command: [source,shell] .... # poudriere jail -u -j JAILNAME .... updates the specified jail to the latest version available. For FreeBSD releases, update to the latest patchlevel with man:freebsd-update[8]. For FreeBSD versions built from source, update to the latest Subversion revision in the branch. [TIP] ==== For jails employing a `git+*` method, it is helpful to add `-J _NumberOfParallelBuildJobs_` to speed up the build by increasing the number of parallel compile jobs used. For example, if the building machine has 6 CPUs, use: [source,shell] .... # poudriere jail -u -J 6 -j JAILNAME .... ==== [[testing-poudriere-ports-tree]] === Setting Up Ports Trees for Use with Poudriere There are multiple ways to use ports trees in Poudriere. -The most straightforward way is to have Poudriere create a default ports tree for itself, using either man:portsnap[8] (if running FreeBSD {rel121-current} or {rel114-current}) or Git (if running FreeBSD-CURRENT): - -[source,shell] -.... -# poudriere ports -c -m portsnap -.... - -or +The most straightforward way is to have Poudriere create a default ports tree for itself, using link:{handbook}mirrors/#git[Git]: [source,shell] .... # poudriere ports -c -m git+https -B main .... -These commands create `tank/poudriere/ports/default`, mount it on [.filename]#/poudriere/ports/default#, and populate it using Git, man:portsnap[8], or Subversion. +These commands create `tank/poudriere/ports/default`, mount it on [.filename]#/poudriere/ports/default#, and populate it using Git. Afterward it is included in the list of known ports trees: [source,shell] .... # poudriere ports -l PORTSTREE METHOD TIMESTAMP PATH default git+https 2020-07-20 04:23:56 /poudriere/ports/default .... [NOTE] ==== Note that the "default" ports tree is special. Each of the build commands explained later will implicitly use this ports tree unless specifically specified otherwise. To use another tree, add `-p _treename_` to the commands. ==== -While useful for regular bulk builds, having this default ports tree with the man:portsnap[8] method may not be the best way to deal with local modifications for a ports contributor. +The best way to deal with local modifications for a ports contributor is to use link:{handbook}mirrors/#git[Git]. As with the creation of jails, it is possible to use a different method for creating the ports tree. To add an additional ports tree for testing local modifications and ports development, checking out the tree via Subversion (as described above) is preferable. [NOTE] ==== The http and https methods need package:devel/subversion[] built with the `SERF` option enabled. It is enabled by default. ==== [TIP] ==== The `svn` method allows extra qualifiers to tell Subversion exactly how to fetch data. This is explained in man:poudriere[8]. For instance, `poudriere ports -c -m svn+ssh -p subversive` uses SSH for the checkout. ==== [[testing-poudriere-ports-tree-manual]] === Using Manually Managed Ports Trees with Poudriere Depending on the workflow, it can be extremely helpful to use ports trees which are maintained manually. For instance, if there is a local copy of the ports tree in [.filename]#/work/ports#, point Poudriere to the location: * For Poudriere older than version 3.1.20: + [source,shell] .... # poudriere ports -c -F -f none -M /work/ports -p development .... * For Poudriere version 3.1.20 and later: + [source,shell] .... # poudriere ports -c -m null -M /work/ports -p development .... This will be listed in the table of known trees: [source,shell] .... # poudriere ports -l PORTSTREE METHOD TIMESTAMP PATH development null 2020-07-20 05:06:33 /work/ports .... [NOTE] ==== The dash or `null` in the `METHOD` column means that Poudriere will not update or change this ports tree, ever. It is completely up to the user to maintain this tree, including all local modifications that may be used for testing new ports and submitting patches. ==== [[testing-poudriere-ports-tree-updating]] === Keeping Poudriere Ports Trees Updated As straightforward as with jails described earlier: [source,shell] .... # poudriere ports -u -p PORTSTREE .... Will update the given _PORTSTREE_, one tree given by the output of `poudriere -l`, to the latest revision available on the official servers. [NOTE] ==== Ports trees without a method, see <>, cannot be updated like this. They must be updated manually by the porter. ==== [[testing-poudriere-testing-ports]] === Testing Ports After jails and ports trees have been set up, the result of a contributor's modifications to the ports tree can be tested. For example, local modifications to the package:www/firefox[] port located in [.filename]#/work/ports/www/firefox# can be tested in the previously created 13.1-RELEASE jail: [source,shell] .... # poudriere testport -j 131Ramd64 -p development -o www/firefox .... This will build all dependencies of Firefox. If a dependency has been built previously and is still up-to-date, the pre-built package is installed. If a dependency has no up-to-date package, one will be built with default options in a jail. Then Firefox itself is built. The complete build of every port is logged to [.filename]#/poudriere/data/logs/bulk/131Ri386-development/build-time/logs#. The directory name `131Ri386-development` is derived from the arguments to `-j` and `-p`, respectively. For convenience, a symbolic link [.filename]#/poudriere/data/logs/bulk/131Ri386-development/latest# is also maintained. The link points to the latest _build-time_ directory. Also in this directory is an [.filename]#index.html# for observing the build process with a web browser. By default, Poudriere cleans up the jails and leaves log files in the directories mentioned above. To ease investigation, jails can be kept running after the build by adding `-i` to `testport`: [source,shell] .... # poudriere testport -j 131Ramd64 -p development -i -o www/firefox .... After the build completes, and regardless of whether it was successful, a shell is provided within the jail. The shell is used to investigate further. Poudriere can be told to leave the jail running after the build finishes with `-I`. Poudriere will show the command to run when the jail is no longer needed. It is then possible to man:jexec[8] into it: [source,shell] .... # poudriere testport -j 131Ramd64 -p development -I -o www/firefox [...] ====>> Installing local Pkg repository to /usr/local/etc/pkg/repos ====>> Leaving jail 131Ramd64-development-n running, mounted at /poudriere/data/.m/131Ramd64-development/ref for interactive run testing ====>> To enter jail: jexec 131Ramd64-development-n env -i TERM=$TERM /usr/bin/login -fp root ====>> To stop jail: poudriere jail -k -j 131Ramd64 -p development # jexec 131Ramd64-development-n env -i TERM=$TERM /usr/bin/login -fp root # [do some stuff in the jail] # exit # poudriere jail -k -j 131Ramd64 -p development ====>> Umounting file systems .... An integral part of the FreeBSD ports build infrastructure is the ability to tweak ports to personal preferences with options. These can be tested with Poudriere as well. Adding the `-c`: [source,shell] .... # poudriere testport -c -o www/firefox .... Presents the port configuration dialog before the port is built. The ports given after `-o` in the format `_category_/_portname_` will use the specified options, all dependencies will use the default options. Testing dependent ports with non-default options can be accomplished using sets, see <>. [TIP] ==== When testing ports where [.filename]#pkg-plist# is altered during build depending on the selected options, it is recommended to perform a test run with all options selected _and_ one with all options deselected. ==== [[testing-poudriere-sets]] === Using Sets For all actions involving builds, a so-called _set_ can be specified using `-z _setname_`. A set refers to a fully independent build. This allows, for instance, usage of `testport` with non-standard options for the dependent ports. To use sets, Poudriere expects an existing directory structure similar to `PORT_DBDIR`, defaults to [.filename]#/var/db/ports# in its configuration directory. This directory is then man:nullfs[5]-mounted into the jails where the ports and their dependencies are built. Usually a suitable starting point can be obtained by recursively copying the existing `PORT_DBDIR` to [.filename]#/usr/local/etc/poudriere.d/jailname-portname-setname-options#. This is described in detail in man:poudriere[8]. For instance, testing package:www/firefox[] in a specific set named `devset`, add the `-z devset` parameter to the testport command: [source,shell] .... # poudriere testport -j 131Ramd64 -p development -z devset -o www/firefox .... This will look for the existence of these directories in this order: * [.filename]#/usr/local/etc/poudriere.d/131Ramd64-development-devset-options# * [.filename]#/usr/local/etc/poudriere.d/131Ramd64-devset-options# * [.filename]#/usr/local/etc/poudriere.d/131Ramd64-development-options# * [.filename]#/usr/local/etc/poudriere.d/devset-options# * [.filename]#/usr/local/etc/poudriere.d/development-options# * [.filename]#/usr/local/etc/poudriere.d/131Ramd64-options# * [.filename]#/usr/local/etc/poudriere.d/options# From this list, Poudriere man:nullfs[5]-mounts the _first existing_ directory tree into the [.filename]#/var/db/ports# directory of the build jails. Hence, all custom options are used for all the ports during this run of `testport`. After the directory structure for a set is provided, the options for a particular port can be altered. For example: [source,shell] .... # poudriere options -c www/firefox -z devset .... The configuration dialog for package:www/firefox[] is shown, and options can be edited. The selected options are saved to the `devset` set. [NOTE] ==== Poudriere is very flexible in the option configuration. They can be set for particular jails, ports trees, and for multiple ports by one command. Refer to man:poudriere[8] for details. ==== [[testing-poudriere-make-conf]] === Providing a Custom [.filename]#make.conf# File Similar to using sets, Poudriere will also use a custom [.filename]#make.conf# if it is provided. No special command line argument is necessary. Instead, Poudriere looks for existing files matching a name scheme derived from the command line. For instance: [source,shell] .... # poudriere testport -j 131Ramd64 -p development -z devset -o www/firefox .... causes Poudriere to check for the existence of these files in this order: * [.filename]#/usr/local/etc/poudriere.d/make.conf# * [.filename]#/usr/local/etc/poudriere.d/devset-make.conf# * [.filename]#/usr/local/etc/poudriere.d/development-make.conf# * [.filename]#/usr/local/etc/poudriere.d/131Ramd64-make.conf# * [.filename]#/usr/local/etc/poudriere.d/131Ramd64-development-make.conf# * [.filename]#/usr/local/etc/poudriere.d/131Ramd64-devset-make.conf# * [.filename]#/usr/local/etc/poudriere.d/131Ramd64-development-devset-make.conf# Unlike with sets, all of the found files will be appended, _in that order_, into one [.filename]#make.conf# inside the build jails. It is hence possible to have general make variables, intended to affect all builds in [.filename]#/usr/local/etc/poudriere.d/make.conf#. Special variables, intended to affect only certain jails or sets can be set in specialised [.filename]#make.conf# files, such as [.filename]#/usr/local/etc/poudriere.d/131Ramd64-development-devset-make.conf#. [[testing-poudriere-sets-perl]] .Using [.filename]#make.conf# to Change Default Perl [example] ==== To build a set with a non default Perl version, for example, `5.20`, using a set named `perl5-20`, create a [.filename]#perl5-20-make.conf# with this line: [.programlisting] .... DEFAULT_VERSIONS+= perl=5.20 .... [NOTE] **** Note the use of `+=` so that if the variable is already set in the default [.filename]#make.conf# its content will not be overwritten. **** ==== [[testing-poudriere-pruning-distfiles]] === Pruning no Longer Needed Distfiles Poudriere comes with a built-in mechanism to remove outdated distfiles that are no longer used by any port of a given tree. The command [source,shell] .... # poudriere distclean -p portstree .... will scan the distfiles folder, `DISTFILES_CACHE` in [.filename]#poudriere.conf#, versus the ports tree given by the `-p _portstree_` argument and prompt for removal of those distfiles. To skip the prompt and remove all unused files unconditionally, the `-y` argument can be added: [source,shell] .... # poudriere distclean -p portstree -y .... diff --git a/website/content/en/ports/installing.adoc b/website/content/en/ports/installing.adoc index ddc619e673..cd69270646 100644 --- a/website/content/en/ports/installing.adoc +++ b/website/content/en/ports/installing.adoc @@ -1,18 +1,18 @@ --- title: "Installing the FreeBSD Ports Collection" sidenav: ports --- include::shared/releases.adoc[] include::shared/en/urls.adoc[] = Installing the FreeBSD Ports Collection {{< form-ports >}} ''''' The FreeBSD installer programs allow the Ports Collection to be installed at the same time as the operating system. See link:{handbook}bsdinstall[installing FreeBSD]. -If not initially installed, it can be added with link:{handbook}ports/#ports-using-portsnap-method[portsnap] or link:{handbook}ports/#ports-using-git-method[Git]. +If not initially installed, it can be added with link:{handbook}ports/#ports-using-git-method[Git].