diff --git a/documentation/content/el/books/handbook/security/_index.adoc b/documentation/content/el/books/handbook/security/_index.adoc index 33114a15cf..615a41c964 100644 --- a/documentation/content/el/books/handbook/security/_index.adoc +++ b/documentation/content/el/books/handbook/security/_index.adoc @@ -1,2315 +1,2315 @@ --- title: Κεφάλαιο 15. Ασφάλεια part: Μέρος III. Διαχείριση Συστήματος prev: books/handbook/users next: books/handbook/jails showBookMenu: true weight: 19 path: "/books/handbook/" --- [[security]] = Ασφάλεια :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :sectnumoffset: 15 :partnums: :source-highlighter: rouge :experimental: :images-path: books/handbook/security/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[security-synopsis]] == Σύνοψη Το κεφάλαιο αυτό παρέχει μια βασική εισαγωγή στις έννοιες της ασφάλειας συστήματος, κάποιους γενικά καλούς κανόνες, και ορισμένα προχωρημένα θέματα σχετικά με το FreeBSD. Αρκετά από τα θέματα που καλύπτονται εδώ, μπορούν να εφαρμοστούν το ίδιο καλά τόσο στο ίδιο το σύστημα, όσο και για ασφάλεια μέσω Internet. Το Internet δεν είναι πλέον ένα "φιλικό" μέρος στο οποίο καθένας θέλει να είναι ο ευγενικός σας γείτονας. Η ανάγκη ασφάλισης του συστήματος σας είναι επιτακτική για να προστατέψετε τα δεδομένα σας,την πνευματική σας ιδιοκτησία, το χρόνο σας, και πολλά περισσότερα από τα χέρια των χάκερς και των ομοίων τους. Το FreeBSD παρέχει μια σειρά από βοηθητικά προγράμματα και μηχανισμούς για να εξασφαλίσει την ακεραιότητα και την ασφάλεια του συστήματος σας και του δικτύου. Αφού διαβάσετε αυτό το κεφάλαιο, θα ξέρετε: * Βασικές έννοιες για την ασφάλεια, σε σχέση με το FreeBSD. * Στοιχεία σχετικά με τους διάφορους μηχανισμούς κρυπτογράφησης που είναι διαθέσιμοι στο FreeBSD, όπως το DES και το MD5. * Πως να ρυθμίσετε το σύστημα σας για κωδικούς μιας χρήσης. * Πως να ρυθμίσετε TCP Wrappers για χρήση με την `inetd`. * Πως να ρυθμίσετε τον KerberosIV σε FreeBSD εκδόσεις πριν τη 5.0. * Πως να ρυθμίσετε τον Kerberos5 στο FreeBSD. * Πως να ρυθμίσετε το IPsec και να δημιουργήσετε ένα VPN μεταξύ μηχανημάτων FreeBSD/Windows(R). * Πως να ρυθμίσετε και να χρησιμοποιήσετε την κατά FreeBSD υλοποίηση SSH του OpenSSH * Τι είναι τα ACLs στο σύστημα αρχείων και πως να τα χρησιμοποιήσετε. * Πως να χρησιμοποιήσετε το βοηθητικό πρόγραμμα Portaudit για να ελέγξετε λογισμικό τρίτου κατασκευαστή που έχει εγκατασταθεί μέσω της συλλογής Ports. * Πως να χρησιμοποιήσετε τις δημοσιεύσεις security advisories του FreeBSD. * Θα έχετε μια ιδέα για το τι είναι το Process Accounting και πως να το ενεργοποιήσετε στο FreeBSD. Πριν διαβάσετε αυτό το κεφάλαιο, θα πρέπει: * Να κατανοείτε βασικές έννοιες του FreeBSD και του Internet. Πρόσθετα θέματα σχετικά με την ασφάλεια καλύπτονται σε ολόκληρο το βιβλίο. Για παράδειγμα, ο Υποχρεωτικός Έλεγχος Πρόσβασης συζητείται στο crossref:mac[mac,Υποχρεωτικός Έλεγχος Πρόσβασης] και τα Internet Firewalls συζητούνται στο crossref:firewalls[firewalls,Firewalls]. [[security-intro]] == Introduction Security is a function that begins and ends with the system administrator. While all BSD UNIX(R) multi-user systems have some inherent security, the job of building and maintaining additional security mechanisms to keep those users "honest" is probably one of the single largest undertakings of the sysadmin. Machines are only as secure as you make them, and security concerns are ever competing with the human necessity for convenience. UNIX(R) systems, in general, are capable of running a huge number of simultaneous processes and many of these processes operate as servers - meaning that external entities can connect and talk to them. As yesterday's mini-computers and mainframes become today's desktops, and as computers become networked and inter-networked, security becomes an even bigger issue. System security also pertains to dealing with various forms of attack, including attacks that attempt to crash, or otherwise make a system unusable, but do not attempt to compromise the `root` account ("break root"). Security concerns can be split up into several categories: . Denial of service attacks. . User account compromises. . Root compromise through accessible servers. . Root compromise via user accounts. . Backdoor creation. A denial of service attack is an action that deprives the machine of needed resources. Typically, DoS attacks are brute-force mechanisms that attempt to crash or otherwise make a machine unusable by overwhelming its servers or network stack. Some DoS attacks try to take advantage of bugs in the networking stack to crash a machine with a single packet. The latter can only be fixed by applying a bug fix to the kernel. Attacks on servers can often be fixed by properly specifying options to limit the load the servers incur on the system under adverse conditions. Brute-force network attacks are harder to deal with. A spoofed-packet attack, for example, is nearly impossible to stop, short of cutting your system off from the Internet. It may not be able to take your machine down, but it can saturate your Internet connection. A user account compromise is even more common than a DoS attack. Many sysadmins still run standard telnetd, rlogind, rshd, and ftpd servers on their machines. These servers, by default, do not operate over encrypted connections. The result is that if you have any moderate-sized user base, one or more of your users logging into your system from a remote location (which is the most common and convenient way to login to a system) will have his or her password sniffed. The attentive system admin will analyze his remote access logs looking for suspicious source addresses even for successful logins. One must always assume that once an attacker has access to a user account, the attacker can break `root`. However, the reality is that in a well secured and maintained system, access to a user account does not necessarily give the attacker access to `root`. The distinction is important because without access to `root` the attacker cannot generally hide his tracks and may, at best, be able to do nothing more than mess with the user's files, or crash the machine. User account compromises are very common because users tend not to take the precautions that sysadmins take. System administrators must keep in mind that there are potentially many ways to break `root` on a machine. The attacker may know the `root` password, the attacker may find a bug in a root-run server and be able to break `root` over a network connection to that server, or the attacker may know of a bug in a suid-root program that allows the attacker to break `root` once he has broken into a user's account. If an attacker has found a way to break `root` on a machine, the attacker may not have a need to install a backdoor. Many of the `root` holes found and closed to date involve a considerable amount of work by the attacker to cleanup after himself, so most attackers install backdoors. A backdoor provides the attacker with a way to easily regain `root` access to the system, but it also gives the smart system administrator a convenient way to detect the intrusion. Making it impossible for an attacker to install a backdoor may actually be detrimental to your security, because it will not close off the hole the attacker found to break in the first place. Security remedies should always be implemented with a multi-layered "onion peel" approach and can be categorized as follows: . Securing `root` and staff accounts. . Securing `root`-run servers and suid/sgid binaries. . Securing user accounts. . Securing the password file. . Securing the kernel core, raw devices, and file systems. . Quick detection of inappropriate changes made to the system. . Paranoia. The next section of this chapter will cover the above bullet items in greater depth. [[securing-freebsd]] == Securing FreeBSD [NOTE] .Command vs. Protocol ==== Throughout this document, we will use bold text to refer to an application, and a `monospaced` font to refer to specific commands. Protocols will use a normal font. This typographical distinction is useful for instances such as ssh, since it is a protocol as well as command. ==== The sections that follow will cover the methods of securing your FreeBSD system that were mentioned in the crossref:mac[security-intro,last section] of this chapter. [[securing-root-and-staff]] === Securing the `root` Account and Staff Accounts First off, do not bother securing staff accounts if you have not secured the `root` account. Most systems have a password assigned to the `root` account. The first thing you do is assume that the password is _always_ compromised. This does not mean that you should remove the password. The password is almost always necessary for console access to the machine. What it does mean is that you should not make it possible to use the password outside of the console or possibly even with the man:su[1] command. For example, make sure that your ptys are specified as being insecure in the [.filename]#/etc/ttys# file so that direct `root` logins via `telnet` or `rlogin` are disallowed. If using other login services such as sshd, make sure that direct `root` logins are disabled there as well. You can do this by editing your [.filename]#/etc/ssh/sshd_config# file, and making sure that `PermitRootLogin` is set to `NO`. Consider every access method - services such as FTP often fall through the cracks. Direct `root` logins should only be allowed via the system console. Of course, as a sysadmin you have to be able to get to `root`, so we open up a few holes. But we make sure these holes require additional password verification to operate. One way to make `root` accessible is to add appropriate staff accounts to the `wheel` group (in [.filename]#/etc/group#). The staff members placed in the `wheel` group are allowed to `su` to `root`. You should never give staff members native `wheel` access by putting them in the `wheel` group in their password entry. Staff accounts should be placed in a `staff` group, and then added to the `wheel` group via the [.filename]#/etc/group# file. Only those staff members who actually need to have `root` access should be placed in the `wheel` group. It is also possible, when using an authentication method such as Kerberos, to use Kerberos' [.filename]#.k5login# file in the `root` account to allow a man:ksu[1] to `root` without having to place anyone at all in the `wheel` group. This may be the better solution since the `wheel` mechanism still allows an intruder to break `root` if the intruder has gotten hold of your password file and can break into a staff account. While having the `wheel` mechanism is better than having nothing at all, it is not necessarily the safest option. An indirect way to secure staff accounts, and ultimately `root` access is to use an alternative login access method and do what is known as "starring" out the encrypted password for the staff accounts. Using the man:vipw[8] command, one can replace each instance of an encrypted password with a single "`*`" character. This command will update the [.filename]#/etc/master.passwd# file and user/password database to disable password-authenticated logins. A staff account entry such as: [.programlisting] .... foobar:R9DT/Fa1/LV9U:1000:1000::0:0:Foo Bar:/home/foobar:/usr/local/bin/tcsh .... Should be changed to this: [.programlisting] .... foobar:*:1000:1000::0:0:Foo Bar:/home/foobar:/usr/local/bin/tcsh .... This change will prevent normal logins from occurring, since the encrypted password will never match "`*`". With this done, staff members must use another mechanism to authenticate themselves such as man:kerberos[1] or man:ssh[1] using a public/private key pair. When using something like Kerberos, one generally must secure the machines which run the Kerberos servers and your desktop workstation. When using a public/private key pair with ssh, one must generally secure the machine used to login _from_ (typically one's workstation). An additional layer of protection can be added to the key pair by password protecting the key pair when creating it with man:ssh-keygen[1]. Being able to "star" out the passwords for staff accounts also guarantees that staff members can only login through secure access methods that you have set up. This forces all staff members to use secure, encrypted connections for all of their sessions, which closes an important hole used by many intruders: sniffing the network from an unrelated, less secure machine. The more indirect security mechanisms also assume that you are logging in from a more restrictive server to a less restrictive server. For example, if your main box is running all sorts of servers, your workstation should not be running any. In order for your workstation to be reasonably secure you should run as few servers as possible, up to and including no servers at all, and you should run a password-protected screen blanker. Of course, given physical access to a workstation an attacker can break any sort of security you put on it. This is definitely a problem that you should consider, but you should also consider the fact that the vast majority of break-ins occur remotely, over a network, from people who do not have physical access to your workstation or servers. Using something like Kerberos also gives you the ability to disable or change the password for a staff account in one place, and have it immediately affect all the machines on which the staff member may have an account. If a staff member's account gets compromised, the ability to instantly change his password on all machines should not be underrated. With discrete passwords, changing a password on N machines can be a mess. You can also impose re-passwording restrictions with Kerberos: not only can a Kerberos ticket be made to timeout after a while, but the Kerberos system can require that the user choose a new password after a certain period of time (say, once a month). === Securing Root-run Servers and SUID/SGID Binaries The prudent sysadmin only runs the servers he needs to, no more, no less. Be aware that third party servers are often the most bug-prone. For example, running an old version of imapd or popper is like giving a universal `root` ticket out to the entire world. Never run a server that you have not checked out carefully. Many servers do not need to be run as `root`. For example, the ntalk, comsat, and finger daemons can be run in special user _sandboxes_. A sandbox is not perfect, unless you go through a large amount of trouble, but the onion approach to security still stands: If someone is able to break in through a server running in a sandbox, they still have to break out of the sandbox. The more layers the attacker must break through, the lower the likelihood of his success. Root holes have historically been found in virtually every server ever run as `root`, including basic system servers. If you are running a machine through which people only login via sshd and never login via telnetd or rshd or rlogind, then turn off those services! FreeBSD now defaults to running ntalkd, comsat, and finger in a sandbox. Another program which may be a candidate for running in a sandbox is man:named[8]. [.filename]#/etc/defaults/rc.conf# includes the arguments necessary to run named in a sandbox in a commented-out form. Depending on whether you are installing a new system or upgrading an existing system, the special user accounts used by these sandboxes may not be installed. The prudent sysadmin would research and implement sandboxes for servers whenever possible. There are a number of other servers that typically do not run in sandboxes: sendmail, popper, imapd, ftpd, and others. There are alternatives to some of these, but installing them may require more work than you are willing to perform (the convenience factor strikes again). You may have to run these servers as `root` and rely on other mechanisms to detect break-ins that might occur through them. The other big potential `root` holes in a system are the suid-root and sgid binaries installed on the system. Most of these binaries, such as rlogin, reside in [.filename]#/bin#, [.filename]#/sbin#, [.filename]#/usr/bin#, or [.filename]#/usr/sbin#. While nothing is 100% safe, the system-default suid and sgid binaries can be considered reasonably safe. Still, `root` holes are occasionally found in these binaries. A `root` hole was found in `Xlib` in 1998 that made xterm (which is typically suid) vulnerable. It is better to be safe than sorry and the prudent sysadmin will restrict suid binaries, that only staff should run, to a special group that only staff can access, and get rid of (`chmod 000`) any suid binaries that nobody uses. A server with no display generally does not need an xterm binary. Sgid binaries can be almost as dangerous. If an intruder can break an sgid-kmem binary, the intruder might be able to read [.filename]#/dev/kmem# and thus read the encrypted password file, potentially compromising any passworded account. Alternatively an intruder who breaks group `kmem` can monitor keystrokes sent through ptys, including ptys used by users who login through secure methods. An intruder that breaks the `tty` group can write to almost any user's tty. If a user is running a terminal program or emulator with a keyboard-simulation feature, the intruder can potentially generate a data stream that causes the user's terminal to echo a command, which is then run as that user. [[secure-users]] === Securing User Accounts User accounts are usually the most difficult to secure. While you can impose draconian access restrictions on your staff and "star" out their passwords, you may not be able to do so with any general user accounts you might have. If you do have sufficient control, then you may win out and be able to secure the user accounts properly. If not, you simply have to be more vigilant in your monitoring of those accounts. Use of ssh and Kerberos for user accounts is more problematic, due to the extra administration and technical support required, but still a very good solution compared to a encrypted password file. === Securing the Password File The only sure fire way is to star out as many passwords as you can and use ssh or Kerberos for access to those accounts. Even though the encrypted password file ([.filename]#/etc/spwd.db#) can only be read by `root`, it may be possible for an intruder to obtain read access to that file even if the attacker cannot obtain root-write access. Your security scripts should always check for and report changes to the password file (see the <> section below). === Securing the Kernel Core, Raw Devices, and File systems If an attacker breaks `root` he can do just about anything, but there are certain conveniences. For example, most modern kernels have a packet sniffing device driver built in. Under FreeBSD it is called the [.filename]#bpf# device. An intruder will commonly attempt to run a packet sniffer on a compromised machine. You do not need to give the intruder the capability and most systems do not have the need for the [.filename]#bpf# device compiled in. But even if you turn off the [.filename]#bpf# device, you still have [.filename]#/dev/mem# and [.filename]#/dev/kmem# to worry about. For that matter, the intruder can still write to raw disk devices. Also, there is another kernel feature called the module loader, man:kldload[8]. An enterprising intruder can use a KLD module to install his own [.filename]#bpf# device, or other sniffing device, on a running kernel. To avoid these problems you have to run the kernel at a higher secure level, at least securelevel 1. The securelevel can be set with a `sysctl` on the `kern.securelevel` variable. Once you have set the securelevel to 1, write access to raw devices will be denied and special `chflags` flags, such as `schg`, will be enforced. You must also ensure that the `schg` flag is set on critical startup binaries, directories, and script files - everything that gets run up to the point where the securelevel is set. This might be overdoing it, and upgrading the system is much more difficult when you operate at a higher secure level. You may compromise and run the system at a higher secure level but not set the `schg` flag for every system file and directory under the sun. Another possibility is to simply mount [.filename]#/# and [.filename]#/usr# read-only. It should be noted that being too draconian in what you attempt to protect may prevent the all-important detection of an intrusion. [[security-integrity]] === Checking File Integrity: Binaries, Configuration Files, Etc. When it comes right down to it, you can only protect your core system configuration and control files so much before the convenience factor rears its ugly head. For example, using `chflags` to set the `schg` bit on most of the files in [.filename]#/# and [.filename]#/usr# is probably counterproductive, because while it may protect the files, it also closes a detection window. The last layer of your security onion is perhaps the most important - detection. The rest of your security is pretty much useless (or, worse, presents you with a false sense of security) if you cannot detect potential intrusions. Half the job of the onion is to slow down the attacker, rather than stop him, in order to be able to catch him in the act. The best way to detect an intrusion is to look for modified, missing, or unexpected files. The best way to look for modified files is from another (often centralized) limited-access system. Writing your security scripts on the extra-secure limited-access system makes them mostly invisible to potential attackers, and this is important. In order to take maximum advantage you generally have to give the limited-access box significant access to the other machines in the business, usually either by doing a read-only NFS export of the other machines to the limited-access box, or by setting up ssh key-pairs to allow the limited-access box to ssh to the other machines. Except for its network traffic, NFS is the least visible method - allowing you to monitor the file systems on each client box virtually undetected. If your limited-access server is connected to the client boxes through a switch, the NFS method is often the better choice. If your limited-access server is connected to the client boxes through a hub, or through several layers of routing, the NFS method may be too insecure (network-wise) and using ssh may be the better choice even with the audit-trail tracks that ssh lays. Once you have given a limited-access box at least read access to the client systems it is supposed to monitor, you must write scripts to do the actual monitoring. Given an NFS mount, you can write scripts out of simple system utilities such as man:find[1] and man:md5[1]. It is best to physically md5 the client-box files at least once a day, and to test control files such as those found in [.filename]#/etc# and [.filename]#/usr/local/etc# even more often. When mismatches are found, relative to the base md5 information the limited-access machine knows is valid, it should scream at a sysadmin to go check it out. A good security script will also check for inappropriate suid binaries and for new or deleted files on system partitions such as [.filename]#/# and [.filename]#/usr#. When using ssh rather than NFS, writing the security script is much more difficult. You essentially have to `scp` the scripts to the client box in order to run them, making them visible, and for safety you also need to `scp` the binaries (such as find) that those scripts use. The ssh client on the client box may already be compromised. All in all, using ssh may be necessary when running over insecure links, but it is also a lot harder to deal with. A good security script will also check for changes to user and staff members access configuration files: [.filename]#.rhosts#, [.filename]#.shosts#, [.filename]#.ssh/authorized_keys# and so forth, files that might fall outside the purview of the `MD5` check. If you have a huge amount of user disk space, it may take too long to run through every file on those partitions. In this case, setting mount flags to disallow suid binaries and devices on those partitions is a good idea. The `nodev` and `nosuid` options (see man:mount[8]) are what you want to look into. You should probably scan them anyway, at least once a week, since the object of this layer is to detect a break-in attempt, whether or not the attempt succeeds. -Process accounting (see man:accton[8]) is a relatively low-overhead feature of the operating system which might help as a post-break-in evaluation mechanism. It is especially useful in tracking down how an intruder has actually broken into a system, assuming the file is still intact after the break-in has occured. +Process accounting (see man:accton[8]) is a relatively low-overhead feature of the operating system which might help as a post-break-in evaluation mechanism. It is especially useful in tracking down how an intruder has actually broken into a system, assuming the file is still intact after the break-in has occurred. Finally, security scripts should process the log files, and the logs themselves should be generated in as secure a manner as possible - remote syslog can be very useful. An intruder will try to cover his tracks, and log files are critical to the sysadmin trying to track down the time and method of the initial break-in. One way to keep a permanent record of the log files is to run the system console to a serial port and collect the information to a secure machine monitoring the consoles. === Paranoia A little paranoia never hurts. As a rule, a sysadmin can add any number of security features, as long as they do not affect convenience, and can add security features that _do_ affect convenience with some added thought. Even more importantly, a security administrator should mix it up a bit - if you use recommendations such as those given by this document verbatim, you give away your methodologies to the prospective attacker who also has access to this document. === Denial of Service Attacks This section covers Denial of Service attacks. A DoS attack is typically a packet attack. While there is not much you can do about modern spoofed packet attacks that saturate your network, you can generally limit the damage by ensuring that the attacks cannot take down your servers by: . Limiting server forks. . Limiting springboard attacks (ICMP response attacks, ping broadcast, etc.). . Overloading the Kernel Route Cache. A common DoS attack scenario is attacking a forking server and making it spawning so many child processes that the host system eventually runs out of memory, file descriptors, etc. and then grinds to a halt. inetd (see man:inetd[8]) has several options to limit this sort of attack. It should be noted that while it is possible to prevent a machine from going down, it is not generally possible to prevent a service from being disrupted by the attack. Read the inetd manual page carefully and pay specific attention to the `-c`, `-C`, and `-R` options. Note that spoofed-IP attacks will circumvent the `-C` option to inetd, so typically a combination of options must be used. Some standalone servers have self-fork-limitation parameters. Sendmail has its `-OMaxDaemonChildren` option, which tends to work much better than trying to use Sendmail's load limiting options due to the load lag. You should specify a `MaxDaemonChildren` parameter, when you start sendmail; high enough to handle your expected load, but not so high that the computer cannot handle that number of Sendmail instances without falling on its face. It is also prudent to run Sendmail in queued mode (`-ODeliveryMode=queued`) and to run the daemon (`sendmail -bd`) separate from the queue-runs (`sendmail -q15m`). If you still want real-time delivery you can run the queue at a much lower interval, such as `-q1m`, but be sure to specify a reasonable `MaxDaemonChildren` option for _that_ Sendmail to prevent cascade failures. Syslogd can be attacked directly and it is strongly recommended that you use the `-s` option whenever possible, and the `-a` option otherwise. You should also be fairly careful with connect-back services such as TCP Wrapper's reverse-identd, which can be attacked directly. You generally do not want to use the reverse-ident feature of TCP Wrapper for this reason. It is a very good idea to protect internal services from external access by firewalling them off at your border routers. The idea here is to prevent saturation attacks from outside your LAN, not so much to protect internal services from network-based `root` compromise. Always configure an exclusive firewall, i.e., "firewall everything _except_ ports A, B, C, D, and M-Z". This way you can firewall off all of your low ports except for certain specific services such as named (if you are primary for a zone), ntalkd, sendmail, and other Internet-accessible services. If you try to configure the firewall the other way - as an inclusive or permissive firewall, there is a good chance that you will forget to "close" a couple of services, or that you will add a new internal service and forget to update the firewall. You can still open up the high-numbered port range on the firewall, to allow permissive-like operation, without compromising your low ports. Also take note that FreeBSD allows you to control the range of port numbers used for dynamic binding, via the various ``net.inet.ip.portrange sysctl``'s (``sysctl -a | fgrep portrange``), which can also ease the complexity of your firewall's configuration. For example, you might use a normal first/last range of 4000 to 5000, and a hiport range of 49152 to 65535, then block off everything under 4000 in your firewall (except for certain specific Internet-accessible ports, of course). Another common DoS attack is called a springboard attack - to attack a server in a manner that causes the server to generate responses which overloads the server, the local network, or some other machine. The most common attack of this nature is the _ICMP ping broadcast attack_. The attacker spoofs ping packets sent to your LAN's broadcast address with the source IP address set to the actual machine they wish to attack. If your border routers are not configured to stomp on ping packets to broadcast addresses, your LAN winds up generating sufficient responses to the spoofed source address to saturate the victim, especially when the attacker uses the same trick on several dozen broadcast addresses over several dozen different networks at once. Broadcast attacks of over a hundred and twenty megabits have been measured. A second common springboard attack is against the ICMP error reporting system. By constructing packets that generate ICMP error responses, an attacker can saturate a server's incoming network and cause the server to saturate its outgoing network with ICMP responses. This type of attack can also crash the server by running it out of memory, especially if the server cannot drain the ICMP responses it generates fast enough. Use the sysctl variable `net.inet.icmp.icmplim` to limit these attacks. The last major class of springboard attacks is related to certain internal inetd services such as the udp echo service. An attacker simply spoofs a UDP packet with the source address being server A's echo port, and the destination address being server B's echo port, where server A and B are both on your LAN. The two servers then bounce this one packet back and forth between each other. The attacker can overload both servers and their LANs simply by injecting a few packets in this manner. Similar problems exist with the internal chargen port. A competent sysadmin will turn off all of these inetd-internal test services. Spoofed packet attacks may also be used to overload the kernel route cache. Refer to the `net.inet.ip.rtexpire`, `rtminexpire`, and `rtmaxcache sysctl` parameters. A spoofed packet attack that uses a random source IP will cause the kernel to generate a temporary cached route in the route table, viewable with `netstat -rna | fgrep W3`. These routes typically timeout in 1600 seconds or so. If the kernel detects that the cached route table has gotten too big it will dynamically reduce the `rtexpire` but will never decrease it to less than `rtminexpire`. There are two problems: . The kernel does not react quickly enough when a lightly loaded server is suddenly attacked. . The `rtminexpire` is not low enough for the kernel to survive a sustained attack. If your servers are connected to the Internet via a T3 or better, it may be prudent to manually override both `rtexpire` and `rtminexpire` via man:sysctl[8]. Never set either parameter to zero (unless you want to crash the machine). Setting both parameters to 2 seconds should be sufficient to protect the route table from attack. === Access Issues with Kerberos and SSH There are a few issues with both Kerberos and ssh that need to be addressed if you intend to use them. Kerberos 5 is an excellent authentication protocol, but there are bugs in the kerberized telnet and rlogin applications that make them unsuitable for dealing with binary streams. Also, by default Kerberos does not encrypt a session unless you use the `-x` option. ssh encrypts everything by default. Ssh works quite well in every respect except that it forwards encryption keys by default. What this means is that if you have a secure workstation holding keys that give you access to the rest of the system, and you ssh to an insecure machine, your keys are usable. The actual keys themselves are not exposed, but ssh installs a forwarding port for the duration of your login, and if an attacker has broken `root` on the insecure machine he can utilize that port to use your keys to gain access to any other machine that your keys unlock. We recommend that you use ssh in combination with Kerberos whenever possible for staff logins. Ssh can be compiled with Kerberos support. This reduces your reliance on potentially exposed ssh keys while at the same time protecting passwords via Kerberos. Ssh keys should only be used for automated tasks from secure machines (something that Kerberos is unsuited to do). We also recommend that you either turn off key-forwarding in the ssh configuration, or that you make use of the `from=IP/DOMAIN` option that ssh allows in its [.filename]#authorized_keys# file to make the key only usable to entities logging in from specific machines. [[crypt]] == DES, MD5, and Crypt Every user on a UNIX(R) system has a password associated with their account. It seems obvious that these passwords need to be known only to the user and the actual operating system. In order to keep these passwords secret, they are encrypted with what is known as a "one-way hash", that is, they can only be easily encrypted but not decrypted. In other words, what we told you a moment ago was obvious is not even true: the operating system itself does not _really_ know the password. It only knows the _encrypted_ form of the password. The only way to get the "plain-text" password is by a brute force search of the space of possible passwords. Unfortunately the only secure way to encrypt passwords when UNIX(R) came into being was based on DES, the Data Encryption Standard. This was not such a problem for users resident in the US, but since the source code for DES could not be exported outside the US, FreeBSD had to find a way to both comply with US law and retain compatibility with all the other UNIX(R) variants that still used DES. The solution was to divide up the encryption libraries so that US users could install the DES libraries and use DES but international users still had an encryption method that could be exported abroad. This is how FreeBSD came to use MD5 as its default encryption method. MD5 is believed to be more secure than DES, so installing DES is offered primarily for compatibility reasons. === Recognizing Your Crypt Mechanism Currently the library supports DES, MD5 and Blowfish hash functions. By default FreeBSD uses MD5 to encrypt passwords. It is pretty easy to identify which encryption method FreeBSD is set up to use. Examining the encrypted passwords in the [.filename]#/etc/master.passwd# file is one way. Passwords encrypted with the MD5 hash are longer than those encrypted with the DES hash and also begin with the characters `$1$`. Passwords starting with `$2a$` are encrypted with the Blowfish hash function. DES password strings do not have any particular identifying characteristics, but they are shorter than MD5 passwords, and are coded in a 64-character alphabet which does not include the `$` character, so a relatively short string which does not begin with a dollar sign is very likely a DES password. The password format used for new passwords is controlled by the `passwd_format` login capability in [.filename]#/etc/login.conf#, which takes values of `des`, `md5` or `blf`. See the man:login.conf[5] manual page for more information about login capabilities. [[one-time-passwords]] == One-time Passwords By default, FreeBSD includes support for OPIE (One-time Passwords In Everything), which uses the MD5 hash by default. There are three different sorts of passwords which we will discuss below. The first is your usual UNIX(R) style or Kerberos password; we will call this a "UNIX(R) password". The second sort is the one-time password which is generated by the OPIE man:opiekey[1] program and accepted by the man:opiepasswd[1] program and the login prompt; we will call this a "one-time password". The final sort of password is the secret password which you give to the `opiekey` program (and sometimes the `opiepasswd` programs) which it uses to generate one-time passwords; we will call it a "secret password" or just unqualified "password". The secret password does not have anything to do with your UNIX(R) password; they can be the same but this is not recommended. OPIE secret passwords are not limited to 8 characters like old UNIX(R) passwords, they can be as long as you like. Passwords of six or seven word long phrases are fairly common. For the most part, the OPIE system operates completely independently of the UNIX(R) password system. Besides the password, there are two other pieces of data that are important to OPIE. One is what is known as the "seed" or "key", consisting of two letters and five digits. The other is what is called the "iteration count", a number between 1 and 100. OPIE creates the one-time password by concatenating the seed and the secret password, then applying the MD5 hash as many times as specified by the iteration count and turning the result into six short English words. These six English words are your one-time password. The authentication system (primarily PAM) keeps track of the last one-time password used, and the user is authenticated if the hash of the user-provided password is equal to the previous password. Because a one-way hash is used it is impossible to generate future one-time passwords if a successfully used password is captured; the iteration count is decremented after each successful login to keep the user and the login program in sync. When the iteration count gets down to 1, OPIE must be reinitialized. There are a few programs involved in each system which we will discuss below. The `opiekey` program accepts an iteration count, a seed, and a secret password, and generates a one-time password or a consecutive list of one-time passwords. The `opiepasswd` program is used to initialize OPIE, and to change passwords, iteration counts, or seeds; it takes either a secret passphrase, or an iteration count, seed, and a one-time password. The `opieinfo` program will examine the relevant credentials files ([.filename]#/etc/opiekeys#) and print out the invoking user's current iteration count and seed. There are four different sorts of operations we will cover. The first is using `opiepasswd` over a secure connection to set up one-time-passwords for the first time, or to change your password or seed. The second operation is using `opiepasswd` over an insecure connection, in conjunction with `opiekey` over a secure connection, to do the same. The third is using `opiekey` to log in over an insecure connection. The fourth is using `opiekey` to generate a number of keys which can be written down or printed out to carry with you when going to some location without secure connections to anywhere. === Secure Connection Initialization To initialize OPIE for the first time, execute the `opiepasswd` command: [source,shell] .... % opiepasswd -c [grimreaper] ~ $ opiepasswd -f -c Adding unfurl: Only use this method from the console; NEVER from remote. If you are using telnet, xterm, or a dial-in, type ^C now or exit with no password. Then run opiepasswd without the -c parameter. Using MD5 to compute responses. Enter new secret pass phrase: Again new secret pass phrase: ID unfurl OTP key is 499 to4268 MOS MALL GOAT ARM AVID COED .... At the `Enter new secret pass phrase:` or `Enter secret password:` prompts, you should enter a password or phrase. Remember, this is not the password that you will use to login with, this is used to generate your one-time login keys. The "ID" line gives the parameters of your particular instance: your login name, the iteration count, and seed. When logging in the system will remember these parameters and present them back to you so you do not have to remember them. The last line gives the particular one-time password which corresponds to those parameters and your secret password; if you were to re-login immediately, this one-time password is the one you would use. === Insecure Connection Initialization To initialize or change your secret password over an insecure connection, you will need to already have a secure connection to some place where you can run `opiekey`; this might be in the form of a shell prompt on a machine you trust. You will also need to make up an iteration count (100 is probably a good value), and you may make up your own seed or use a randomly-generated one. Over on the insecure connection (to the machine you are initializing), use `opiepasswd`: [source,shell] .... % opiepasswd Updating unfurl: You need the response from an OTP generator. Old secret pass phrase: otp-md5 498 to4268 ext Response: GAME GAG WELT OUT DOWN CHAT New secret pass phrase: otp-md5 499 to4269 Response: LINE PAP MILK NELL BUOY TROY ID mark OTP key is 499 gr4269 LINE PAP MILK NELL BUOY TROY .... To accept the default seed press kbd:[Return]. Then before entering an access password, move over to your secure connection and give it the same parameters: [source,shell] .... % opiekey 498 to4268 Using the MD5 algorithm to compute response. Reminder: Don't use opiekey from telnet or dial-in sessions. Enter secret pass phrase: GAME GAG WELT OUT DOWN CHAT .... Now switch back over to the insecure connection, and copy the one-time password generated over to the relevant program. === Generating a Single One-time Password Once you have initialized OPIE and login, you will be presented with a prompt like this: [source,shell] .... % telnet example.com Trying 10.0.0.1... Connected to example.com Escape character is '^]'. FreeBSD/i386 (example.com) (ttypa) login: otp-md5 498 gr4269 ext Password: .... As a side note, the OPIE prompts have a useful feature (not shown here): if you press kbd:[Return] at the password prompt, the prompter will turn echo on, so you can see what you are typing. This can be extremely useful if you are attempting to type in a password by hand, such as from a printout. At this point you need to generate your one-time password to answer this login prompt. This must be done on a trusted system that you can run `opiekey` on. (There are versions of these for DOS, Windows(R) and Mac OS(R) as well.) They need the iteration count and the seed as command line options. You can cut-and-paste these right from the login prompt on the machine that you are logging in to. On the trusted system: [source,shell] .... % opiekey 498 to4268 Using the MD5 algorithm to compute response. Reminder: Don't use opiekey from telnet or dial-in sessions. Enter secret pass phrase: GAME GAG WELT OUT DOWN CHAT .... Now that you have your one-time password you can continue logging in. === Generating Multiple One-time Passwords Sometimes you have to go places where you do not have access to a trusted machine or secure connection. In this case, it is possible to use the `opiekey` command to generate a number of one-time passwords beforehand to be printed out and taken with you. For example: [source,shell] .... % opiekey -n 5 30 zz99999 Using the MD5 algorithm to compute response. Reminder: Don't use opiekey from telnet or dial-in sessions. Enter secret pass phrase: 26: JOAN BORE FOSS DES NAY QUIT 27: LATE BIAS SLAY FOLK MUCH TRIG 28: SALT TIN ANTI LOON NEAL USE 29: RIO ODIN GO BYE FURY TIC 30: GREW JIVE SAN GIRD BOIL PHI .... The `-n 5` requests five keys in sequence, the `30` specifies what the last iteration number should be. Note that these are printed out in _reverse_ order of eventual use. If you are really paranoid, you might want to write the results down by hand; otherwise you can cut-and-paste into `lpr`. Note that each line shows both the iteration count and the one-time password; you may still find it handy to scratch off passwords as you use them. === Restricting Use of UNIX(R) Passwords OPIE can restrict the use of UNIX(R) passwords based on the IP address of a login session. The relevant file is [.filename]#/etc/opieaccess#, which is present by default. Please check man:opieaccess[5] for more information on this file and which security considerations you should be aware of when using it. Here is a sample [.filename]#opieaccess# file: [.programlisting] .... permit 192.168.0.0 255.255.0.0 .... This line allows users whose IP source address (which is vulnerable to spoofing) matches the specified value and mask, to use UNIX(R) passwords at any time. If no rules in [.filename]#opieaccess# are matched, the default is to deny non-OPIE logins. [[tcpwrappers]] == TCP Wrappers Anyone familiar with man:inetd[8] has probably heard of TCP Wrappers at some point. But few individuals seem to fully comprehend its usefulness in a network environment. It seems that everyone wants to install a firewall to handle network connections. While a firewall has a wide variety of uses, there are some things that a firewall not handle such as sending text back to the connection originator. The TCP software does this and much more. In the next few sections many of the TCP Wrappers features will be discussed, and, when applicable, example configuration lines will be provided. The TCP Wrappers software extends the abilities of `inetd` to provide support for every server daemon under its control. Using this method it is possible to provide logging support, return messages to connections, permit a daemon to only accept internal connections, etc. While some of these features can be provided by implementing a firewall, this will add not only an extra layer of protection but go beyond the amount of control a firewall can provide. The added functionality of TCP Wrappers should not be considered a replacement for a good firewall. TCP Wrappers can be used in conjunction with a firewall or other security enhancements though and it can serve nicely as an extra layer of protection for the system. Since this is an extension to the configuration of `inetd`, the reader is expected have read the crossref:network-servers[network-inetd,inetd configuration] section. [NOTE] ==== While programs run by man:inetd[8] are not exactly "daemons", they have traditionally been called daemons. This is the term we will use in this section too. ==== === Initial Configuration The only requirement of using TCP Wrappers in FreeBSD is to ensure the `inetd` server is started from [.filename]#rc.conf# with the `-Ww` option; this is the default setting. Of course, proper configuration of [.filename]#/etc/hosts.allow# is also expected, but man:syslogd[8] will throw messages in the system logs in these cases. [NOTE] ==== Unlike other implementations of TCP Wrappers, the use of [.filename]#hosts.deny# has been deprecated. All configuration options should be placed in [.filename]#/etc/hosts.allow#. ==== In the simplest configuration, daemon connection policies are set to either be permitted or blocked depending on the options in [.filename]#/etc/hosts.allow#. The default configuration in FreeBSD is to allow a connection to every daemon started with `inetd`. Changing this will be discussed only after the basic configuration is covered. Basic configuration usually takes the form of `daemon : address : action`. Where `daemon` is the daemon name which `inetd` started. The `address` can be a valid hostname, an IP address or an IPv6 address enclosed in brackets ([ ]). The action field can be either allow or deny to grant or deny access appropriately. Keep in mind that configuration works off a first rule match semantic, meaning that the configuration file is scanned in ascending order for a matching rule. When a match is found the rule is applied and the search process will halt. Several other options exist but they will be explained in a later section. A simple configuration line may easily be constructed from that information alone. For example, to allow POP3 connections via the package:mail/qpopper[] daemon, the following lines should be appended to [.filename]#hosts.allow#: [.programlisting] .... # This line is required for POP3 connections: qpopper : ALL : allow .... After adding this line, `inetd` will need restarted. This can be accomplished by use of the man:kill[1] command, or with the [parameter]#restart# parameter with [.filename]#/etc/rc.d/inetd#. === Advanced Configuration TCP Wrappers has advanced options too; they will allow for more control over the way connections are handled. In some cases it may be a good idea to return a comment to certain hosts or daemon connections. In other cases, perhaps a log file should be recorded or an email sent to the administrator. Other situations may require the use of a service for local connections only. This is all possible through the use of configuration options known as `wildcards`, expansion characters and external command execution. The next two sections are written to cover these situations. ==== External Commands Suppose that a situation occurs where a connection should be denied yet a reason should be sent to the individual who attempted to establish that connection. How could it be done? That action can be made possible by using the `twist` option. When a connection attempt is made, `twist` will be called to execute a shell command or script. An example already exists in the [.filename]#hosts.allow# file: [.programlisting] .... # The rest of the daemons are protected. ALL : ALL \ : severity auth.info \ : twist /bin/echo "You are not welcome to use %d from %h." .... This example shows that the message, "You are not allowed to use `daemon` from `hostname`." will be returned for any daemon not previously configured in the access file. This is extremely useful for sending a reply back to the connection initiator right after the established connection is dropped. Note that any message returned _must_ be wrapped in quote `"` characters; there are no exceptions to this rule. [WARNING] ==== It may be possible to launch a denial of service attack on the server if an attacker, or group of attackers could flood these daemons with connection requests. ==== Another possibility is to use the `spawn` option in these cases. Like `twist`, the `spawn` implicitly denies the connection and may be used to run external shell commands or scripts. Unlike `twist`, `spawn` will not send a reply back to the individual who established the connection. For an example, consider the following configuration line: [.programlisting] .... # We do not allow connections from example.com: ALL : .example.com \ : spawn (/bin/echo %a from %h attempted to access %d >> \ /var/log/connections.log) \ : deny .... This will deny all connection attempts from the `*.example.com` domain; simultaneously logging the hostname, IP address and the daemon which they attempted to access in the [.filename]#/var/log/connections.log# file. Aside from the already explained substitution characters above, e.g. %a, a few others exist. See the man:hosts_access[5] manual page for the complete list. ==== Wildcard Options Thus far the `ALL` example has been used continuously throughout the examples. Other options exist which could extend the functionality a bit further. For instance, `ALL` may be used to match every instance of either a daemon, domain or an IP address. Another wildcard available is `PARANOID` which may be used to match any host which provides an IP address that may be forged. In other words, `paranoid` may be used to define an action to be taken whenever a connection is made from an IP address that differs from its hostname. The following example may shed some more light on this discussion: [.programlisting] .... # Block possibly spoofed requests to sendmail: sendmail : PARANOID : deny .... In that example all connection requests to `sendmail` which have an IP address that varies from its hostname will be denied. [CAUTION] ==== Using the `PARANOID` may severely cripple servers if the client or server has a broken DNS setup. Administrator discretion is advised. ==== To learn more about wildcards and their associated functionality, see the man:hosts_access[5] manual page. Before any of the specific configuration lines above will work, the first configuration line should be commented out in [.filename]#hosts.allow#. This was noted at the beginning of this section. [[kerberosIV]] == KerberosIV Kerberos is a network add-on system/protocol that allows users to authenticate themselves through the services of a secure server. Services such as remote login, remote copy, secure inter-system file copying and other high-risk tasks are made considerably safer and more controllable. The following instructions can be used as a guide on how to set up Kerberos as distributed for FreeBSD. However, you should refer to the relevant manual pages for a complete description. === Installing KerberosIV Kerberos is an optional component of FreeBSD. The easiest way to install this software is by selecting the `krb4` or `krb5` distribution in sysinstall during the initial installation of FreeBSD. This will install the "eBones" (KerberosIV) or "Heimdal" (Kerberos5) implementation of Kerberos. These implementations are included because they are developed outside the USA/Canada and were thus available to system owners outside those countries during the era of restrictive export controls on cryptographic code from the USA. Alternatively, the MIT implementation of Kerberos is available from the Ports Collection as package:security/krb5[]. === Creating the Initial Database This is done on the Kerberos server only. First make sure that you do not have any old Kerberos databases around. You should change to the directory [.filename]#/etc/kerberosIV# and check that only the following files are present: [source,shell] .... d /etc/kerberosIV # ls README krb.conf krb.realms .... If any additional files (such as [.filename]#principal.*# or [.filename]#master_key#) exist, then use the `kdb_destroy` command to destroy the old Kerberos database, or if Kerberos is not running, simply delete the extra files. You should now edit the [.filename]#krb.conf# and [.filename]#krb.realms# files to define your Kerberos realm. In this case the realm will be `EXAMPLE.COM` and the server is `grunt.example.com`. We edit or create the [.filename]#krb.conf# file: [source,shell] .... # cat krb.conf EXAMPLE.COM EXAMPLE.COM grunt.example.com admin server CS.BERKELEY.EDU okeeffe.berkeley.edu ATHENA.MIT.EDU kerberos.mit.edu ATHENA.MIT.EDU kerberos-1.mit.edu ATHENA.MIT.EDU kerberos-2.mit.edu ATHENA.MIT.EDU kerberos-3.mit.edu LCS.MIT.EDU kerberos.lcs.mit.edu TELECOM.MIT.EDU bitsy.mit.edu ARC.NASA.GOV trident.arc.nasa.gov .... In this case, the other realms do not need to be there. They are here as an example of how a machine may be made aware of multiple realms. You may wish to not include them for simplicity. The first line names the realm in which this system works. The other lines contain realm/host entries. The first item on a line is a realm, and the second is a host in that realm that is acting as a "key distribution center". The words `admin server` following a host's name means that host also provides an administrative database server. For further explanation of these terms, please consult the Kerberos manual pages. Now we have to add `grunt.example.com` to the `EXAMPLE.COM` realm and also add an entry to put all hosts in the `.example.com` domain in the `EXAMPLE.COM` realm. The [.filename]#krb.realms# file would be updated as follows: [source,shell] .... # cat krb.realms grunt.example.com EXAMPLE.COM .example.com EXAMPLE.COM .berkeley.edu CS.BERKELEY.EDU .MIT.EDU ATHENA.MIT.EDU .mit.edu ATHENA.MIT.EDU .... Again, the other realms do not need to be there. They are here as an example of how a machine may be made aware of multiple realms. You may wish to remove them to simplify things. The first line puts the _specific_ system into the named realm. The rest of the lines show how to default systems of a particular subdomain to a named realm. Now we are ready to create the database. This only needs to run on the Kerberos server (or Key Distribution Center). Issue the `kdb_init` command to do this: [source,shell] .... # kdb_init Realm name [default ATHENA.MIT.EDU ]: EXAMPLE.COM You will be prompted for the database Master Password. It is important that you NOT FORGET this password. Enter Kerberos master key: .... Now we have to save the key so that servers on the local machine can pick it up. Use the `kstash` command to do this: [source,shell] .... # kstash Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! .... This saves the encrypted master password in [.filename]#/etc/kerberosIV/master_key#. === Making It All Run Two principals need to be added to the database for _each_ system that will be secured with Kerberos. Their names are `kpasswd` and `rcmd`. These two principals are made for each system, with the instance being the name of the individual system. These daemons, kpasswd and rcmd allow other systems to change Kerberos passwords and run commands like man:rcp[1], man:rlogin[1] and man:rsh[1]. Now let us add these entries: [source,shell] .... # kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: passwd Instance: grunt , Create [y] ? y Principal: passwd, Instance: grunt, kdc_key_ver: 1 New Password: <---- enter RANDOM here Verifying password New Password: <---- enter RANDOM here Random password [y] ? y Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: rcmd Instance: grunt , Create [y] ? Principal: rcmd, Instance: grunt, kdc_key_ver: 1 New Password: <---- enter RANDOM here Verifying password New Password: <---- enter RANDOM here Random password [y] ? Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit .... === Creating the Server File We now have to extract all the instances which define the services on each machine. For this we use the `ext_srvtab` command. This will create a file which must be copied or moved _by secure means_ to each Kerberos client's [.filename]#/etc# directory. This file must be present on each server and client, and is crucial to the operation of Kerberos. [source,shell] .... # ext_srvtab grunt Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Generating 'grunt-new-srvtab'.... .... Now, this command only generates a temporary file which must be renamed to [.filename]#srvtab# so that all the servers can pick it up. Use the man:mv[1] command to move it into place on the original system: [source,shell] .... # mv grunt-new-srvtab srvtab .... If the file is for a client system, and the network is not deemed safe, then copy the [.filename]#client-new-srvtab# to removable media and transport it by secure physical means. Be sure to rename it to [.filename]#srvtab# in the client's [.filename]#/etc# directory, and make sure it is mode 600: [source,shell] .... # mv grumble-new-srvtab srvtab # chmod 600 srvtab .... === Populating the Database We now have to add some user entries into the database. First let us create an entry for the user `jane`. Use the `kdb_edit` command to do this: [source,shell] .... # kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: jane Instance: , Create [y] ? y Principal: jane, Instance: , kdc_key_ver: 1 New Password: <---- enter a secure password here Verifying password New Password: <---- re-enter the password here Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit .... === Testing It All Out First we have to start the Kerberos daemons. Note that if you have correctly edited your [.filename]#/etc/rc.conf# then this will happen automatically when you reboot. This is only necessary on the Kerberos server. Kerberos clients will automatically get what they need from the [.filename]#/etc/kerberosIV# directory. [source,shell] .... # kerberos & Kerberos server starting Sleep forever on error Log file is /var/log/kerberos.log Current Kerberos master key version is 1. Master key entered. BEWARE! Current Kerberos master key version is 1 Local realm: EXAMPLE.COM # kadmind -n & KADM Server KADM0.0A initializing Please do not use 'kill -9' to kill this job, use a regular kill instead Current Kerberos master key version is 1. Master key entered. BEWARE! .... Now we can try using the `kinit` command to get a ticket for the ID `jane` that we created above: [source,shell] .... % kinit jane MIT Project Athena (grunt.example.com) Kerberos Initialization for "jane" Password: .... Try listing the tokens using `klist` to see if we really have them: [source,shell] .... % klist Ticket file: /tmp/tkt245 Principal: jane@EXAMPLE.COM Issued Expires Principal Apr 30 11:23:22 Apr 30 19:23:22 krbtgt.EXAMPLE.COM@EXAMPLE.COM .... Now try changing the password using man:passwd[1] to check if the kpasswd daemon can get authorization to the Kerberos database: [source,shell] .... % passwd realm EXAMPLE.COM Old password for jane: New Password for jane: Verifying password New Password for jane: Password changed. .... === Adding `su` Privileges Kerberos allows us to give _each_ user who needs `root` privileges their own _separate_ man:su[1] password. We could now add an ID which is authorized to man:su[1] to `root`. This is controlled by having an instance of `root` associated with a principal. Using `kdb_edit` we can create the entry `jane.root` in the Kerberos database: [source,shell] .... # kdb_edit Opening database... Enter Kerberos master key: Current Kerberos master key version is 1. Master key entered. BEWARE! Previous or default values are in [brackets] , enter return to leave the same, or new value. Principal name: jane Instance: root , Create [y] ? y Principal: jane, Instance: root, kdc_key_ver: 1 New Password: <---- enter a SECURE password here Verifying password New Password: <---- re-enter the password here Principal's new key version = 1 Expiration date (enter yyyy-mm-dd) [ 2000-01-01 ] ? Max ticket lifetime (*5 minutes) [ 255 ] ? 12 <--- Keep this short! Attributes [ 0 ] ? Edit O.K. Principal name: <---- null entry here will cause an exit .... Now try getting tokens for it to make sure it works: [source,shell] .... # kinit jane.root MIT Project Athena (grunt.example.com) Kerberos Initialization for "jane.root" Password: .... Now we need to add the user to ``root``'s [.filename]#.klogin# file: [source,shell] .... # cat /root/.klogin jane.root@EXAMPLE.COM .... Now try doing the man:su[1]: [source,shell] .... % su Password: .... and take a look at what tokens we have: [source,shell] .... # klist Ticket file: /tmp/tkt_root_245 Principal: jane.root@EXAMPLE.COM Issued Expires Principal May 2 20:43:12 May 3 04:43:12 krbtgt.EXAMPLE.COM@EXAMPLE.COM .... === Using Other Commands In an earlier example, we created a principal called `jane` with an instance `root`. This was based on a user with the same name as the principal, and this is a Kerberos default; that a `.` of the form `. root` will allow that `` to man:su[1] to `root` if the necessary entries are in the [.filename]#.klogin# file in ``root``'s home directory: [source,shell] .... # cat /root/.klogin jane.root@EXAMPLE.COM .... Likewise, if a user has in their own home directory lines of the form: [source,shell] .... % cat ~/.klogin jane@EXAMPLE.COM jack@EXAMPLE.COM .... This allows anyone in the `EXAMPLE.COM` realm who has authenticated themselves as `jane` or `jack` (via `kinit`, see above) to access to ``jane``'s account or files on this system (`grunt`) via man:rlogin[1], man:rsh[1] or man:rcp[1]. For example, `jane` now logs into another system using Kerberos: [source,shell] .... % kinit MIT Project Athena (grunt.example.com) Password: % rlogin grunt Last login: Mon May 1 21:14:47 from grumble Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD BUILT-19950429 (GR386) #0: Sat Apr 29 17:50:09 SAT 1995 .... Or `jack` logs into ``jane``'s account on the same machine (`jane` having set up the [.filename]#.klogin# file as above, and the person in charge of Kerberos having set up principal _jack_ with a null instance): [source,shell] .... % kinit % rlogin grunt -l jane MIT Project Athena (grunt.example.com) Password: Last login: Mon May 1 21:16:55 from grumble Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD BUILT-19950429 (GR386) #0: Sat Apr 29 17:50:09 SAT 1995 .... [[kerberos5]] == Kerberos5 Every FreeBSD release beyond FreeBSD-5.1 includes support only for Kerberos5. Hence Kerberos5 is the only version included, and its configuration is similar in many aspects to that of KerberosIV. The following information only applies to Kerberos5 in post FreeBSD-5.0 releases. Users who wish to use the KerberosIV package may install the package:security/krb4[] port. Kerberos is a network add-on system/protocol that allows users to authenticate themselves through the services of a secure server. Services such as remote login, remote copy, secure inter-system file copying and other high-risk tasks are made considerably safer and more controllable. Kerberos can be described as an identity-verifying proxy system. It can also be described as a trusted third-party authentication system. Kerberos provides only one function - the secure authentication of users on the network. It does not provide authorization functions (what users are allowed to do) or auditing functions (what those users did). After a client and server have used Kerberos to prove their identity, they can also encrypt all of their communications to assure privacy and data integrity as they go about their business. Therefore it is highly recommended that Kerberos be used with other security methods which provide authorization and audit services. The following instructions can be used as a guide on how to set up Kerberos as distributed for FreeBSD. However, you should refer to the relevant manual pages for a complete description. For purposes of demonstrating a Kerberos installation, the various name spaces will be handled as follows: * The DNS domain ("zone") will be example.org. * The Kerberos realm will be EXAMPLE.ORG. [NOTE] ==== Please use real domain names when setting up Kerberos even if you intend to run it internally. This avoids DNS problems and assures inter-operation with other Kerberos realms. ==== === History Kerberos was created by MIT as a solution to network security problems. The Kerberos protocol uses strong cryptography so that a client can prove its identity to a server (and vice versa) across an insecure network connection. Kerberos is both the name of a network authentication protocol and an adjective to describe programs that implement the program (Kerberos telnet, for example). The current version of the protocol is version 5, described in RFC 1510. Several free implementations of this protocol are available, covering a wide range of operating systems. The Massachusetts Institute of Technology (MIT), where Kerberos was originally developed, continues to develop their Kerberos package. It is commonly used in the US as a cryptography product, as such it has historically been affected by US export regulations. The MITKerberos is available as a port (package:security/krb5[]). Heimdal Kerberos is another version 5 implementation, and was explicitly developed outside of the US to avoid export regulations (and is thus often included in non-commercial UNIX(R) variants). The Heimdal Kerberos distribution is available as a port (package:security/heimdal[]), and a minimal installation of it is included in the base FreeBSD install. In order to reach the widest audience, these instructions assume the use of the Heimdal distribution included in FreeBSD. === Setting up a Heimdal KDC The Key Distribution Center (KDC) is the centralized authentication service that Kerberos provides - it is the computer that issues Kerberos tickets. The KDC is considered "trusted" by all other computers in the Kerberos realm, and thus has heightened security concerns. Note that while running the Kerberos server requires very few computing resources, a dedicated machine acting only as a KDC is recommended for security reasons. To begin setting up a KDC, ensure that your [.filename]#/etc/rc.conf# file contains the correct settings to act as a KDC (you may need to adjust paths to reflect your own system): [.programlisting] .... kerberos5_server_enable="YES" kadmind5_server_enable="YES" .... Next we will set up your Kerberos config file, [.filename]#/etc/krb5.conf#: [.programlisting] .... [libdefaults] default_realm = EXAMPLE.ORG [realms] EXAMPLE.ORG = { kdc = kerberos.example.org admin_server = kerberos.example.org } [domain_realm] .example.org = EXAMPLE.ORG .... Note that this [.filename]#/etc/krb5.conf# file implies that your KDC will have the fully-qualified hostname of `kerberos.example.org`. You will need to add a CNAME (alias) entry to your zone file to accomplish this if your KDC has a different hostname. [NOTE] ==== For large networks with a properly configured BINDDNS server, the above example could be trimmed to: [.programlisting] .... [libdefaults] default_realm = EXAMPLE.ORG .... With the following lines being appended to the `example.org` zonefile: [.programlisting] .... _kerberos._udp IN SRV 01 00 88 kerberos.example.org. _kerberos._tcp IN SRV 01 00 88 kerberos.example.org. _kpasswd._udp IN SRV 01 00 464 kerberos.example.org. _kerberos-adm._tcp IN SRV 01 00 749 kerberos.example.org. _kerberos IN TXT EXAMPLE.ORG .... ==== [NOTE] ==== For clients to be able to find the Kerberos services, you _must_ have either a fully configured [.filename]#/etc/krb5.conf# or a minimally configured [.filename]#/etc/krb5.conf#_and_ a properly configured DNS server. ==== Next we will create the Kerberos database. This database contains the keys of all principals encrypted with a master password. You are not required to remember this password, it will be stored in a file ([.filename]#/var/heimdal/m-key#). To create the master key, run `kstash` and enter a password. Once the master key has been created, you can initialize the database using the `kadmin` program with the `-l` option (standing for "local"). This option instructs `kadmin` to modify the database files directly rather than going through the `kadmind` network service. This handles the chicken-and-egg problem of trying to connect to the database before it is created. Once you have the `kadmin` prompt, use the `init` command to create your realms initial database. Lastly, while still in `kadmin`, create your first principal using the `add` command. Stick to the defaults options for the principal for now, you can always change them later with the `modify` command. Note that you can use the `?` command at any prompt to see the available options. A sample database creation session is shown below: [source,shell] .... # kstash Master key: xxxxxxxx Verifying password - Master key: xxxxxxxx # kadmin -l kadmin> init EXAMPLE.ORG Realm max ticket life [unlimited]: kadmin> add tillman Max ticket life [unlimited]: Max renewable life [unlimited]: Attributes []: Password: xxxxxxxx Verifying password - Password: xxxxxxxx .... Now it is time to start up the KDC services. Run `/etc/rc.d/kerberos start` and `/etc/rc.d/kadmind start` to bring up the services. Note that you will not have any kerberized daemons running at this point but you should be able to confirm the that the KDC is functioning by obtaining and listing a ticket for the principal (user) that you just created from the command-line of the KDC itself: [source,shell] .... % kinit tillman tillman@EXAMPLE.ORG's Password: % klist Credentials cache: FILE:/tmp/krb5cc_500 Principal: tillman@EXAMPLE.ORG Issued Expires Principal Aug 27 15:37:58 Aug 28 01:37:58 krbtgt/EXAMPLE.ORG@EXAMPLE.ORG .... The ticket can then be revoked when you have finished: [source,shell] .... % k5destroy .... === Kerberos enabling a server with Heimdal services First, we need a copy of the Kerberos configuration file, [.filename]#/etc/krb5.conf#. To do so, simply copy it over to the client computer from the KDC in a secure fashion (using network utilities, such as man:scp[1], or physically via a floppy disk). Next you need a [.filename]#/etc/krb5.keytab# file. This is the major difference between a server providing Kerberos enabled daemons and a workstation - the server must have a [.filename]#keytab# file. This file contains the server's host key, which allows it and the KDC to verify each others identity. It must be transmitted to the server in a secure fashion, as the security of the server can be broken if the key is made public. This explicitly means that transferring it via a clear text channel, such as FTP, is a very bad idea. Typically, you transfer to the [.filename]#keytab# to the server using the `kadmin` program. This is handy because you also need to create the host principal (the KDC end of the [.filename]#krb5.keytab#) using `kadmin`. Note that you must have already obtained a ticket and that this ticket must be allowed to use the `kadmin` interface in the [.filename]#kadmind.acl#. See the section titled "Remote administration" in the Heimdal info pages (`info heimdal`) for details on designing access control lists. If you do not want to enable remote `kadmin` access, you can simply securely connect to the KDC (via local console, man:ssh[1] or Kerberos man:telnet[1]) and perform administration locally using `kadmin -l`. After installing the [.filename]#/etc/krb5.conf# file, you can use `kadmin` from the Kerberos server. The `add --random-key` command will let you add the server's host principal, and the `ext` command will allow you to extract the server's host principal to its own keytab. For example: [source,shell] .... # kadmin kadmin> add --random-key host/myserver.example.org Max ticket life [unlimited]: Max renewable life [unlimited]: Attributes []: kadmin> ext host/myserver.example.org kadmin> exit .... Note that the `ext` command (short for "extract") stores the extracted key in [.filename]#/etc/krb5.keytab# by default. If you do not have `kadmind` running on the KDC (possibly for security reasons) and thus do not have access to `kadmin` remotely, you can add the host principal (`host/myserver.EXAMPLE.ORG`) directly on the KDC and then extract it to a temporary file (to avoid over-writing the [.filename]#/etc/krb5.keytab# on the KDC) using something like this: [source,shell] .... # kadmin kadmin> ext --keytab=/tmp/example.keytab host/myserver.example.org kadmin> exit .... You can then securely copy the keytab to the server computer (using `scp` or a floppy, for example). Be sure to specify a non-default keytab name to avoid over-writing the keytab on the KDC. At this point your server can communicate with the KDC (due to its [.filename]#krb5.conf# file) and it can prove its own identity (due to the [.filename]#krb5.keytab# file). It is now ready for you to enable some Kerberos services. For this example we will enable the `telnet` service by putting a line like this into your [.filename]#/etc/inetd.conf# and then restarting the man:inetd[8] service with `/etc/rc.d/inetd restart`: [.programlisting] .... telnet stream tcp nowait root /usr/libexec/telnetd telnetd -a user .... The critical bit is that the `-a` (for authentication) type is set to user. Consult the man:telnetd[8] manual page for more details. === Kerberos enabling a client with Heimdal Setting up a client computer is almost trivially easy. As far as Kerberos configuration goes, you only need the Kerberos configuration file, located at [.filename]#/etc/krb5.conf#. Simply securely copy it over to the client computer from the KDC. Test your client computer by attempting to use `kinit`, `klist`, and `kdestroy` from the client to obtain, show, and then delete a ticket for the principal you created above. You should also be able to use Kerberos applications to connect to Kerberos enabled servers, though if that does not work and obtaining a ticket does the problem is likely with the server and not with the client or the KDC. When testing an application like `telnet`, try using a packet sniffer (such as man:tcpdump[1]) to confirm that your password is not sent in the clear. Try using `telnet` with the `-x` option, which encrypts the entire data stream (similar to `ssh`). Various non-core Kerberos client applications are also installed by default. This is where the "minimal" nature of the base Heimdal installation is felt: `telnet` is the only Kerberos enabled service. The Heimdal port adds some of the missing client applications: Kerberos enabled versions of `ftp`, `rsh`, `rcp`, `rlogin`, and a few other less common programs. The MIT port also contains a full suite of Kerberos client applications. === User configuration files: [.filename]#.k5login# and [.filename]#.k5users# Users within a realm typically have their Kerberos principal (such as `tillman@EXAMPLE.ORG`) mapped to a local user account (such as a local account named `tillman`). Client applications such as `telnet` usually do not require a user name or a principal. Occasionally, however, you want to grant access to a local user account to someone who does not have a matching Kerberos principal. For example, `tillman@EXAMPLE.ORG` may need access to the local user account `webdevelopers`. Other principals may also need access to that local account. The [.filename]#.k5login# and [.filename]#.k5users# files, placed in a users home directory, can be used similar to a powerful combination of [.filename]#.hosts# and [.filename]#.rhosts#, solving this problem. For example, if a [.filename]#.k5login# with the following contents: [source,shell] .... tillman@example.org jdoe@example.org .... Were to be placed into the home directory of the local user `webdevelopers` then both principals listed would have access to that account without requiring a shared password. Reading the manual pages for these commands is recommended. Note that the `ksu` manual page covers [.filename]#.k5users#. === Kerberos Tips, Tricks, and Troubleshooting * When using either the Heimdal or MITKerberos ports ensure that your `PATH` environment variable lists the Kerberos versions of the client applications before the system versions. * Do all the computers in your realm have synchronized time settings? If not, authentication may fail. crossref:network-servers[network-ntp,Συγχρονισμός Ρολογιού Συστήματος με NTP] describes how to synchronize clocks using NTP. * MIT and Heimdal inter-operate nicely. Except for `kadmin`, the protocol for which is not standardized. * If you change your hostname, you also need to change your `host/` principal and update your keytab. This also applies to special keytab entries like the `www/` principal used for Apache's package:www/mod_auth_kerb[]. * All hosts in your realm must be resolvable (both forwards and reverse) in DNS (or [.filename]#/etc/hosts# as a minimum). CNAMEs will work, but the A and PTR records must be correct and in place. The error message is not very intuitive: `Kerberos5 refuses authentication because Read req failed: Key table entry not found`. * Some operating systems that may being acting as clients to your KDC do not set the permissions for `ksu` to be setuid `root`. This means that `ksu` does not work, which is a good security idea but annoying. This is not a KDC error. * With MITKerberos, if you want to allow a principal to have a ticket life longer than the default ten hours, you must use `modify_principal` in `kadmin` to change the maxlife of both the principal in question and the `krbtgt` principal. Then the principal can use the `-l` option with `kinit` to request a ticket with a longer lifetime. [NOTE] ==== If you run a packet sniffer on your KDC to add in troubleshooting and then run `kinit` from a workstation, you will notice that your TGT is sent immediately upon running `kinit` - even before you type your password! The explanation is that the Kerberos server freely transmits a TGT (Ticket Granting Ticket) to any unauthorized request; however, every TGT is encrypted in a key derived from the user's password. Therefore, when a user types their password it is not being sent to the KDC, it is being used to decrypt the TGT that `kinit` already obtained. If the decryption process results in a valid ticket with a valid time stamp, the user has valid Kerberos credentials. These credentials include a session key for establishing secure communications with the Kerberos server in the future, as well as the actual ticket-granting ticket, which is actually encrypted with the Kerberos server's own key. This second layer of encryption is unknown to the user, but it is what allows the Kerberos server to verify the authenticity of each TGT. ==== * If you want to use long ticket lifetimes (a week, for example) and you are using OpenSSH to connect to the machine where your ticket is stored, make sure that Kerberos `TicketCleanup` is set to `no` in your [.filename]#sshd_config# or else your tickets will be deleted when you log out. * Remember that host principals can have a longer ticket lifetime as well. If your user principal has a lifetime of a week but the host you are connecting to has a lifetime of nine hours, you will have an expired host principal in your cache and the ticket cache will not work as expected. * When setting up a [.filename]#krb5.dict# file to prevent specific bad passwords from being used (the manual page for `kadmind` covers this briefly), remember that it only applies to principals that have a password policy assigned to them. The [.filename]#krb5.dict# files format is simple: one string per line. Creating a symbolic link to [.filename]#/usr/shared/dict/words# might be useful. === Differences with the MIT port The major difference between the MIT and Heimdal installs relates to the `kadmin` program which has a different (but equivalent) set of commands and uses a different protocol. This has a large implications if your KDC is MIT as you will not be able to use the Heimdal `kadmin` program to administer your KDC remotely (or vice versa, for that matter). The client applications may also take slightly different command line options to accomplish the same tasks. Following the instructions on the MITKerberos web site (http://web.mit.edu/Kerberos/www/[http://web.mit.edu/Kerberos/www/]) is recommended. Be careful of path issues: the MIT port installs into [.filename]#/usr/local/# by default, and the "normal" system applications may be run instead of MIT if your `PATH` environment variable lists the system directories first. [NOTE] ==== With the MITpackage:security/krb5[] port that is provided by FreeBSD, be sure to read the [.filename]#/usr/local/shared/doc/krb5/README.FreeBSD# file installed by the port if you want to understand why logins via `telnetd` and `klogind` behave somewhat oddly. Most importantly, correcting the "incorrect permissions on cache file" behavior requires that the `login.krb5` binary be used for authentication so that it can properly change ownership for the forwarded credentials. ==== The [.filename]#rc.conf# must also be modified to contain the following configuration: [.programlisting] .... kerberos5_server="/usr/local/sbin/krb5kdc" kadmind5_server="/usr/local/sbin/kadmind" kerberos5_server_enable="YES" kadmind5_server_enable="YES" .... This is done because the applications for MIT kerberos installs binaries in the [.filename]#/usr/local# hierarchy. === Mitigating limitations found in Kerberos ==== Kerberos is an all-or-nothing approach Every service enabled on the network must be modified to work with Kerberos (or be otherwise secured against network attacks) or else the users credentials could be stolen and re-used. An example of this would be Kerberos enabling all remote shells (via `rsh` and `telnet`, for example) but not converting the POP3 mail server which sends passwords in plain text. ==== Kerberos is intended for single-user workstations In a multi-user environment, Kerberos is less secure. This is because it stores the tickets in the [.filename]#/tmp# directory, which is readable by all users. If a user is sharing a computer with several other people simultaneously (i.e. multi-user), it is possible that the user's tickets can be stolen (copied) by another user. This can be overcome with the `-c` filename command-line option or (preferably) the `KRB5CCNAME` environment variable, but this is rarely done. In principal, storing the ticket in the users home directory and using simple file permissions can mitigate this problem. ==== The KDC is a single point of failure By design, the KDC must be as secure as the master password database is contained on it. The KDC should have absolutely no other services running on it and should be physically secured. The danger is high because Kerberos stores all passwords encrypted with the same key (the "master" key), which in turn is stored as a file on the KDC. As a side note, a compromised master key is not quite as bad as one might normally fear. The master key is only used to encrypt the Kerberos database and as a seed for the random number generator. As long as access to your KDC is secure, an attacker cannot do much with the master key. Additionally, if the KDC is unavailable (perhaps due to a denial of service attack or network problems) the network services are unusable as authentication can not be performed, a recipe for a denial-of-service attack. This can alleviated with multiple KDCs (a single master and one or more slaves) and with careful implementation of secondary or fall-back authentication (PAM is excellent for this). ==== Kerberos Shortcomings Kerberos allows users, hosts and services to authenticate between themselves. It does not have a mechanism to authenticate the KDC to the users, hosts or services. This means that a trojanned `kinit` (for example) could record all user names and passwords. Something like package:security/tripwire[] or other file system integrity checking tools can alleviate this. === Resources and further information * http://www.faqs.org/faqs/Kerberos-faq/general/preamble.html[ The Kerberos FAQ] * http://web.mit.edu/Kerberos/www/dialogue.html[Designing an Authentication System: a Dialog in Four Scenes] * http://www.ietf.org/rfc/rfc1510.txt?number=1510[RFC 1510, The Kerberos Network Authentication Service (V5)] * http://web.mit.edu/Kerberos/www/[MIT Kerberos home page] * http://www.pdc.kth.se/heimdal/[Heimdal Kerberos home page] [[openssl]] == OpenSSL One feature that many users overlook is the OpenSSL toolkit included in FreeBSD. OpenSSL provides an encryption transport layer on top of the normal communications layer; thus allowing it to be intertwined with many network applications and services. Some uses of OpenSSL may include encrypted authentication of mail clients, web based transactions such as credit card payments and more. Many ports such as package:www/apache13-ssl[], and package:mail/sylpheed-claws[] will offer compilation support for building with OpenSSL. [NOTE] ==== In most cases the Ports Collection will attempt to build the package:security/openssl[] port unless the `WITH_OPENSSL_BASE` make variable is explicitly set to "yes". ==== The version of OpenSSL included in FreeBSD supports Secure Sockets Layer v2/v3 (SSLv2/SSLv3), Transport Layer Security v1 (TLSv1) network security protocols and can be used as a general cryptographic library. [NOTE] ==== While OpenSSL supports the IDEA algorithm, it is disabled by default due to United States patents. To use it, the license should be reviewed and, if the restrictions are acceptable, the `MAKE_IDEA` variable must be set in [.filename]#make.conf#. ==== One of the most common uses of OpenSSL is to provide certificates for use with software applications. These certificates ensure that the credentials of the company or individual are valid and not fraudulent. If the certificate in question has not been verified by one of the several "Certificate Authorities", or CAs, a warning is usually produced. A Certificate Authority is a company, such as http://www.verisign.com[VeriSign], which will sign certificates in order to validate credentials of individuals or companies. This process has a cost associated with it and is definitely not a requirement for using certificates; however, it can put some of the more paranoid users at ease. === Generating Certificates To generate a certificate, the following command is available: [source,shell] .... # openssl req -new -nodes -out req.pem -keyout cert.pem Generating a 1024 bit RSA private key ................++++++ .......................................++++++ writing new private key to 'cert.pem' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:US State or Province Name (full name) [Some-State]:PA Locality Name (eg, city) []:Pittsburgh Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Company Organizational Unit Name (eg, section) []:Systems Administrator Common Name (eg, YOUR name) []:localhost.example.org Email Address []:trhodes@FreeBSD.org Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []:SOME PASSWORD An optional company name []:Another Name .... Notice the response directly after the "Common Name" prompt shows a domain name. This prompt requires a server name to be entered for verification purposes; placing anything but a domain name would yield a useless certificate. Other options, for instance expire time, alternate encryption algorithms, etc. are available. A complete list may be obtained by viewing the man:openssl[1] manual page. Two files should now exist in the directory in which the aforementioned command was issued. The certificate request, [.filename]#req.pem#, may be sent to a certificate authority who will validate the credentials that you entered, sign the request and return the certificate to you. The second file created will be named [.filename]#cert.pem# and is the private key for the certificate and should be protected at all costs; if this falls in the hands of others it can be used to impersonate you (or your server). In cases where a signature from a CA is not required, a self signed certificate can be created. First, generate the RSA key: [source,shell] .... # openssl dsaparam -rand -genkey -out myRSA.key 1024 .... Next, generate the CA key: [source,shell] .... # openssl gendsa -des3 -out myca.key myRSA.key .... Use this key to create the certificate: [source,shell] .... # openssl req -new -x509 -days 365 -key myca.key -out new.crt .... Two new files should appear in the directory: a certificate authority signature file, [.filename]#myca.key# and the certificate itself, [.filename]#new.crt#. These should be placed in a directory, preferably under [.filename]#/etc#, which is readable only by `root`. Permissions of 0700 should be fine for this and they can be set with the `chmod` utility. === Using Certificates, an Example So what can these files do? A good use would be to encrypt connections to the SendmailMTA. This would dissolve the use of clear text authentication for users who send mail via the local MTA. [NOTE] ==== This is not the best use in the world as some MUAs will present the user with an error if they have not installed the certificate locally. Refer to the documentation included with the software for more information on certificate installation. ==== The following lines should be placed inside the local [.filename]#.mc# file: [.programlisting] .... dnl SSL Options define(`confCACERT_PATH',`/etc/certs')dnl define(`confCACERT',`/etc/certs/new.crt')dnl define(`confSERVER_CERT',`/etc/certs/new.crt')dnl define(`confSERVER_KEY',`/etc/certs/myca.key')dnl define(`confTLS_SRV_OPTIONS', `V')dnl .... Where [.filename]#/etc/certs/# is the directory to be used for storing the certificate and key files locally. The last few requirements are a rebuild of the local [.filename]#.cf# file. This is easily achieved by typing `make`[parameter]#install# within the [.filename]#/etc/mail# directory. Follow that up with `make`[parameter]#restart# which should start the Sendmail daemon. If all went well there will be no error messages in the [.filename]#/var/log/maillog# file and Sendmail will show up in the process list. For a simple test, simply connect to the mail server using the man:telnet[1] utility: [source,shell] .... # telnet example.com 25 Trying 192.0.34.166... Connected to example.com. Escape character is '^]'. 220 example.com ESMTP Sendmail 8.12.10/8.12.10; Tue, 31 Aug 2004 03:41:22 -0400 (EDT) ehlo example.com 250-example.com Hello example.com [192.0.34.166], pleased to meet you 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-8BITMIME 250-SIZE 250-DSN 250-ETRN 250-AUTH LOGIN PLAIN 250-STARTTLS 250-DELIVERBY 250 HELP quit 221 2.0.0 example.com closing connection Connection closed by foreign host. .... If the "STARTTLS" line appears in the output then everything is working correctly. [[ipsec]] == VPN over IPsec Creating a VPN between two networks, separated by the Internet, using FreeBSD gateways. === Understanding IPsec This section will guide you through the process of setting up IPsec, and to use it in an environment which consists of FreeBSD and Microsoft(R) Windows(R) 2000/XP machines, to make them communicate securely. In order to set up IPsec, it is necessary that you are familiar with the concepts of building a custom kernel (see crossref:kernelconfig[kernelconfig,Ρυθμίζοντας τον Πυρήνα του FreeBSD]). _IPsec_ is a protocol which sits on top of the Internet Protocol (IP) layer. It allows two or more hosts to communicate in a secure manner (hence the name). The FreeBSD IPsec "network stack" is based on the http://www.kame.net/[KAME] implementation, which has support for both protocol families, IPv4 and IPv6. [NOTE] ==== FreeBSD contains a "hardware accelerated" IPsec stack, known as "Fast IPsec", that was obtained from OpenBSD. It employs cryptographic hardware (whenever possible) via the man:crypto[4] subsystem to optimize the performance of IPsec. This subsystem is new, and does not support all the features that are available in the KAME version of IPsec. However, in order to enable hardware-accelerated IPsec, the following kernel option has to be added to your kernel configuration file: [source,shell] .... options FAST_IPSEC # new IPsec (cannot define w/ IPSEC) .... Note, that it is not currently possible to use the "Fast IPsec" subsystem in lieu of the KAME implementation of IPsec. Consult the manual page for more information. ==== [NOTE] ==== To let firewalls properly track state for man:gif[4] tunnels too, you have to enable the `IPSEC_FILTERGIF` in your kernel configuration: [source,shell] .... options IPSEC_FILTERGIF #filter ipsec packets from a tunnel .... ==== IPsec consists of two sub-protocols: * _Encapsulated Security Payload (ESP)_, protects the IP packet data from third party interference, by encrypting the contents using symmetric cryptography algorithms (like Blowfish, 3DES). * _Authentication Header (AH)_, protects the IP packet header from third party interference and spoofing, by computing a cryptographic checksum and hashing the IP packet header fields with a secure hashing function. This is then followed by an additional header that contains the hash, to allow the information in the packet to be authenticated. ESP and AH can either be used together or separately, depending on the environment. IPsec can either be used to directly encrypt the traffic between two hosts (known as _Transport Mode_); or to build "virtual tunnels" between two subnets, which could be used for secure communication between two corporate networks (known as _Tunnel Mode_). The latter is more commonly known as a _Virtual Private Network (VPN)_. The man:ipsec[4] manual page should be consulted for detailed information on the IPsec subsystem in FreeBSD. To add IPsec support to your kernel, add the following options to your kernel configuration file: [source,shell] .... options IPSEC #IP security options IPSEC_ESP #IP security (crypto; define w/ IPSEC) .... If IPsec debugging support is desired, the following kernel option should also be added: [source,shell] .... options IPSEC_DEBUG #debug for IP security .... === The Problem There is no standard for what constitutes a VPN. VPNs can be implemented using a number of different technologies, each of which have their own strengths and weaknesses. This section presents a scenario, and the strategies used for implementing a VPN for this scenario. === The Scenario: Two networks, connected to the Internet, to behave as one The premise is as follows: * You have at least two sites * Both sites are using IP internally * Both sites are connected to the Internet, through a gateway that is running FreeBSD. * The gateway on each network has at least one public IP address. * The internal addresses of the two networks can be public or private IP addresses, it does not matter. You can be running NAT on the gateway machine if necessary. * The internal IP addresses of the two networks _do not collide_. While I expect it is theoretically possible to use a combination of VPN technology and NAT to get this to work, I expect it to be a configuration nightmare. If you find that you are trying to connect two networks, both of which, internally, use the same private IP address range (e.g. both of them use `192.168.1.x`), then one of the networks will have to be renumbered. The network topology might look something like this: image::ipsec-network.png[] Notice the two public IP addresses. I will use the letters to refer to them in the rest of this article. Anywhere you see those letters in this article, replace them with your own public IP addresses. Note also that internally, the two gateway machines have .1 IP addresses, and that the two networks have different private IP addresses (`192.168.1.x` and `192.168.2.x` respectively). All the machines on the private networks have been configured to use the `.1` machine as their default gateway. The intention is that, from a network point of view, each network should view the machines on the other network as though they were directly attached the same router -- albeit a slightly slow router with an occasional tendency to drop packets. This means that (for example), machine `192.168.1.20` should be able to run [.programlisting] .... ping 192.168.2.34 .... and have it work, transparently. Windows(R) machines should be able to see the machines on the other network, browse file shares, and so on, in exactly the same way that they can browse machines on the local network. And the whole thing has to be secure. This means that traffic between the two networks has to be encrypted. Creating a VPN between these two networks is a multi-step process. The stages are as follows: . Create a "virtual" network link between the two networks, across the Internet. Test it, using tools like man:ping[8], to make sure it works. . Apply security policies to ensure that traffic between the two networks is transparently encrypted and decrypted as necessary. Test this, using tools like man:tcpdump[1], to ensure that traffic is encrypted. . Configure additional software on the FreeBSD gateways, to allow Windows(R) machines to see one another across the VPN. ==== Step 1: Creating and testing a "virtual" network link Suppose that you were logged in to the gateway machine on network #1 (with public IP address `A.B.C.D`, private IP address `192.168.1.1`), and you ran `ping 192.168.2.1`, which is the private address of the machine with IP address `W.X.Y.Z`. What needs to happen in order for this to work? . The gateway machine needs to know how to reach `192.168.2.1`. In other words, it needs to have a route to `192.168.2.1`. . Private IP addresses, such as those in the `192.168.x` range are not supposed to appear on the Internet at large. Instead, each packet you send to `192.168.2.1` will need to be wrapped up inside another packet. This packet will need to appear to be from `A.B.C.D`, and it will have to be sent to `W.X.Y.Z`. This process is called _encapsulation_. . Once this packet arrives at `W.X.Y.Z` it will need to "unencapsulated", and delivered to `192.168.2.1`. You can think of this as requiring a "tunnel" between the two networks. The two "tunnel mouths" are the IP addresses `A.B.C.D` and `W.X.Y.Z`, and the tunnel must be told the addresses of the private IP addresses that will be allowed to pass through it. The tunnel is used to transfer traffic with private IP addresses across the public Internet. This tunnel is created by using the generic interface, or [.filename]#gif# devices on FreeBSD. As you can imagine, the [.filename]#gif# interface on each gateway host must be configured with four IP addresses; two for the public IP addresses, and two for the private IP addresses. Support for the gif device must be compiled in to the FreeBSD kernel on both machines. You can do this by adding the line: [.programlisting] .... device gif .... to the kernel configuration files on both machines, and then compile, install, and reboot as normal. Configuring the tunnel is a two step process. First the tunnel must be told what the outside (or public) IP addresses are, using man:ifconfig[8]. Then the private IP addresses must be configured using man:ifconfig[8]. On the gateway machine on network #1 you would run the following commands to configure the tunnel. [source,shell] .... # ifconfig gif0 create # ifconfig gif0 tunnel A.B.C.D W.X.Y.Z # ifconfig gif0 inet 192.168.1.1 192.168.2.1 netmask 0xffffffff .... On the other gateway machine you run the same commands, but with the order of the IP addresses reversed. [source,shell] .... # ifconfig gif0 create # ifconfig gif0 tunnel W.X.Y.Z A.B.C.D # ifconfig gif0 inet 192.168.2.1 192.168.1.1 netmask 0xffffffff .... You can then run: [.programlisting] .... ifconfig gif0 .... to see the configuration. For example, on the network #1 gateway, you would see this: [source,shell] .... # ifconfig gif0 gif0: flags=8051 mtu 1280 tunnel inet A.B.C.D --> W.X.Y.Z inet 192.168.1.1 --> 192.168.2.1 netmask 0xffffffff .... As you can see, a tunnel has been created between the physical addresses `A.B.C.D` and `W.X.Y.Z`, and the traffic allowed through the tunnel is that between `192.168.1.1` and `192.168.2.1`. This will also have added an entry to the routing table on both machines, which you can examine with the command `netstat -rn`. This output is from the gateway host on network #1. [source,shell] .... # netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire ... 192.168.2.1 192.168.1.1 UH 0 0 gif0 ... .... As the "Flags" value indicates, this is a host route, which means that each gateway knows how to reach the other gateway, but they do not know how to reach the rest of their respective networks. That problem will be fixed shortly. It is likely that you are running a firewall on both machines. This will need to be circumvented for your VPN traffic. You might want to allow all traffic between both networks, or you might want to include firewall rules that protect both ends of the VPN from one another. It greatly simplifies testing if you configure the firewall to allow all traffic through the VPN. You can always tighten things up later. If you are using man:ipfw[8] on the gateway machines then a command like [.programlisting] .... ipfw add 1 allow ip from any to any via gif0 .... will allow all traffic between the two end points of the VPN, without affecting your other firewall rules. Obviously you will need to run this command on both gateway hosts. This is sufficient to allow each gateway machine to ping the other. On `192.168.1.1`, you should be able to run [.programlisting] .... ping 192.168.2.1 .... and get a response, and you should be able to do the same thing on the other gateway machine. However, you will not be able to reach internal machines on either network yet. This is because of the routing -- although the gateway machines know how to reach one another, they do not know how to reach the network behind each one. To solve this problem you must add a static route on each gateway machine. The command to do this on the first gateway would be: [.programlisting] .... route add 192.168.2.0 192.168.2.1 netmask 0xffffff00 .... This says "In order to reach the hosts on the network `192.168.2.0`, send the packets to the host ``192.168.2.1``". You will need to run a similar command on the other gateway, but with the `192.168.1.x` addresses instead. IP traffic from hosts on one network will now be able to reach hosts on the other network. That has now created two thirds of a VPN between the two networks, in as much as it is "virtual" and it is a "network". It is not private yet. You can test this using man:ping[8] and man:tcpdump[1]. Log in to the gateway host and run [.programlisting] .... tcpdump dst host 192.168.2.1 .... In another log in session on the same host run [.programlisting] .... ping 192.168.2.1 .... You will see output that looks something like this: [.programlisting] .... 16:10:24.018080 192.168.1.1 192.168.2.1: icmp: echo request 16:10:24.018109 192.168.1.1 192.168.2.1: icmp: echo reply 16:10:25.018814 192.168.1.1 192.168.2.1: icmp: echo request 16:10:25.018847 192.168.1.1 192.168.2.1: icmp: echo reply 16:10:26.028896 192.168.1.1 192.168.2.1: icmp: echo request 16:10:26.029112 192.168.1.1 192.168.2.1: icmp: echo reply .... As you can see, the ICMP messages are going back and forth unencrypted. If you had used the `-s` parameter to man:tcpdump[1] to grab more bytes of data from the packets you would see more information. Obviously this is unacceptable. The next section will discuss securing the link between the two networks so that all traffic is automatically encrypted. .Summary: * Configure both kernels with "device gif". * Edit [.filename]#/etc/rc.conf# on gateway host #1 and add the following lines (replacing IP addresses as necessary). + [.programlisting] .... gif_interfaces="gif0" gifconfig_gif0="A.B.C.D W.X.Y.Z" ifconfig_gif0="inet 192.168.1.1 192.168.2.1 netmask 0xffffffff" static_routes="vpn" route_vpn="192.168.2.0 192.168.2.1 netmask 0xffffff00" .... * Edit your firewall script ([.filename]#/etc/rc.firewall#, or similar) on both hosts, and add + [.programlisting] .... ipfw add 1 allow ip from any to any via gif0 .... * Make similar changes to [.filename]#/etc/rc.conf# on gateway host #2, reversing the order of IP addresses. ==== Step 2: Securing the link To secure the link we will be using IPsec. IPsec provides a mechanism for two hosts to agree on an encryption key, and to then use this key in order to encrypt data between the two hosts. The are two areas of configuration to be considered here. . There must be a mechanism for two hosts to agree on the encryption mechanism to use. Once two hosts have agreed on this mechanism there is said to be a "security association" between them. . There must be a mechanism for specifying which traffic should be encrypted. Obviously, you do not want to encrypt all your outgoing traffic -- you only want to encrypt the traffic that is part of the VPN. The rules that you put in place to determine what traffic will be encrypted are called "security policies". Security associations and security policies are both maintained by the kernel, and can be modified by userland programs. However, before you can do this you must configure the kernel to support IPsec and the Encapsulated Security Payload (ESP) protocol. This is done by configuring a kernel with: [.programlisting] .... options IPSEC options IPSEC_ESP .... and recompiling, reinstalling, and rebooting. As before you will need to do this to the kernels on both of the gateway hosts. You have two choices when it comes to setting up security associations. You can configure them by hand between two hosts, which entails choosing the encryption algorithm, encryption keys, and so forth, or you can use daemons that implement the Internet Key Exchange protocol (IKE) to do this for you. I recommend the latter. Apart from anything else, it is easier to set up. Editing and displaying security policies is carried out using man:setkey[8]. By analogy, `setkey` is to the kernel's security policy tables as man:route[8] is to the kernel's routing tables. `setkey` can also display the current security associations, and to continue the analogy further, is akin to `netstat -r` in that respect. There are a number of choices for daemons to manage security associations with FreeBSD. This article will describe how to use one of these, racoon - which is available from package:security/ipsec-tools[] in the FreeBSD Ports collection. The racoon software must be run on both gateway hosts. On each host it is configured with the IP address of the other end of the VPN, and a secret key (which you choose, and must be the same on both gateways). The two daemons then contact one another, confirm that they are who they say they are (by using the secret key that you configured). The daemons then generate a new secret key, and use this to encrypt the traffic over the VPN. They periodically change this secret, so that even if an attacker were to crack one of the keys (which is as theoretically close to unfeasible as it gets) it will not do them much good -- by the time they have cracked the key the two daemons have chosen another one. The configuration file for racoon is stored in [.filename]#${PREFIX}/etc/racoon#. You should find a configuration file there, which should not need to be changed too much. The other component of racoon's configuration, which you will need to change, is the "pre-shared key". The default racoon configuration expects to find this in the file [.filename]#${PREFIX}/etc/racoon/psk.txt#. It is important to note that the pre-shared key is _not_ the key that will be used to encrypt your traffic across the VPN link, it is simply a token that allows the key management daemons to trust one another. [.filename]#psk.txt# contains a line for each remote site you are dealing with. In this example, where there are two sites, each [.filename]#psk.txt# file will contain one line (because each end of the VPN is only dealing with one other end). On gateway host #1 this line should look like this: [.programlisting] .... W.X.Y.Z secret .... That is, the _public_ IP address of the remote end, whitespace, and a text string that provides the secret. Obviously, you should not use "secret" as your key -- the normal rules for choosing a password apply. On gateway host #2 the line would look like this [.programlisting] .... A.B.C.D secret .... That is, the public IP address of the remote end, and the same secret key. [.filename]#psk.txt# must be mode `0600` (i.e., only read/write to `root`) before racoon will run. You must run racoon on both gateway machines. You will also need to add some firewall rules to allow the IKE traffic, which is carried over UDP to the ISAKMP (Internet Security Association Key Management Protocol) port. Again, this should be fairly early in your firewall ruleset. [.programlisting] .... ipfw add 1 allow udp from A.B.C.D to W.X.Y.Z isakmp ipfw add 1 allow udp from W.X.Y.Z to A.B.C.D isakmp .... Once racoon is running you can try pinging one gateway host from the other. The connection is still not encrypted, but racoon will then set up the security associations between the two hosts -- this might take a moment, and you may see this as a short delay before the ping commands start responding. Once the security association has been set up you can view it using man:setkey[8]. Run [.programlisting] .... setkey -D .... on either host to view the security association information. That's one half of the problem. The other half is setting your security policies. To create a sensible security policy, let's review what's been set up so far. This discussions hold for both ends of the link. Each IP packet that you send out has a header that contains data about the packet. The header includes the IP addresses of both the source and destination. As we already know, private IP addresses, such as the `192.168.x.y` range are not supposed to appear on the public Internet. Instead, they must first be encapsulated inside another packet. This packet must have the public source and destination IP addresses substituted for the private addresses. So if your outgoing packet started looking like this: image::ipsec-out-pkt.png[] Then it will be encapsulated inside another packet, looking something like this: image::ipsec-encap-pkt.png[] This encapsulation is carried out by the [.filename]#gif# device. As you can see, the packet now has real IP addresses on the outside, and our original packet has been wrapped up as data inside the packet that will be put out on the Internet. Obviously, we want all traffic between the VPNs to be encrypted. You might try putting this in to words, as: "If a packet leaves from `A.B.C.D`, and it is destined for `W.X.Y.Z`, then encrypt it, using the necessary security associations." "If a packet arrives from `W.X.Y.Z`, and it is destined for `A.B.C.D`, then decrypt it, using the necessary security associations." That's close, but not quite right. If you did this, all traffic to and from `W.X.Y.Z`, even traffic that was not part of the VPN, would be encrypted. That's not quite what you want. The correct policy is as follows "If a packet leaves from `A.B.C.D`, and that packet is encapsulating another packet, and it is destined for `W.X.Y.Z`, then encrypt it, using the necessary security associations." "If a packet arrives from `W.X.Y.Z`, and that packet is encapsulating another packet, and it is destined for `A.B.C.D`, then decrypt it, using the necessary security associations." A subtle change, but a necessary one. Security policies are also set using man:setkey[8]. man:setkey[8] features a configuration language for defining the policy. You can either enter configuration instructions via stdin, or you can use the `-f` option to specify a filename that contains configuration instructions. The configuration on gateway host #1 (which has the public IP address `A.B.C.D`) to force all outbound traffic to `W.X.Y.Z` to be encrypted is: [.programlisting] .... spdadd A.B.C.D/32 W.X.Y.Z/32 ipencap -P out ipsec esp/tunnel/A.B.C.D-W.X.Y.Z/require; .... Put these commands in a file (e.g. [.filename]#/etc/ipsec.conf#) and then run [source,shell] .... # setkey -f /etc/ipsec.conf .... `spdadd` tells man:setkey[8] that we want to add a rule to the secure policy database. The rest of this line specifies which packets will match this policy. `A.B.C.D/32` and `W.X.Y.Z/32` are the IP addresses and netmasks that identify the network or hosts that this policy will apply to. In this case, we want it to apply to traffic between these two hosts. `ipencap` tells the kernel that this policy should only apply to packets that encapsulate other packets. `-P out` says that this policy applies to outgoing packets, and `ipsec` says that the packet will be secured. The second line specifies how this packet will be encrypted. `esp` is the protocol that will be used, while `tunnel` indicates that the packet will be further encapsulated in an IPsec packet. The repeated use of `A.B.C.D` and `W.X.Y.Z` is used to select the security association to use, and the final `require` mandates that packets must be encrypted if they match this rule. This rule only matches outgoing packets. You will need a similar rule to match incoming packets. [.programlisting] .... spdadd W.X.Y.Z/32 A.B.C.D/32 ipencap -P in ipsec esp/tunnel/W.X.Y.Z-A.B.C.D/require; .... Note the `in` instead of `out` in this case, and the necessary reversal of the IP addresses. The other gateway host (which has the public IP address `W.X.Y.Z`) will need similar rules. [.programlisting] .... spdadd W.X.Y.Z/32 A.B.C.D/32 ipencap -P out ipsec esp/tunnel/W.X.Y.Z-A.B.C.D/require; spdadd A.B.C.D/32 W.X.Y.Z/32 ipencap -P in ipsec esp/tunnel/A.B.C.D-W.X.Y.Z/require; .... Finally, you need to add firewall rules to allow ESP and IPENCAP packets back and forth. These rules will need to be added to both hosts. [.programlisting] .... ipfw add 1 allow esp from A.B.C.D to W.X.Y.Z ipfw add 1 allow esp from W.X.Y.Z to A.B.C.D ipfw add 1 allow ipencap from A.B.C.D to W.X.Y.Z ipfw add 1 allow ipencap from W.X.Y.Z to A.B.C.D .... Because the rules are symmetric you can use the same rules on each gateway host. Outgoing packets will now look something like this: image::ipsec-crypt-pkt.png[] When they are received by the far end of the VPN they will first be decrypted (using the security associations that have been negotiated by racoon). Then they will enter the [.filename]#gif# interface, which will unwrap the second layer, until you are left with the innermost packet, which can then travel in to the inner network. You can check the security using the same man:ping[8] test from earlier. First, log in to the `A.B.C.D` gateway machine, and run: [.programlisting] .... tcpdump dst host 192.168.2.1 .... In another log in session on the same host run [.programlisting] .... ping 192.168.2.1 .... This time you should see output like the following: [.programlisting] .... XXX tcpdump output .... Now, as you can see, man:tcpdump[1] shows the ESP packets. If you try to examine them with the `-s` option you will see (apparently) gibberish, because of the encryption. Congratulations. You have just set up a VPN between two remote sites. .Summary * Configure both kernels with: + [.programlisting] .... options IPSEC options IPSEC_ESP .... * Install package:security/ipsec-tools[]. Edit [.filename]#${PREFIX}/etc/racoon/psk.txt# on both gateway hosts, adding an entry for the remote host's IP address and a secret key that they both know. Make sure this file is mode 0600. * Add the following lines to [.filename]#/etc/rc.conf# on each host: + [.programlisting] .... ipsec_enable="YES" ipsec_file="/etc/ipsec.conf" .... * Create an [.filename]#/etc/ipsec.conf# on each host that contains the necessary spdadd lines. On gateway host #1 this would be: + [.programlisting] .... spdadd A.B.C.D/32 W.X.Y.Z/32 ipencap -P out ipsec esp/tunnel/A.B.C.D-W.X.Y.Z/require; spdadd W.X.Y.Z/32 A.B.C.D/32 ipencap -P in ipsec esp/tunnel/W.X.Y.Z-A.B.C.D/require; .... + On gateway host #2 this would be: + [.programlisting] .... spdadd W.X.Y.Z/32 A.B.C.D/32 ipencap -P out ipsec esp/tunnel/W.X.Y.Z-A.B.C.D/require; spdadd A.B.C.D/32 W.X.Y.Z/32 ipencap -P in ipsec esp/tunnel/A.B.C.D-W.X.Y.Z/require; .... * Add firewall rules to allow IKE, ESP, and IPENCAP traffic to both hosts: + [.programlisting] .... ipfw add 1 allow udp from A.B.C.D to W.X.Y.Z isakmp ipfw add 1 allow udp from W.X.Y.Z to A.B.C.D isakmp ipfw add 1 allow esp from A.B.C.D to W.X.Y.Z ipfw add 1 allow esp from W.X.Y.Z to A.B.C.D ipfw add 1 allow ipencap from A.B.C.D to W.X.Y.Z ipfw add 1 allow ipencap from W.X.Y.Z to A.B.C.D .... The previous two steps should suffice to get the VPN up and running. Machines on each network will be able to refer to one another using IP addresses, and all traffic across the link will be automatically and securely encrypted. [[openssh]] == OpenSSH OpenSSH is a set of network connectivity tools used to access remote machines securely. It can be used as a direct replacement for `rlogin`, `rsh`, `rcp`, and `telnet`. Additionally, TCP/IP connections can be tunneled/forwarded securely through SSH. OpenSSH encrypts all traffic to effectively eliminate eavesdropping, connection hijacking, and other network-level attacks. OpenSSH is maintained by the OpenBSD project, and is based upon SSH v1.2.12 with all the recent bug fixes and updates. It is compatible with both SSH protocols 1 and 2. === Advantages of Using OpenSSH Normally, when using man:telnet[1] or man:rlogin[1], data is sent over the network in an clear, un-encrypted form. Network sniffers anywhere in between the client and server can steal your user/password information or data transferred in your session. OpenSSH offers a variety of authentication and encryption methods to prevent this from happening. === Enabling sshd The sshd is an option presented during a `Standard` install of FreeBSD. To see if sshd is enabled, check the [.filename]#rc.conf# file for: [source,shell] .... sshd_enable="YES" .... This will load man:sshd[8], the daemon program for OpenSSH, the next time your system initializes. Alternatively, it is possible to use [.filename]#/etc/rc.d/sshd# man:rc[8] script to start OpenSSH: [.programlisting] .... /etc/rc.d/sshd start .... === SSH Client The man:ssh[1] utility works similarly to man:rlogin[1]. [source,shell] .... # ssh user@example.com Host key not found from the list of known hosts. Are you sure you want to continue connecting (yes/no)? yes Host 'example.com' added to the list of known hosts. user@example.com's password: ******* .... The login will continue just as it would have if a session was created using `rlogin` or `telnet`. SSH utilizes a key fingerprint system for verifying the authenticity of the server when the client connects. The user is prompted to enter `yes` only when connecting for the first time. Future attempts to login are all verified against the saved fingerprint key. The SSH client will alert you if the saved fingerprint differs from the received fingerprint on future login attempts. The fingerprints are saved in [.filename]#~/.ssh/known_hosts#, or [.filename]#~/.ssh/known_hosts2# for SSH v2 fingerprints. By default, recent versions of the OpenSSH servers only accept SSH v2 connections. The client will use version 2 if possible and will fall back to version 1. The client can also be forced to use one or the other by passing it the `-1` or `-2` for version 1 or version 2, respectively. The version 1 compatibility is maintained in the client for backwards compatibility with older versions. === Secure Copy The man:scp[1] command works similarly to man:rcp[1]; it copies a file to or from a remote machine, except in a secure fashion. [source,shell] .... # scp user@example.com:/COPYRIGHT COPYRIGHT user@example.com's password: ******* COPYRIGHT 100% |*****************************| 4735 00:00 # .... Since the fingerprint was already saved for this host in the previous example, it is verified when using man:scp[1] here. The arguments passed to man:scp[1] are similar to man:cp[1], with the file or files in the first argument, and the destination in the second. Since the file is fetched over the network, through SSH, one or more of the file arguments takes on the form `user@host:path_to_remote_file`. === Configuration The system-wide configuration files for both the OpenSSH daemon and client reside within the [.filename]#/etc/ssh# directory. [.filename]#ssh_config# configures the client settings, while [.filename]#sshd_config# configures the daemon. Additionally, the `sshd_program` ([.filename]#/usr/sbin/sshd# by default), and `sshd_flags`[.filename]#rc.conf# options can provide more levels of configuration. [[security-ssh-keygen]] === ssh-keygen Instead of using passwords, man:ssh-keygen[1] can be used to generate DSA or RSA keys to authenticate a user: [source,shell] .... % ssh-keygen -t dsa Generating public/private dsa key pair. Enter file in which to save the key (/home/user/.ssh/id_dsa): Created directory '/home/user/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/user/.ssh/id_dsa. Your public key has been saved in /home/user/.ssh/id_dsa.pub. The key fingerprint is: bb:48:db:f2:93:57:80:b6:aa:bc:f5:d5:ba:8f:79:17 user@host.example.com .... man:ssh-keygen[1] will create a public and private key pair for use in authentication. The private key is stored in [.filename]#~/.ssh/id_dsa# or [.filename]#~/.ssh/id_rsa#, whereas the public key is stored in [.filename]#~/.ssh/id_dsa.pub# or [.filename]#~/.ssh/id_rsa.pub#, respectively for DSA and RSA key types. The public key must be placed in [.filename]#~/.ssh/authorized_keys# of the remote machine in order for the setup to work. Similarly, RSA version 1 public keys should be placed in [.filename]#~/.ssh/authorized_keys#. This will allow connection to the remote machine based upon SSH keys instead of passwords. If a passphrase is used in man:ssh-keygen[1], the user will be prompted for a password each time in order to use the private key. man:ssh-agent[1] can alleviate the strain of repeatedly entering long passphrases, and is explored in the <> section below. [WARNING] ==== The various options and files can be different according to the OpenSSH version you have on your system; to avoid problems you should consult the man:ssh-keygen[1] manual page. ==== [[security-ssh-agent]] === ssh-agent and ssh-add The man:ssh-agent[1] and man:ssh-add[1] utilities provide methods for SSH keys to be loaded into memory for use, without needing to type the passphrase each time. The man:ssh-agent[1] utility will handle the authentication using the private key(s) that are loaded into it. man:ssh-agent[1] should be used to launch another application. At the most basic level, it could spawn a shell or at a more advanced level, a window manager. To use man:ssh-agent[1] in a shell, first it will need to be spawned with a shell as an argument. Secondly, the identity needs to be added by running man:ssh-add[1] and providing it the passphrase for the private key. Once these steps have been completed the user will be able to man:ssh[1] to any host that has the corresponding public key installed. For example: [source,shell] .... % ssh-agent csh % ssh-add Enter passphrase for /home/user/.ssh/id_dsa: Identity added: /home/user/.ssh/id_dsa (/home/user/.ssh/id_dsa) % .... To use man:ssh-agent[1] in X11, a call to man:ssh-agent[1] will need to be placed in [.filename]#~/.xinitrc#. This will provide the man:ssh-agent[1] services to all programs launched in X11. An example [.filename]#~/.xinitrc# file might look like this: [.programlisting] .... exec ssh-agent startxfce4 .... This would launch man:ssh-agent[1], which would in turn launch XFCE, every time X11 starts. Then once that is done and X11 has been restarted so that the changes can take effect, simply run man:ssh-add[1] to load all of your SSH keys. [[security-ssh-tunneling]] === SSH Tunneling OpenSSH has the ability to create a tunnel to encapsulate another protocol in an encrypted session. The following command tells man:ssh[1] to create a tunnel for telnet: [source,shell] .... % ssh -2 -N -f -L 5023:localhost:23 user@foo.example.com % .... The `ssh` command is used with the following options: `-2`:: Forces `ssh` to use version 2 of the protocol. (Do not use if you are working with older SSH servers) `-N`:: Indicates no command, or tunnel only. If omitted, `ssh` would initiate a normal session. `-f`:: Forces `ssh` to run in the background. `-L`:: Indicates a local tunnel in _localport:remotehost:remoteport_ fashion. `user@foo.example.com`:: The remote SSH server. An SSH tunnel works by creating a listen socket on `localhost` on the specified port. It then forwards any connection received on the local host/port via the SSH connection to the specified remote host and port. In the example, port _5023_ on `localhost` is being forwarded to port _23_ on `localhost` of the remote machine. Since _23_ is telnet, this would create a secure telnet session through an SSH tunnel. This can be used to wrap any number of insecure TCP protocols such as SMTP, POP3, FTP, etc. .Using SSH to Create a Secure Tunnel for SMTP [example] ==== [source,shell] .... % ssh -2 -N -f -L 5025:localhost:25 user@mailserver.example.com user@mailserver.example.com's password: ***** % telnet localhost 5025 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mailserver.example.com ESMTP .... This can be used in conjunction with an man:ssh-keygen[1] and additional user accounts to create a more seamless/hassle-free SSH tunneling environment. Keys can be used in place of typing a password, and the tunnels can be run as a separate user. ==== ==== Practical SSH Tunneling Examples ===== Secure Access of a POP3 Server At work, there is an SSH server that accepts connections from the outside. On the same office network resides a mail server running a POP3 server. The network, or network path between your home and office may or may not be completely trustable. Because of this, you need to check your e-mail in a secure manner. The solution is to create an SSH connection to your office's SSH server, and tunnel through to the mail server. [source,shell] .... % ssh -2 -N -f -L 2110:mail.example.com:110 user@ssh-server.example.com user@ssh-server.example.com's password: ****** .... When the tunnel is up and running, you can point your mail client to send POP3 requests to `localhost` port 2110. A connection here will be forwarded securely across the tunnel to `mail.example.com`. ===== Bypassing a Draconian Firewall Some network administrators impose extremely draconian firewall rules, filtering not only incoming connections, but outgoing connections. You may be only given access to contact remote machines on ports 22 and 80 for SSH and web surfing. You may wish to access another (perhaps non-work related) service, such as an Ogg Vorbis server to stream music. If this Ogg Vorbis server is streaming on some other port than 22 or 80, you will not be able to access it. The solution is to create an SSH connection to a machine outside of your network's firewall, and use it to tunnel to the Ogg Vorbis server. [source,shell] .... % ssh -2 -N -f -L 8888:music.example.com:8000 user@unfirewalled-system.example.org user@unfirewalled-system.example.org's password: ******* .... Your streaming client can now be pointed to `localhost` port 8888, which will be forwarded over to `music.example.com` port 8000, successfully evading the firewall. === The `AllowUsers` Users Option It is often a good idea to limit which users can log in and from where. The `AllowUsers` option is a good way to accomplish this. For example, to only allow the `root` user to log in from `192.168.1.32`, something like this would be appropriate in the [.filename]#/etc/ssh/sshd_config# file: [.programlisting] .... AllowUsers root@192.168.1.32 .... To allow the user `admin` to log in from anywhere, just list the username by itself: [.programlisting] .... AllowUsers admin .... Multiple users should be listed on the same line, like so: [.programlisting] .... AllowUsers root@192.168.1.32 admin .... [NOTE] ==== It is important that you list each user that needs to log in to this machine; otherwise they will be locked out. ==== After making changes to [.filename]#/etc/ssh/sshd_config# you must tell man:sshd[8] to reload its config files, by running: [source,shell] .... # /etc/rc.d/sshd reload .... === Further Reading http://www.openssh.com/[OpenSSH] man:ssh[1] man:scp[1] man:ssh-keygen[1] man:ssh-agent[1] man:ssh-add[1] man:ssh_config[5] man:sshd[8] man:sftp-server[8] man:sshd_config[5] [[fs-acl]] == File System Access Control Lists In conjunction with file system enhancements like snapshots, FreeBSD 5.0 and later offers the security of File System Access Control Lists (ACLs). Access Control Lists extend the standard UNIX(R) permission model in a highly compatible (POSIX(R).1e) way. This feature permits an administrator to make use of and take advantage of a more sophisticated security model. To enable ACL support for UFS file systems, the following: [.programlisting] .... options UFS_ACL .... must be compiled into the kernel. If this option has not been compiled in, a warning message will be displayed when attempting to mount a file system supporting ACLs. This option is included in the [.filename]#GENERIC# kernel. ACLs rely on extended attributes being enabled on the file system. Extended attributes are natively supported in the next generation UNIX(R) file system, UFS2. [NOTE] ==== A higher level of administrative overhead is required to configure extended attributes on UFS1 than on UFS2. The performance of extended attributes on UFS2 is also substantially higher. As a result, UFS2 is generally recommended in preference to UFS1 for use with access control lists. ==== ACLs are enabled by the mount-time administrative flag, `acls`, which may be added to [.filename]#/etc/fstab#. The mount-time flag can also be automatically set in a persistent manner using man:tunefs[8] to modify a superblock ACLs flag in the file system header. In general, it is preferred to use the superblock flag for several reasons: * The mount-time ACLs flag cannot be changed by a remount (man:mount[8] `-u`), only by means of a complete man:umount[8] and fresh man:mount[8]. This means that ACLs cannot be enabled on the root file system after boot. It also means that you cannot change the disposition of a file system once it is in use. * Setting the superblock flag will cause the file system to always be mounted with ACLs enabled even if there is not an [.filename]#fstab# entry or if the devices re-order. This prevents accidental mounting of the file system without ACLs enabled, which can result in ACLs being improperly enforced, and hence security problems. [NOTE] ==== We may change the ACLs behavior to allow the flag to be enabled without a complete fresh man:mount[8], but we consider it desirable to discourage accidental mounting without ACLs enabled, because you can shoot your feet quite nastily if you enable ACLs, then disable them, then re-enable them without flushing the extended attributes. In general, once you have enabled ACLs on a file system, they should not be disabled, as the resulting file protections may not be compatible with those intended by the users of the system, and re-enabling ACLs may re-attach the previous ACLs to files that have since had their permissions changed, resulting in other unpredictable behavior. ==== File systems with ACLs enabled will show a `+` (plus) sign in their permission settings when viewed. For example: [.programlisting] .... drwx------ 2 robert robert 512 Dec 27 11:54 private drwxrwx---+ 2 robert robert 512 Dec 23 10:57 directory1 drwxrwx---+ 2 robert robert 512 Dec 22 10:20 directory2 drwxrwx---+ 2 robert robert 512 Dec 27 11:57 directory3 drwxr-xr-x 2 robert robert 512 Nov 10 11:54 public_html .... Here we see that the [.filename]#directory1#, [.filename]#directory2#, and [.filename]#directory3# directories are all taking advantage of ACLs. The [.filename]#public_html# directory is not. === Making Use of ACLs The file system ACLs can be viewed by the man:getfacl[1] utility. For instance, to view the ACL settings on the [.filename]#test# file, one would use the command: [source,shell] .... % getfacl test #file:test #owner:1001 #group:1001 user::rw- group::r-- other::r-- .... To change the ACL settings on this file, invoke the man:setfacl[1] utility. Observe: [source,shell] .... % setfacl -k test .... The `-k` flag will remove all of the currently defined ACLs from a file or file system. The more preferable method would be to use `-b` as it leaves the basic fields required for ACLs to work. [source,shell] .... % setfacl -m u:trhodes:rwx,group:web:r--,o::--- test .... In the aforementioned command, the `-m` option was used to modify the default ACL entries. Since there were no pre-defined entries, as they were removed by the previous command, this will restore the default options and assign the options listed. Take care to notice that if you add a user or group which does not exist on the system, an `Invalid argument` error will be printed to [.filename]#stdout#. [[security-portaudit]] == Monitoring Third Party Security Issues In recent years, the security world has made many improvements to how vulnerability assessment is handled. The threat of system intrusion increases as third party utilities are installed and configured for virtually any operating system available today. Vulnerability assessment is a key factor in security, and while FreeBSD releases advisories for the base system, doing so for every third party utility is beyond the FreeBSD Project's capability. There is a way to mitigate third party vulnerabilities and warn administrators of known security issues. A FreeBSD add on utility known as Portaudit exists solely for this purpose. The package:ports-mgmt/portaudit[] port polls a database, updated and maintained by the FreeBSD Security Team and ports developers, for known security issues. To begin using Portaudit, one must install it from the Ports Collection: [source,shell] .... # cd /usr/ports/ports-mgmt/portaudit make install clean .... During the install process, the configuration files for man:periodic[8] will be updated, permitting Portaudit output in the daily security runs. Ensure the daily security run emails, which are sent to ``root``'s email account, are being read. No more configuration will be required here. After installation, an administrator can update the database and view known vulnerabilities in installed packages by invoking the following command: [source,shell] .... # portaudit -Fda .... [NOTE] ==== The database will automatically be updated during the man:periodic[8] run; thus, the previous command is completely optional. It is only required for the following examples. ==== To audit the third party utilities installed as part of the Ports Collection at anytime, an administrator need only run the following command: [source,shell] .... # portaudit -a .... Portaudit will produce something like this for vulnerable packages: [.programlisting] .... Affected package: cups-base-1.1.22.0_1 Type of problem: cups-base -- HPGL buffer overflow vulnerability. Reference: http://www.FreeBSD.org/ports/portaudit/40a3bca2-6809-11d9-a9e7-0001020eed82.html 1 problem(s) in your installed packages found. You are advised to update or deinstall the affected package(s) immediately. .... By pointing a web browser to the URL shown, an administrator may obtain more information about the vulnerability in question. This will include versions affected, by FreeBSD Port version, along with other web sites which may contain security advisories. In short, Portaudit is a powerful utility and extremely useful when coupled with the Portupgrade port. [[security-advisories]] == FreeBSD Security Advisories Like many production quality operating systems, FreeBSD publishes "Security Advisories". These advisories are usually mailed to the security lists and noted in the Errata only after the appropriate releases have been patched. This section will work to explain what an advisory is, how to understand it, and what measures to take in order to patch a system. === What does an advisory look like? The FreeBSD security advisories look similar to the one below, taken from the {freebsd-security-notifications} mailing list. [.programlisting] .... ============================================================================= FreeBSD-SA-XX:XX.UTIL Security Advisory The FreeBSD Project Topic: denial of service due to some problem <.> Category: core <.> Module: sys <.> Announced: 2003-09-23 <.> Credits: Person@EMAIL-ADDRESS <.> Affects: All releases of FreeBSD <.> FreeBSD 4-STABLE prior to the correction date Corrected: 2003-09-23 16:42:59 UTC (RELENG_4, 4.9-PRERELEASE) 2003-09-23 20:08:42 UTC (RELENG_5_1, 5.1-RELEASE-p6) 2003-09-23 20:07:06 UTC (RELENG_5_0, 5.0-RELEASE-p15) 2003-09-23 16:44:58 UTC (RELENG_4_8, 4.8-RELEASE-p8) 2003-09-23 16:47:34 UTC (RELENG_4_7, 4.7-RELEASE-p18) 2003-09-23 16:49:46 UTC (RELENG_4_6, 4.6-RELEASE-p21) 2003-09-23 16:51:24 UTC (RELENG_4_5, 4.5-RELEASE-p33) 2003-09-23 16:52:45 UTC (RELENG_4_4, 4.4-RELEASE-p43) 2003-09-23 16:54:39 UTC (RELENG_4_3, 4.3-RELEASE-p39) <.> CVE Name: CVE-XXXX-XXXX <.> For general information regarding FreeBSD Security Advisories, including descriptions of the fields above, security branches, and the following sections, please visit http://www.FreeBSD.org/security/. I. Background <.> II. Problem Description <.> III. Impact <.> IV. Workaround <.> V. Solution <.> VI. Correction details <.> VII. References <.> .... <.> The `Topic` field indicates exactly what the problem is. It is basically an introduction to the current security advisory and notes the utility with the vulnerability. <.> The `Category` refers to the affected part of the system which may be one of `core`, `contrib`, or `ports`. The `core` category means that the vulnerability affects a core component of the FreeBSD operating system. The `contrib` category means that the vulnerability affects software contributed to the FreeBSD Project, such as sendmail. Finally the `ports` category indicates that the vulnerability affects add on software available as part of the Ports Collection. <.> The `Module` field refers to the component location, for instance `sys`. In this example, we see that the module, `sys`, is affected; therefore, this vulnerability affects a component used within the kernel. <.> The `Announced` field reflects the date said security advisory was published, or announced to the world. This means that the security team has verified that the problem does exist and that a patch has been committed to the FreeBSD source code repository. <.> The `Credits` field gives credit to the individual or organization who noticed the vulnerability and reported it. <.> The `Affects` field explains which releases of FreeBSD are affected by this vulnerability. For the kernel, a quick look over the output from `ident` on the affected files will help in determining the revision. For ports, the version number is listed after the port name in [.filename]#/var/db/pkg#. If the system does not sync with the FreeBSD CVS repository and rebuild daily, chances are that it is affected. <.> The `Corrected` field indicates the date, time, time offset, and release that was corrected. Reserved for the identification information used to look up vulnerabilities in the Common Vulnerabilities Database system. <.> The `Background` field gives information on exactly what the affected utility is. Most of the time this is why the utility exists in FreeBSD, what it is used for, and a bit of information on how the utility came to be. <.> The `Problem Description` field explains the security hole in depth. This can include information on flawed code, or even how the utility could be maliciously used to open a security hole. <.> The `Impact` field describes what type of impact the problem could have on a system. For example, this could be anything from a denial of service attack, to extra privileges available to users, or even giving the attacker superuser access. <.> The `Workaround` field offers a feasible workaround to system administrators who may be incapable of upgrading the system. This may be due to time constraints, network availability, or a slew of other reasons. Regardless, security should not be taken lightly, and an affected system should either be patched or the security hole workaround should be implemented. <.> The `Solution` field offers instructions on patching the affected system. This is a step by step tested and verified method for getting a system patched and working securely. <.> The `Correction Details` field displays the CVS branch or release name with the periods changed to underscore characters. It also shows the revision number of the affected files within each branch. <.> The `References` field usually offers sources of other information. This can include web URLs, books, mailing lists, and newsgroups. [[security-accounting]] == Process Accounting Process accounting is a security method in which an administrator may keep track of system resources used, their allocation among users, provide for system monitoring, and minimally track a user's commands. This indeed has its own positive and negative points. One of the positives is that an intrusion may be narrowed down to the point of entry. A negative is the amount of logs generated by process accounting, and the disk space they may require. This section will walk an administrator through the basics of process accounting. === Enable and Utilizing Process Accounting Before making use of process accounting, it must be enabled. To do this, execute the following commands: [source,shell] .... # touch /var/account/acct # accton /var/account/acct # echo 'accounting_enable="YES"' >> /etc/rc.conf .... Once enabled, accounting will begin to track CPU stats, commands, etc. All accounting logs are in a non-human readable format and may be viewed using the man:sa[8] utility. If issued without any options, `sa` will print information relating to the number of per user calls, the total elapsed time in minutes, total CPU and user time in minutes, average number of I/O operations, etc. To view information about commands being issued, one would use the man:lastcomm[1] utility. The `lastcomm` may be used to print out commands issued by users on specific man:ttys[5], for example: [source,shell] .... # lastcomm ls trhodes ttyp1 .... Would print out all known usage of the `ls` by `trhodes` on the ttyp1 terminal. Many other useful options exist and are explained in the man:lastcomm[1], man:acct[5] and man:sa[8] manual pages. diff --git a/documentation/content/it/books/handbook/security/_index.adoc b/documentation/content/it/books/handbook/security/_index.adoc index e57891ac4c..57c8b7e3aa 100644 --- a/documentation/content/it/books/handbook/security/_index.adoc +++ b/documentation/content/it/books/handbook/security/_index.adoc @@ -1,318 +1,318 @@ --- title: Capitolo 14. Sicurezza part: Parte II. Compiti Ordinari prev: books/handbook/users next: books/handbook/jails showBookMenu: true weight: 18 path: "/books/handbook/" --- [[security]] = Sicurezza :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :sectnumoffset: 14 :partnums: :source-highlighter: rouge :experimental: :images-path: books/handbook/security/ ifdef::env-beastie[] ifdef::backend-html5[] :imagesdir: ../../../../images/{images-path} endif::[] ifndef::book[] include::shared/authors.adoc[] include::shared/mirrors.adoc[] include::shared/releases.adoc[] include::shared/attributes/attributes-{{% lang %}}.adoc[] include::shared/{{% lang %}}/teams.adoc[] include::shared/{{% lang %}}/mailing-lists.adoc[] include::shared/{{% lang %}}/urls.adoc[] toc::[] endif::[] ifdef::backend-pdf,backend-epub3[] include::../../../../../shared/asciidoctor.adoc[] endif::[] endif::[] ifndef::env-beastie[] toc::[] include::../../../../../shared/asciidoctor.adoc[] endif::[] [[security-synopsis]] == Sinossi Questo capitolo dà un'introduzione di base sui concetti dei sistemi di sicurezza, alcune buone regole di comportamento e alcuni argomenti avanzati per FreeBSD. Molti degli argomenti qua trattati possono essere applicati anche ai sistemi e alla sicurezza su Internet in generale. Internet non è più il luogo "amichevole" dove ognuno vuole essere il tuo gentile vicino. Mettere in sicurezza il tuo sistema è un imperativo per la protezione dei tuoi dati, della tua proprietà intelletuale, del tuo tempo e molto altro dalla mano di hacker e simili. FreeBSD dà un insieme di utility e di meccanismi per assicurare l'integrità e la sicurezza del tuo sistema e della tua rete. Dopo la lettura di questo capitolo, conoscerai: * Concetti di base dei sistemi di sicurezza, rispetto a FreeBSD. * Vari meccanismi di crittografia disponibili in FreeBSD, come DES e MD5. * Come configurare l'autenticazione OTP (password a singolo uso). * Come configurare i TCP Wrapper per l'uso con `inetd`. * Come configurare KerberosIV su FreeBSD. * Come configurare Kerberos5 su FreeBSD 5.0 o successivi. * Come configurare IPsec e creare una VPN tra macchine FreeBSD/Windows(R). * Come configurare e usare OpenSSH, l'implementaizone SSH usata da FreeBSD. * Cosa sono le ACL del file system e come usarle. * Come usare l'utility Portaudit per monitorare i pacchetti software di terze parti installati dalla Ports Collection. * Come utilizzare le pubblicazioni sugli avvisi di sicurezza di FreeBSD. * Avere un'idea di cosa sia il Process Accounting e come abilitarlo su FreeBSD. Prima di leggere questo capitolo dovresti: * Capire concetti base di FreeBSD e Internet. Altri argomenti inerenti la sicurezza sono trattati in altre parti di questo libro. Ad esempio i meccanismy di MAC sono discussi in crossref:mac[mac,Controllo di Accesso Vincolato] e la gestione dei firewall in crossref:firewalls[firewalls,Firewall]. [[security-intro]] == Introduzione La sicurezza è una funzione che inizia e finisce con l'amministratore di sistema. Nonostante ogni sistema multi-utente UNIX(R) BSD abbia della sicurezza insita, il lavoro di costruire e mantenere meccanismi di sicurezza aggiuntivi in modo da mantenere "onesti" gli utenti è probabilmente uno dei maggiori lavori di un amministratore di sistema. La macchine sono sicure solo quanto le si rende e le richieste di sicurezza si scontrano sempre con l'umana necessità per la comodità. I sistemi UNIX(R), in generale, sono capaci di eseguire un gran numero di processi contemporanei e ognuno di questi processi opera come server - nel senso che entità esterne possono connettersi e parlarci. Mentre i mini e i mainframe di ieri diventano i desktop di oggi, mentre i computer diventano interconnessi e internet-connessi, la sicurezza diventa un problema sempre maggiore. La sicurezza di un sistema riguarda anche il gestire varie forme di attacco, compresi attacchi che tentano di bloccare, o comunque rendere inusabile, il sistema, anche se non necessariamente cercano di compromettere l'account di root `root` ("rompere root"). I problemi di sicurezza possono essere suddivisi in svariate categorie: . Attacchi che limitano la disponibilità dei servizi ("Denial of service" o, in breve, DoS). . Compromissione degli account utente. . Compromissione di root tramite server accessibili. . Compromissione di root tramite gli account utente. . Crazione di backdoor (letteralmente "porte sul retro", ovvero accessi secondari personalizzati). Un attacco DoS è un'azione che priva la macchina di risorse. Tipicamente un attacco DoS è un meccanismo a forza-bruta che tenta di bloccare e comunque rendere inusabile una macchina travolgendo di richieste i server che rende disponibili o direttamente lo stack di rete. Alcuni attacchi DoS tentano di trarre vantaggio da bug nello stack di rete per bloccare la macchina con un singolo pacchetto. Questo genere di attacchi può evitato solo mettendo a posto il bug direttamente nel kernel. Gli attacchi sui server possono spesso essere evitati specificando con attenzione dei limiti sul carico che i server stessi devono accettare in caso che il sistema lavori in condizioni avverse. Gli attacchi a forza-bruta generati da un'intera rete di attaccanti sono più difficili da gestire. Ad esempio un attacco con pacchetti in spoof (ovvero con il campo mittente falsato) è praticamente impossibile da fermare, a meno di staccare del tutto il sistema da Internet. Potrà anche non fermare la tua macchina, ma sicuramente può saturare la tua connessione Internet. La compromissione di un account utente è ancora più comune di un attacco DoS. Molti sysadmin usano ancora i server standard telnetd, rlogind, rshd e ftpd sulle loro macchine. Questi programmi, normalmente, non usano connessioni crittate. Il risultato è che quando hai una base utenti di medie dimensioni, uno o più degli utenti connessi al tuo sistema da remoto (il modo più comune e conveniente per collegarsi a un sisetma) avrà una password compromessa da un'operaizone di sniffing. Gli amministratori di sistema attenti controllano i registri degli accessi remoto cercando indirizzi sospetti anche tra gli accessi permessi. Bisogna sempre dare per scontato che una volta che un attaccante ha accesso ad un account utente, può rompere anche `root`. In realtà, comunque, in un sistema ben configurato e mantenuto, questo non è necessariamente vero. La distinzione è importante perché senza accesso a `root` l'attaccante in genere non può nascondere le proprie tracce e può, alla peggio, rovinare i file dell'utente o mandare la macchina in crash. La compromissione degli account utente è molto comune dato che gli utenti tendono a non prendere precauzioni tanto quanto gli amministratori di sistema. Gli amministratori di sistema devono ricordare che su una macchina ci sono potenzialmente molti modi per rompere `root`. L'attaccante potrebbe conoscere la password di `root`, potrebbe trovare un bug in un programma server in esecuzione con diritti di `root` e sfruttarlo per entrare da remoto, oppure una volta ottenuto un account utente potrebbe fare lo stesso con un bug in un programma con suid `root`. Se un attaccante rompe `root` su una macchina, potrebbe non aver bisogno di installare una backdoor. Molti dei buchi per l'accesso come `root` trovati (e chiusi) fino ad oggi richiedono un considerevole lavoro da parte dell'attaccante per pulire le tracce lasciate, quindi molti attaccanti installano delle backdoor. Una backdoor dà all'attaccante un modo semplice per riottenere accesso `root` al sistema, ma danno anche un modo semplice per individuare l'intrusione, all'amministratore di sistema furbo. Rendere impossibile installare backdoor all'attaccante potrebbe in realtà diminuire la sicurezza del sistema, dato che comunque non chiuderà il buco che l'attaccante ha trovato la prima volta. Le soluzioni di sicurezza devono sempre essere implementate con un approccio multi-strato a "cipolla" e possono essere categorizzate come segue: . Rendere sicuro `root` e gli account dello staff. . Rendere sicuri i server e i binari suid/sgid in esecuzione come `root`. . Rendere sicuri gli account utente. . Rendere sicuro il file delle password. . Rendere sicuro il nucleo del kernel, i device raw e il file system. . Individuazione rapida delle modifiche non appropriate fatte al sistema. . Paranoia. La prossima sezione di questo capitolo coprirà questi punti in maggior dettaglio. [[securing-freebsd]] == Rendere sicuro FreeBSD [NOTE] .Comandi o Protocolli ==== In questo documento useremo testo grassetto per riferirci ad applicazioni e testo `a spaziatura fissa` per riferirci a specifici comandi. I protocolli useranno un font normale. Questa distinzione tipografica è utile per casi come ssh, che è un protocollo oltre che un comando. ==== Le sezioni seguenti descrivono i metodi per rendere sicuro il vostro sistema FreeBSD che sono stati menzionati nella <> di questo capitolo. [[securing-root-and-staff]] === Rendere sicuro `root` e gli account dello staff. Innanzitutto, non preoccuparti di rendere sicuri gli account di staff se non hai reso sicuro l'account `root`. La maggior parte dei sistemi hanno una password assegnata per l'account `root`. La prima cosa che devi dare per assunta è che la password è _sempre_ compromessa. Questo non significa che devi togliere la password; la password è quasi sempre necessaria per l'accesso dalla console della macchina. Quello che questo significa è che non dovresti render possibile l'uso di questa password tranne che da console e possibilmente neanche dal comando man:su[1]. Per esempio, assicurati che le tue pty siano specificate come `insecure` nel file [.filename]#/etc/ttys# in modo che accessi diretti `root` tramite `telnet` o `rlogin` non siano permessi. Se usi altri servizi di login come ad esempio sshd, fai in modo che accessi diretti come `root` siano vietati anche per questi. Puoi farlo modificando il file [.filename]#/etc/ssh/sshd_config# e assicurandoti che `PermitRootLogin` sia impostato a `NO`. Tieni conto di tutti i modi di accesso - servizi come ad esempio FTP vengono spesso trascurati. Login `root` diretti dovrebbero essere permessi solo tramite la console di sistema. Ovviamente, come sysadmin (amministratore di sistema) hai bisogno di accesso a `root`, quindi apriremo alcuni passaggi; ci assicureremo però che questi passaggi richiedano ulteriori verifiche di password per funzionare. Un modo per accedere a `root` è aggiungere gli appropriati account di staff al gruppo `wheel` (in [.filename]#/etc/group#). I membri del gruppo `wheel` possono accedere a `root` tramite `su`. Non dovresti mai dare ai membri dello staff accesso nativo al gruppo `wheel` mettendoli in quel gruppo nel file [.filename]#/etc/passwd#; dovresti metterli nel gruppo `staff` e quindi aggiungerli al gruppo `wheel` tramite il file [.filename]#/etc/group#. Solo i membri dello staff che hanno effettivo bisogno di accesso a `root` dovrebbero essere nel gruppo `wheel` group. Altra possibilità, quando si utilizzi Kerberos come metodo di autenticazione, ` quella di utilizzare il file [.filename]#.k5login# dell'account `root` in modo da permettere l'accesso a `root` tramite man:ksu[1] senza bisogno di mettere nessuno nel gruppo `wheel`. Questa potrebbe essere la soluzione migliore dato che il meccanismo `wheel` permette all'attaccante di diventare `root` se è riuscito ad ottenere accesso ad un account di staff. Benché il meccanismo `wheel` sia meglio di niente, non è necessariamente la soluzione più sicura. Un metodo indiretto per rendere sicuri gli account di staff e quindi l'accesso a `root` è quello di eseguire l'operazione nota come "starring" delle password cifrate. password for the staff accounts: utilizzando il comando man:vipw[8] si può rimpiazzare ogni password cifrata con un singolo carattere "`*`" (asterisco, in inglese "star"). Questo comando aggiorna il file [.filename]#/etc/master.passwd# e il database utenti/password in modo da disabilitare i login autenticati da password. Un account di staff come il seguente: [.programlisting] .... foobar:R9DT/Fa1/LV9U:1000:1000::0:0:Foo Bar:/home/foobar:/usr/local/bin/tcsh .... Andrebbe modificato così: [.programlisting] .... foobar:*:1000:1000::0:0:Foo Bar:/home/foobar:/usr/local/bin/tcsh .... Questo previene i normali login dato che la password cifrata non sarà mai "`*`". Fatto questo i membri dello staff dovranno utilizzare un diverso meccanismo di autenticazione come ad esempio man:kerberos[1] o man:ssh[1] utilizzando una coppia di chiavi pubblica/privata. Utilizzando Kerberos bisogna generalmente rendere sicure sia le macchine su cui viene eseguito il server Kerberos che la propria workstation. Utilizzando una coppia di chiavi bisogna in generale rendere sicura la macchina _da cui_ ci si sta collegando (in genere la propria workstation); si può aggiungere un ulteriore strato di protezione proteggendo la coppia di chiavi con una password all'atto della creazione con man:ssh-keygen[1]. Eseguire lo "starring" degli account dello staff garantisce che questi possano eseguire il login solo tramite i metodi di accesso sicuri che sono stati configutati. Quest forze l'intero staff all'uso di connessioni sicure e cifrate in tutte le loro sessioni, chiudendo un importante falla di sicurezza utilizzata da molti attaccanti: ascoltare il traffico di rete da un'altra macchina meno sicura. I meccanismi di sicurezza più indiretti assumono anche che ci si colleghi da un server più restrittivo a uno che lo è di meno; per esempio se il tuo server primario ha in esecuzione una grande varietà di servizi, la tua workstation non dovrebbe averne in esecuzione nessuno. Per fare in modo che la tua workstation sia ragionevolmente sicura dovresti eseguire meno servizi possibile, o perfino nessuno del tutto, e dovresti utilizzare uno screen saver protetto da password. Ovviamente, avendo accesso fisico alla workstation un attaccante può rompere qualsiasi protezione che tu possa aver importato, ma bisogna sempre considerare che la magior parte degli attacchi avviene remotamente, tramite una rete, da parte di persone che non hanno accesso fisico alle tue workstation o ai tuoi server. L'uso di sistemi come Kerberos permette di disabilitare o cambiare la pasword ad un account di staff in un solo posto ed avere effeto immediato su tutte le macchine in cui il membro dello staff ha un account. Nel caso l'account di un membro dello staff venga compromesso, la possibilità di poter cambiare la sua password su tutte le macchine non ` cosa di poco conto. Con password separate, cambiare una password su molte macchine può essere un bel problema. Con Kerberos puoi anche imporre restrizioni di cambio password: non solo un ticket Kerberos può essere fatto per scadere dopo un tempo predeterminato, ma il sistema Kerberos può richiedere all'utente di scegliere una nuova passsword dopo un certo periodo di tempo (per esempio, una volta al mese). === Rendere sicuri i server Root e i binari SUID/SGID Il sysadmin prudente esegue soltanto i server che gli sono necessari, ná di più né di meno. Bisogna tenere conto del fatto che i server di terze parti sono generalmente i più affetti da bug. Per esempio, utilizzare una versione obsoleta di imapd o popper è equivalente a dare accesso `root` al mondo intero. Non eseguire mai un server senza controllarlo accuratamente. Molti server non hanno bisogno di essere eseguiti come `root`. Per esempio i demoni ntalk, comsat e finger possono essere eseguiti in speciali _sandbox_ utente. Difficilmente una sandbox sarà una soluzione completa del problema, a meno di dedicarci parecchio tempo, ma resta valido l'approccio a cipolla alla sicurezza: se qualcuno riesce ad irrompere in un server eseguito in una sandbox, deve ancora riuscire ad evadere da quest'ultima. Più strati l'attaccante deve superare, minore la sua probabilità di successo. Storicamente sono state trovate falle di accesso a root in virtualmente ogni server mai eseguito come `root`, inclusi i server del sistema base. Se hai una macchina alla quale la gente accede solamente tramite sshd e mai tramite telnetd o rshd o rlogind, allora disattiva questi servizi! FreeBSD attualmente esegue per default ntalkd, comsat e finger in una sandbox. Un altro programma candidato ad essere eseguito in una sandbox è man:named[8]. [.filename]#/etc/defaults/rc.conf# comprende le opzioni necessarie per eseguire named in una sandbox in forma comentata. A seconda se state installando un nuovo sistema o aggiornando un sistema esistente, gli speciali account utente utilizzati da queste sandbox potrebbero non essere presenti. Il sysadmin prudente dovrebbe cercar di utilizzare delle sandbox per i server ogniqualvolta possibile. Esiste un certo numero di altri servizi che generalmente non vengono eseguiti in una sandbox: sendmail, popper, imapd, ftpd e altri. Ci sono software alternativi ad alcuni di questi ma installarli potrebbe richiedere più lavoro di quello che si intende dedicargli (il fattore convenienza colpisce ancora). Potresti dover eseguire questi servizi come `root` ed affidarti ad altri meccanismi per individuare le intrusioni che potrebbero essere fatte attraverso questi. L'altra grande potenziale fonte di falle per l'accesso a `root` sono i binari suid-root e sgid installati nel sistema, come ad esempio rlogin, nelle directory [.filename]#/bin#, [.filename]#/sbin#, [.filename]#/usr/bin# o [.filename]#/usr/sbin#. Benché niente sia sicuro al 100%, i binari suid e sgid presenti nel sistema per default possono essere considerati ragionevolmente sicuri. In ogni caso, delle falle da `root` sono occasionalmente trovate anche in questi. Nel 1998 è stata trovata una falla da `root` in `Xlib` che rendeva vulnerabile xterm (che tipicamente è suid). It is better to be safe than sorry and the prudent sysadmin will restrict suid binaries, that only staff should run, to a special group that only staff can access, and get rid of (`chmod 000`) any suid binaries that nobody uses. A server with no display generally does not need an xterm binary. Sgid binaries can be almost as dangerous. If an intruder can break an sgid-kmem binary, the intruder might be able to read [.filename]#/dev/kmem# and thus read the encrypted password file, potentially compromising any passworded account. Alternatively an intruder who breaks group `kmem` can monitor keystrokes sent through ptys, including ptys used by users who login through secure methods. An intruder that breaks the `tty` group can write to almost any user's tty. If a user is running a terminal program or emulator with a keyboard-simulation feature, the intruder can potentially generate a data stream that causes the user's terminal to echo a command, which is then run as that user. [[secure-users]] === Rendere sicuri gli account utente Gli account utente sono generalmente i più difficili da rendere sicuri. Bench*eacute; tu possa imporre restrizioni d'accesso allo staff ed eseguire lo "starring" delle loro password, potresti non poter farlo con l'account di un generico utente. Se hai sufficiente controllo potesti farcela e rendere gli account utente sufficientemente sicuri, altrimenti dovrai essere più vigile nel controllo di questi account. L'uso di ssh e Kerberos per gli account utente è più problematico, a causa del maggiore supporto amministrativo e tecnico richiesto, ma è sempre un'ottima soluzione se confrontata all'uso di un file password cifrato. === Rendere sicuro il file password L'unica strada sicura è quella di eseguire lo starring so più password possibile e utilizzare ssh o Kerberos per accedere a quegli account. Anche se il file di password cifrato ([.filename]#/etc/spwd.db#) può essere letto solo da `root`, potrebbe essere possibile per un attaccante ottenere accesso in lettura a quel file anche senza aver ottenuto accesso in scrittura. I tuoi script di sicurezza dovrebbero sempre verificare che il file password non venga modificato e in caso riportarlo ad un amministratore (cfr. la sezione <> sottostante). === Rendere sicuri il kernel, i raw device e i file system Quando un attaccante irrompe nell'account di `root` può fare qualsiasi cosa, ma alcune cose sono più comode di altre. Per esempio, la maggior parte dei kernel moderni comprende un device per l'ascolto dei pacchetti di rete. In FreeBSD questo device si chiama [.filename]#bpf#. Un intrusore generalmente cercherà di ascoltare i pacchetti delle reti a cui la macchina compromessa è collegata. Non ò obbligatorio dare all'intrusore questa possibilità e d'altro canto la maggior parte dei sistemi non ha bisogno di avere il device [.filename]#bpf#. Anche nel caso di aver disattivato il device [.filename]#bpf#, bisogna comunque preoccuparsi di [.filename]#/dev/mem# e [.filename]#/dev/kmem#; tra l'altro l'intrusore ha anche la possibilità di scrivere sui device disco raw o utilizzare il comando di caricamento moduli del kernel, man:kldload[8]. Un intrusore intraprendente può utilizzare un proprio modulo del kernel per l'ascolto dei pacchetti e caricarlo su un kernel in esecuzione. Per evitare questi problemi bisogna eseguire il kernel ad un livello di sicurezza più alto, almeno al livello 1. Il livello di sicurezza può essere impostato con `sysctl` modificando la variabile `kern.securelevel`. Se il livello di sicurezza è impostato ad 1, l'accesso in scrittura ai device raw sarà negato e alcuni `chflags` speciali, come ad esempio `schg`, verranno verificati. Devi anche verificare che il flag `schg` sia impostato sui binari, cartelle e script utilizzati all'avvio prima dell'impostazione del livello di sicurezza. L'uso di un livello di sicurezza superiore potrebbe essere una misura eccesiva, dato che rende l aggiornamento del sistema molto più complesso. You may compromise and run the system at a higher secure level but not set the `schg` flag for every system file and directory under the sun. Another possibility is to simply mount [.filename]#/# and [.filename]#/usr# read-only. It should be noted that being too draconian in what you attempt to protect may prevent the all-important detection of an intrusion. [[security-integrity]] === Verifica dell'integrità dei file: binari, file di configurazione, etc. TODO:When it comes right down to it, you can only protect your core system configuration and control files so much before the convenience factor rears its ugly head. For example, using `chflags` to set the `schg` bit on most of the files in [.filename]#/# and [.filename]#/usr# is probably counterproductive, because while it may protect the files, it also closes a detection window. The last layer of your security onion is perhaps the most important - detection. The rest of your security is pretty much useless (or, worse, presents you with a false sense of security) if you cannot detect potential intrusions. Half the job of the onion is to slow down the attacker, rather than stop him, in order to be able to catch him in the act. The best way to detect an intrusion is to look for modified, missing, or unexpected files. The best way to look for modified files is from another (often centralized) limited-access system. Writing your security scripts on the extra-secure limited-access system makes them mostly invisible to potential attackers, and this is important. In order to take maximum advantage you generally have to give the limited-access box significant access to the other machines in the business, usually either by doing a read-only NFS export of the other machines to the limited-access box, or by setting up ssh key-pairs to allow the limited-access box to ssh to the other machines. Except for its network traffic, NFS is the least visible method - allowing you to monitor the file systems on each client box virtually undetected. If your limited-access server is connected to the client boxes through a switch, the NFS method is often the better choice. If your limited-access server is connected to the client boxes through a hub, or through several layers of routing, the NFS method may be too insecure (network-wise) and using ssh may be the better choice even with the audit-trail tracks that ssh lays. Once you have given a limited-access box at least read access to the client systems it is supposed to monitor, you must write scripts to do the actual monitoring. Given an NFS mount, you can write scripts out of simple system utilities such as man:find[1] and man:md5[1]. It is best to physically md5 the client-box files at least once a day, and to test control files such as those found in [.filename]#/etc# and [.filename]#/usr/local/etc# even more often. When mismatches are found, relative to the base md5 information the limited-access machine knows is valid, it should scream at a sysadmin to go check it out. A good security script will also check for inappropriate suid binaries and for new or deleted files on system partitions such as [.filename]#/# and [.filename]#/usr#. When using ssh rather than NFS, writing the security script is much more difficult. You essentially have to `scp` the scripts to the client box in order to run them, making them visible, and for safety you also need to `scp` the binaries (such as find) that those scripts use. The ssh client on the client box may already be compromised. All in all, using ssh may be necessary when running over insecure links, but it is also a lot harder to deal with. A good security script will also check for changes to user and staff members access configuration files: [.filename]#.rhosts#, [.filename]#.shosts#, [.filename]#.ssh/authorized_keys# and so forth, files that might fall outside the purview of the `MD5` check. If you have a huge amount of user disk space, it may take too long to run through every file on those partitions. In this case, setting mount flags to disallow suid binaries and devices on those partitions is a good idea. The `nodev` and `nosuid` options (see man:mount[8]) are what you want to look into. You should probably scan them anyway, at least once a week, since the object of this layer is to detect a break-in attempt, whether or not the attempt succeeds. -Process accounting (see man:accton[8]) is a relatively low-overhead feature of the operating system which might help as a post-break-in evaluation mechanism. It is especially useful in tracking down how an intruder has actually broken into a system, assuming the file is still intact after the break-in has occured. +Process accounting (see man:accton[8]) is a relatively low-overhead feature of the operating system which might help as a post-break-in evaluation mechanism. It is especially useful in tracking down how an intruder has actually broken into a system, assuming the file is still intact after the break-in has occurred. Finally, security scripts should process the log files, and the logs themselves should be generated in as secure a manner as possible - remote syslog can be very useful. An intruder will try to cover his tracks, and log files are critical to the sysadmin trying to track down the time and method of the initial break-in. One way to keep a permanent record of the log files is to run the system console to a serial port and collect the information to a secure machine monitoring the consoles. === Paranoia Un po' di paranoia non fa mai male. Come regola, un sysadmin può aggiungere qualsiasi feature di sicurezza fintantoché non impattano la comodità e può aggiungerne altre _che la_ impattano, ma solo dopo averci pensato bene. Even more importantly, a security administrator should mix it up a bit - if you use recommendations such as those given by this document verbatim, you give away your methodologies to the prospective attacker who also has access to this document. === Attacchi Denial of Service Questa sezione parla degli attacchi Denial of Service, ovvero quelli atti ad interrompere i servizi in esecuzione su una macchina. Tipicamente un attacco DoS è un attacco a pacchetto; benché non si possa fare molto riguardo ad un attacco moderno che satura la vostra rete con pacchetti , si può cercare di limitare il danno assicurandosi che l'attacco non blocchi i vostri servizi, utilizzando le seguenti tecniche: . Limitare le fork dei server. . TODO:Limiting springboard attacks (ICMP response attacks, ping broadcast, etc.). . Sovraccaricare la Kernel Route Cache. Un comune scenario è l'attacco di un server che fa fork e fargli creare così tanti processi figli da esaurire le risorse della macchina, come ad esempio la memoria, i file descriptor o altri e costringerlo quindi a fermarsi. inetd (cfr. man:inetd[8]) ha molte opzioni per limitare questo tipo di attacchi. Si deve notare che benché sia possibile evitare che la macchina si fermi, non è generalmente possibile evitare che i servizi vengano resi non disponibili dall'attacco. Leggete attentamente la pagina del manuale di inetd, con particolare attenzione alle opzioni `-c`, `-C` e `-R`. Un attacco con IP aggira l'opzione `-C` quindi è bene utilizzare una combinazione di opzioni. Alcuni server indipendenti hanno meccanismi interni per la limitazione delle fork. Sendmail ha l'opzione `-OMaxDaemonChildren` che generalmente funziona molto meglio che cercare di utilizzare le funzioni di limitazione basate sul carico della macchina, a causa del ritardo di aggiornamento del valore di carico. Quando lanci sendmail dovresti specificare un parametro `MaxDaemonChildren` abbastanza alto da gestire il carico previsto , ma non così alto da non essere gestibile dal computer. È anche prudente eseguire Sendmail in modalità queued (`-ODeliveryMode=queued`) ed eseguire il demone (`sendmail -bd`) separatamente dalla gestione code (`sendmail -q15m`). Se vuoi che i messaggi vengano consegnati in tempo reale puoi utilizzare un intervallo molto più breve, come ad esempio `-q1m`, ma assicurati di utilizzare un valore `MaxDaemonChildren` adatto per _quel_ Sendmail, in modo da prevenire problemi a catena. Syslogd può essere attaccato direttamente ed è fortemente consigliato l'uso dell'opzione `-s` quando possibile, o al limite l'opzione `-a`. You should also be fairly careful with connect-back services such as TCP Wrapper's reverse-identd, which can be attacked directly. You generally do not want to use the reverse-ident feature of TCP Wrapper for this reason. È un'ottima idea quella di proteggere i servizi interni dall'accesso esterno chiudendoli tramite regole del firewall ai bordi della vostra rete. L'idea è di prevenire gli attacchi a saturazione provenienti dall'esterno della vostra rete, non tanto di proteggere i servizi da attacchi di rete atti a compromettere `root`. Utilizza sempre un firewall , ovvero "blocca tutto _tranne_ le porte A, B, C, D e M-Z"; puoi bloccare tutte le porte basse ad eccezione di specifici servizi quali named (se sei primario per una zona), ntalkd, sendmail e altri servizi accessibili da Internet. Se tu cercassi di configurare il firewall in maniera opposta (inclusivo o permissivo) c'è una buona probabilità che tu ti scordi di "chiudere" qualche servizio o che tu aggiunga un nuovo servizio interno e dimentichi di aggiornare il firewall. Puoi comunque lasciare aperte tutte le porte , permettendo un uso permissivo, senza però compromettere le porte . Nota anche che FreeBSD ti permette di controllare l'intervallo di porte utilizzate per il binding dinamico tramite vari `sysctl net.inet.ip.portrange` (`sysctl -a | fgrep portrange`), che possono semplificare la complessità di configurazione del tuo firewall. Another common DoS attack is called a springboard attack - to attack a server in a manner that causes the server to generate responses which overloads the server, the local network, or some other machine. The most common attack of this nature is the _ICMP ping broadcast attack_. The attacker spoofs ping packets sent to your LAN's broadcast address with the source IP address set to the actual machine they wish to attack. If your border routers are not configured to stomp on ping packets to broadcast addresses, your LAN winds up generating sufficient responses to the spoofed source address to saturate the victim, especially when the attacker uses the same trick on several dozen broadcast addresses over several dozen different networks at once. Broadcast attacks of over a hundred and twenty megabits have been measured. A second common springboard attack is against the ICMP error reporting system. By constructing packets that generate ICMP error responses, an attacker can saturate a server's incoming network and cause the server to saturate its outgoing network with ICMP responses. This type of attack can also crash the server by running it out of memory, especially if the server cannot drain the ICMP responses it generates fast enough. Use the sysctl variable `net.inet.icmp.icmplim` to limit these attacks. The last major class of springboard attacks is related to certain internal inetd services such as the udp echo service. An attacker simply spoofs a UDP packet with the source address being server A's echo port, and the destination address being server B's echo port, where server A and B are both on your LAN. The two servers then bounce this one packet back and forth between each other. The attacker can overload both servers and their LANs simply by injecting a few packets in this manner. Similar problems exist with the internal chargen port. A competent sysadmin will turn off all of these inetd-internal test services. Spoofed packet attacks may also be used to overload the kernel route cache. Refer to the `net.inet.ip.rtexpire`, `rtminexpire`, and `rtmaxcache sysctl` parameters. A spoofed packet attack that uses a random source IP will cause the kernel to generate a temporary cached route in the route table, viewable with `netstat -rna | fgrep W3`. These routes typically timeout in 1600 seconds or so. If the kernel detects that the cached route table has gotten too big it will dynamically reduce the `rtexpire` but will never decrease it to less than `rtminexpire`. There are two problems: . The kernel does not react quickly enough when a lightly loaded server is suddenly attacked. . The `rtminexpire` is not low enough for the kernel to survive a sustained attack. If your servers are connected to the Internet via a T3 or better, it may be prudent to manually override both `rtexpire` and `rtminexpire` via man:sysctl[8]. Never set either parameter to zero (unless you want to crash the machine). Setting both parameters to 2 seconds should be sufficient to protect the route table from attack. === Access Issues with Kerberos and SSH There are a few issues with both Kerberos and ssh that need to be addressed if you intend to use them. Kerberos 5 is an excellent authentication protocol, but there are bugs in the kerberized telnet and rlogin applications that make them unsuitable for dealing with binary streams. Also, by default Kerberos does not encrypt a session unless you use the `-x` option. ssh encrypts everything by default. Ssh works quite well in every respect except that it forwards encryption keys by default. What this means is that if you have a secure workstation holding keys that give you access to the rest of the system, and you ssh to an insecure machine, your keys are usable. The actual keys themselves are not exposed, but ssh installs a forwarding port for the duration of your login, and if an attacker has broken `root` on the insecure machine he can utilize that port to use your keys to gain access to any other machine that your keys unlock. We recommend that you use ssh in combination with Kerberos whenever possible for staff logins. Ssh can be compiled with Kerberos support. This reduces your reliance on potentially exposed ssh keys while at the same time protecting passwords via Kerberos. Ssh keys should only be used for automated tasks from secure machines (something that Kerberos is unsuited to do). We also recommend that you either turn off key-forwarding in the ssh configuration, or that you make use of the `from=IP/DOMAIN` option that ssh allows in its [.filename]#authorized_keys# file to make the key only usable to entities logging in from specific machines. [[crypt]] == DES, MD5 e Crypt Ogni utente su un sistema UNIX(R) ha una password associata con il proprio account. È pvviamente necessario che queste password siano note solamente all'utente e al sistema operativo vero e proprio. Per poter mantenere segrete queste password, sono cifrate con quello che si chiama un "one-way hash", ovvero possono essere facilmente cifrate ma non decifrate. In altre parole, quel che poco fa abbiamo dato per ovvio non è neanche vero: il sistema operativo in sé non conosce _realmente_ la password, conosce soltanto la forma _cifrata_ della password. L'unico modo per ricavare la password _in chiaro_ è una brutale ricerca nell'intero spazio delle password possibili. Sfortunatamente l'unico modo sicuro di cifrare le password quando UNIX(R) è nato era di utilizzare DES (Data Encryption Standard). Questo non era un grosso problema per gli utenti residenti in USA, ma dato che il codice sorgente riguardante DES non poteva essere esportato al di fuori degli USA, FreeBSD ha dovuto cercare un modo per poter contemporaneamente essere in regola con la legge USA e mantenere la compatibilità con tutte le altre varianti UNIX(R) che ancora utilizzavano DES. La soluzione è stata quella di suddividere le librerie di cifratura in modo tale che gli utenti USA potessero installare le librerie DES ed utilizzarlo ma gli utenti internazionali avessero comunque a disposizioni metodi crittografici che potessero essere esportati all'estero. Questo è il modo in cui FreeBSD adottò MD5 come metodo di cifratura di default. MD5 è considerato più sicuro di DES, quindi installare DES è una possibilità pensata principalmente per motivi di compatibilià. === Riconoscere il funzionamento del tuo crypt Attualmente la libreria supporta gli algoritmi DES, MD5 e Blowfish. Per default FreeBSD utilizza MD5 per cifrare le password. È piuttosto semplice identificare il tipo di cifratura utilizzato; ad esempio uno dei metodi è di leggere il file [.filename]#/etc/master.passwd#. Le password cifrate con l'hash MD5 sono più lunghe e iniziano con i caratteri `$1$`. Le password che iniziano con `$2a$` sono cifrate con Blowfish. Le password DES non hanno alcun carattere identificativo particolare, ma sono più corte e sono codificate in un alfabeto di 64 caratteri che non include il `$`, quindi una stringa relativamente corta che non inizia con un simbolo di dollaro è molto probabilmente una password DES. Il formato utilizzato per le nuove password è deciso dal valore del campo `passwd_format` nel file [.filename]#/etc/login.conf#, che può avere i valori di `des`, `md5` oo `blf`. Fai riferimento alla pagina del manuale man:login.conf[5] per avere ulteriori informazioni sulle configurazioni di login. [[one-time-passwords]] == Password One-time Per default FreeBSD include il supporto per OPIE (One-time Passwords In Everything), configurato per utilizzare l'hash MD5. Ci sono tre tipi di diverse password di cui parleremo in seguito. Le prime sono le normali pasword UNIX(R) o Kerberos, che verranno chiamate "password UNIX(R)". Il secondo tipo sono le password one-time generate dal programma OPIE man:opiekey[1] e accettate dal programma man:opiepasswd[1] e dal prompt di login, che chiameremo "password one-time". L'ultimo tipo di password è la password segreta che darai al programma `opiekey` (e qualche volte al programma `opiepasswd`) e che viene utilizzata per generare le password one-time, che chiameremo "password segreta" o più semplicemente "password". La password segreta non ha niente a che vedere con la password UNIX(R); possono essere uguali ma questo è sconsigliato. Le password segrete di OPIE non sono limitate a 8 caratteri come le vecchie password UNIX(R), possono essere lunghe quanto ti pare. Sono abbastana diffuse password composte da frasi di sei o sette parole. Per la maggior parte, il sistema OPIE funziona in modo totalmente indipendente dal sistema di password UNIX(R). Oltre alla password, ci sono altre due informazioni utili a OPIE. Una è nota come "seme" o "chiave" e consiste di due lettere e cinque numeri. L'altra è nota come "numero di iterazioni" ed è un valore tra 1 e 100. OPIE crea la password one-time concatenando il seme e la password segreta ed applicandovi l'hash MD5 tante volte quanto specificate dal numero di iterazioni, trasformando poi il risultato in sei corte parole inglesi, che saranno la tua password one-time. Il sistema di autenticazione (principalmente PAM) mantiene traccia dell'ultima password one-time usata e autentica l'utente se l'hash della password fornita dall'utente è uguale alla password precedente. Dato che viene usato un hash, ovvero una funzione matematica a senso unico è impossibile generare password one-time future se viene catturata una password durante il suo utilizzo; il numero di iterazioni viene decrementato dopo un login avvenuto con successo per mantenere l'utente e il programma di login in sincrono. Quando il numero di iterazioni scende a 1, OPIE deve essere reinizializzato. Nelle seguenti spiegazioni si farà riferimento a vari programmi: il programma `opiekey` richiede un numero di iterazioni, un seme e una password segreta e genera una password one-time o una lista di password one-time consecutive; il programma `opiepasswd` viene utilizzato per inizializzzare OPIE e per cambiare password, numeri di iterazioni, semi e password one-time; il programma `opieinfo` analizza i file di credenziali ([.filename]#/etc/opiekeys#) e stampa il numero di iterazioni e il seme correnti dell'utente che lo richiama. Traduzione in corso [[tcpwrappers]] == TCP Wrappers Traduzione in corso [[kerberosIV]] == KerberosIV Traduzione in corso [[kerberos5]] == Kerberos5 Traduzione in corso [[openssl]] == OpenSSL Traduzione in corso [[ipsec]] == IPsec Traduzione in corso [[openssh]] == OpenSSH [[security-ssh-tunneling]] === SSH Tunneling Traduzione in corso [[fs-acl]] == File System Access Control Lists Traduzione in corso [[security-portaudit]] == Monitoring Third Party Security Issues Traduzione in corso [[security-advisories]] == FreeBSD Security Advisories Traduzione in corso [[security-accounting]] == Process Accounting Traduzione in corso diff --git a/website/content/en/status/report-2017-07-2017-09.html b/website/content/en/status/report-2017-07-2017-09.html index af02ff0be7..6ca1fdf2b0 100644 --- a/website/content/en/status/report-2017-07-2017-09.html +++ b/website/content/en/status/report-2017-07-2017-09.html @@ -1,1122 +1,1122 @@ FreeBSD Quarterly Status Report
Skip site navigation (1) Skip section navigation (2)

Introduction

This quarter's FreeBSD developments continue to provide excitement and promise for further developments. I myself have a soft spot for manual pages, so it is especially good to see that we have gained some documentation for writing them (and I hope that this will translate to more and improved manual pages in the future!). The core@ entry is also of particular note, with the introduction of the FCP process and the recognition of the first non-committer FreeBSD Project Member (and more). Read on to find out more about these, as well as improved support for the AMD Zen family of processors (e.g., Ryzen), and a whole lot more!

—Benjamin Kaduk

The deadline for submissions covering the period from October to December 2017 is January 14, 2017.

FreeBSD Team Reports

Projects

Kernel

Architectures

Userland Programs

Ports

Documentation

Third-Party Projects


    FreeBSD Team Reports

    Entries from the various official and semi-official teams, as found in the Administration Page.


    FreeBSD Release Engineering Team

    Links
    FreeBSD11.1-RELEASE Announcement URL: https://www.FreeBSD.org/releases/11.1R/announce.html
    FreeBSD10.4-RELEASE Schedule URL: https://www.FreeBSD.org/releases/10.4R/schedule.html
    FreeBSD Development Snapshots URL: https://download.FreeBSD.org/ftp/snapshots/ISO-IMAGES/

    Contact: FreeBSDRelease Engineering Team <re@FreeBSD.org>

    The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes, and maintaining the respective branches, among other things.

    The FreeBSD Release Engineering Team continued finalizing the 11.1-RELEASE cycle, with the final release builds starting on July 21 and the official release announcement email sent on July 26. Thank you to everyone who helped test 11.1-RELEASE, ensuring its quality and stability.[1]

    FreeBSD11.1-RELEASE is the second release from the stable/11 branch.

    Additionally, the FreeBSD Release Engineering Team started the 10.4-RELEASE cycle, with the code slush starting on July 28. With the final release build expected to start on September 29 and the official announcement overlapping the end of the quarter, everything is on schedule as of this writing.[2]

    FreeBSD10.4-RELEASE will be the fifth release from the stable/10 branch, and is planned to be the final release of the 10.x series.

    This project was sponsored by The FreeBSD Foundation[1].

    This project was sponsored in part by The FreeBSD Foundation[2].

    Ports Collection

    Links
    About FreeBSD Ports URL: https://www.FreeBSD.org/ports/
    Contributing to Ports URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html
    FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/index.html
    Ports Management Team Website URL: https://www.freebsd.org/portmgr/index.html
    FreeBSD portmgr on Twitter (@freebsd_portmgr) URL: https://twitter.com/freebsd_portmgr/
    FreeBSD Ports Management Team on Facebook URL: https://www.facebook.com/portmgr
    FreeBSD Ports Management Team on Google+ URL: https://plus.google.com/communities/108335846196454338383

    Contact: Ren Ladan <portmgr-secretary@FreeBSD.org>
    Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

    The Ports Collection now features over 31,600 ports. There are currently 2671 problem reports, of which 718 are unassigned. This quarter saw almost 5,900 commits from 175 committers. The number of open PRs grew compared to last quarter, and outpaced the number of changes.

    This quarter, we welcomed Zach Leslie (zleslie@), Luca Pizzamiglio (pizzamig@), Craig Leres (leres@), Adriaan de Groot (adridg@), and Dave Cottlehuber (dch@) as new committers. The commit bits of the following committers were taken in for safekeeping: alonso@ after 19 months of inactivity, rpaulo@ per his request, and ache@ after he passed away. Despite several tries and changing mentors, kami@ lacked interest in completing his mentorship, so his commit bit was also taken in for safekeeping.

    On the infrastructure side, two USES values were removed because they outlived their usefulness:

    • execinfo: libexecinfo is now available in the base system of all supported FreeBSD versions
    • twisted: there is only one Twisted port left

    The default version of GCC was bumped from 5 to 6. Firefox was updated to version 56.0 and Chromium to version 61.0.3163.100. The version of pkg itself was updated to 1.10.1.

    During this quarter, antoine@ performed 28 exp-runs to test version updates of major ports, improving USE_GITHUB and SHEBANG_FILES, and API changes to the base system. This quarter, the foundation for ports "flavors" was committed, though more development and testing will be performed in the coming quarter before it goes live.

    Open tasks:

    1. The PR load needs more attention, as the number of open PRs has started to increase again.

    The FreeBSD Core Team

    Contact: FreeBSD Core Team <core@FreeBSD.org>

    The new "FreeBSD Community Process" was drafted during BSDCan earlier this year. The first such document, FCP 0, defines how the whole process works. After some time for discussion and revision, FCP 0 was voted on and accepted by core, following the procedure laid down within that document. Currently the use of FCPs is entirely optional; we shall see how the community begins to adopt their usage and evolve the process based on experience.

    A draft update to the Code of Conduct has been prepared by the advisory committee. Core is currently reviewing the text, and will soon vote on accepting it. Core is keen to avoid the trap of "rules lawyering". At the moment, the feeling is that we need to add a preamble to the CoC to articulate the goals of the project and to act as a general guide to the exercise of the code.

    This quarter has been quite a busy one concerning changes to the roster of committers and project members. We have elected our first new Project Member: John Hixson, who will be familiar from many conferences where he has given presentations and ably represented iXsystems. A second proposed Project Member was not accepted by core, but only because core felt that Fedor Uporov really deserved a commit bit instead.

    In addition to Fedor Uporov, please also welcome (in no particular order) Matt Joras, Marcin Wojtas, Chuck Tuffli, Ilya Bakulin and Alex Richardson as brand-new committers. We have also awarded Steven Hurd and Eugene Grosbein src commit bits to go with their existing ports bits. Welcome back Gordon Tetlow as a src committer, essential for his new role within secteam. Eric Davis and Rui Paulo have both decided to hang up their commit bits: we wish them well in their future endeavours. Finally, we must report the sad death of Andrey Chernov, who will be sorely missed by his colleagues and collaborators.

    Andrey's death has highlighted another question which is only going to become more complex over time. Keeping track of copyrights is already hard enough within a mature source tree with many contributors, such as the FreeBSD sources. Now we need to consider trying to keep track of the heirs and beneficiaries of contributors who have sadly passed away. Core will consult with the Foundation legal team to discuss possible approaches to alleviate this.

    There have been complaints that the workings of Core are being kept overly confidential, and that consequently the majority of the project has too little idea of what is going on. This is certainly not intentional by Core, and we are keen to open up Core's business to more general community scrutiny as far as seems reasonable.

    Core dealt with a number of licensing questions:

    • When upstreaming patches and other original works to VirtualBox or other Oracle properties, pragmatically it works best to provide them under the terms of the MIT license (one of two opensource licenses accepted by Oracle). Of course, this only applies to work upstreamed by or with the permission of the original author.
    • The Viking software license is sufficiently BSD-like that magic constants from their drivers can be used in FreeBSD code.
    • There is no separate register of deviations from the allowed BSD-like licenses in the source tree: any code in the tree under other than BSD-like license terms can be assumed to have been approved by core.
    • At the moment the FreeBSD copyright requirement to include the copyright notice in redistributions in binary form is satisfied by making the FreeBSD sources, with all of the detailed copyright information included in the different source code files, available alongside pre-compiled system images. However, this does not necessarily meet the needs of downstream projects based on FreeBSD, and given the new "packaged base", adding per-package licensing metadata in a way similar to how the Ports Collection works is under consideration as an alternative mechanism.

    Concerns were raised regarding the pending HardenedBSD entry in the previous quarterly report prior to publication. The FreeBSD project welcomes reports from separate (but derived) projects in quarterly reports and has included similar reports in the past from other projects (such as TrueOS and pfSense). The HardenedBSD report was edited for length and to concentrate on activities during the quarter in question.

    Amazon is proposing to set up mirrors of the freebsd-update and pkg servers within AWS in order to provide faster access for EC2 users. These mirrors will be publicly accessible, but the expectation is that use will primarily be from within EC2. FreeBSD AMIs will have a preset configuration that references the Amazon servers.

    The old, long-deprecated, and insecure "r-commands" (rsh, rlogin, rcp) are being removed from the base system for 12.0-RELEASE. Notice of this was added to the man pages and release notes in time for 11.1-RELEASE and 10.4-RELEASE. Anyone requiring these commands for backwards compatibility can use the new net/bsdrcmds port.

    Work to replace Heimdal Kerberos in base with the more widely compatible MIT Kerberos has begun in a new projects/krb5 branch. This should not fall afoul of any US cryptography export regulations: the project is required to notify the US government that cryptographic software can be downloaded from FreeBSD servers, and this already covers MIT Kerberos, already available within ports.

    A number of Bay Area FreeBSD User Group-related domain names are being given up by their original owner. The current BAFUG organisers have been made aware.

    Core has voted on a change to the Doceng voting rules to provide for a "did not vote" status during doceng voting similar to how portmgr and core voting operates. The current requirement for all five members of doceng to register a vote on issues was proving to be a significant bottleneck.

    The FreeBSD Foundation

    Links
    FreeBSD Foundation Website URL: https://www.FreeBSDFoundation.org/
    FreeBSD Foundation Quarterly Newsletter URL: https://www.FreeBSDfoundation.org/wp-content/uploads/2017/08/FreeBSD-Foundation-July-August-2017-Update.pdf

    Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

    The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides full-time Release Engineering support; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.

    Here are some highlights of what we did to help FreeBSD last quarter:

    Fundraising Efforts

    Our work is 100% funded by your donations. This year we have raised over $860,000 from over 500 donors. Our 2017 fundraising goal is $1,250,00 and we are continuing to work hard to meet and exceed this goal! Please consider making a donation to help us continue and increase our support for FreeBSD: https://www.FreeBSDfoundation.org/donate/.

    We also have a new Partnership Program, to provide more benefits for our larger commercial donors. Find out more information at https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share with your companies!

    OS Improvements

    The Foundation improves the FreeBSD operating system by employing our technical staff to maintain and improve critical kernel subsystems, add features and functionality, and fix problems. This also includes funding separate project grants like the arm64 port, blacklistd access control daemon, and the integration of VIMAGE support, to make sure that FreeBSD remains a viable solution for research, education, computing, products and more.

    We kicked off or continued the following projects last quarter:

    • OpenZFS RAID-Z Expansion project
    • Broadcom Wi-Fi infrastructural improvements (bhnd(4) driver)
    • Headless mode out-of-the-box for the Beaglebone Black
    • Extending bhyve/ARMv7 features
    • Porting bhyve/ARM to an ARMv8 platform

    Having software developers on staff has allowed us to jump in and work directly on projects to improve FreeBSD like:

    • ZFS improvements
    • New Intel server support
    • kqueue(2) updates
    • 64-bit inode support
    • Stack guard
    • Kernel Undefined Behavior Sanitizer
    • Toolchain projects
    • i915 driver investigation
    • NVDIMM support in acpiconf(8)
    • Continuous integration dashboard (web page and physical hardware)
    • FAT filesystem support in makefs(8)

    Staff and board members continued hosting bi-weekly conference calls to facilitate efforts for individuals to collaborate on different technologies.

    Release Engineering

    The Foundation provides a full-time staff member to lead the release engineering efforts. This has provided timely and reliable releases over the last few years.

    Last quarter, our full-time staff member worked with the FreeBSD Release Engineering and Security Teams to finalize 11.1-RELEASE. He also supported the 10.4 release effort, and has continued producing 10-STABLE, 11-STABLE, and 12-CURRENT development snapshot builds throughout the quarter. At the vBSDCon Developer Summit, he gave a presentation on the state of the release engineering team.

    You can find out more about the support we provided to the Release Engineering Team by reading their status update in this report.

    Supporting FreeBSD Infrastructure

    The Foundation provides hardware and support to improve the FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware located around the world.

    FreeBSD Advocacy and Education

    A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations.

    The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project.

    Here is a list highlighting some of the advocacy and education work we did last quarter:

    • Organized and ran the Essen FreeBSD Hackathon in Essen Germany
    • Sponsored and participated in the FreeBSD Developer Summit BSDCam, in Cambridge, England
    • Represented FreeBSD at the ARM Partner Meeting
    • Presented and taught about FreeBSD at SdNOG 4 in Khartoum, Sudan
    • Sponsored and gave presentations and tutorials at EuroBSDCon in Paris, France
    • Organized and ran the Paris FreeBSD Developer Summit
    • Organized and ran the FreeBSD Developer Summit at vBSDCon
    • Sponsored and attended vBSDCon
    • Proved travel grants to FreeBSD contributors to attend the above events.
    • Sponsored the 2017 USENIX Security Symposium in Vancouver BC as an Industry Partner
    • Provided FreeBSD advocacy material
    • Sponsored the 2017 USENIX Annual Technical Conference in Santa Clara, CA as an Industry Partner

    We continued producing FreeBSD advocacy material to help people promote FreeBSD around the world.

    We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. Last quarter we published the July/August issue that you can find at https://www.FreeBSDfoundation.org/journal/.

    You can find out more about events we attended and upcoming events at https://www.FreeBSDfoundation.org/news-and-events/.

    Legal/FreeBSD IP

    The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.

    Go to http://www.FreeBSDfoundation.org to find out how we support FreeBSD and how we can help you!


    Projects

    Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.


    FreeBSD CI

    Links
    freebsd-ci Repository URL: https://github.com/freebsd/freebsd-ci
    freebsd-testing Mailing List URL: https://lists.FreeBSD.org/mailman/listinfo/freebsd-testing
    FreeBSD Jenkins Instance URL: http://ci.FreeBSD.org

    Contact: Jenkins Admins <jenkins-admin@FreeBSD.org>

    The FreeBSD CI team runs various continuous integration solutions for FreeBSD, regularly checking that the current state of the Subversion repository can successfully build, and performing various tests and analysis upon the build results.

    We have introduced a DTrace test pipeline, with the results and artifacts available at:

    We had team meetings at two developer summits during Q3:

    Open tasks:

    1. Fix the failing test cases and builds.
    2. Create builds for additional architectures.
    3. Add more tests.
    4. The additional TODO items listed at https://wiki.FreeBSD.org/Jenkins/TODO.

    Kernel

    Updates to kernel subsystems/features, driver support, filesystems, and more.


    Intel 10G iflib Driver Update

    Links
    ixgbe iflib Conversion URL: https://reviews.FreeBSD.org/D11727

    Contact: Chris Galazka <krzysztof.galazka@intel.com>
    Contact: Piotr Pietruszewski <piotr.pietruszewski@intel.com>

    The ix and ixv network interface drivers support a variety of Intel network interfaces, with line speeds at 10 Gbit/second.

    This quarter, with the help of Matt Macy and Sean Bruno (among others), we have submitted a review in Phabricator for the conversion of the ixgbe driver to use the new (and evolving) iflib framework.

    Stay tuned for the conversion of the 40G driver (ixl), as it is currently being ported to use iflib.

    Open tasks:

    1. Additional testing.

    Intel iWARP Support

    Links
    iWARP for ixl URL: https://reviews.FreeBSD.org/D11378

    Contact: Bartosz Sobczak <bartosz.sobczak@intel.com>

    iWARP is a protocol suite that enables efficient movement of data across the network, building on Remote Direct Memory Access, Direct Data Placement, and Marker PDU Aligned Framing. It endeavors to avoid unnecessary (local) data copies and to offload work from the main CPU to dedicated hardware.

    An initial commit adding iWARP support for the Intel X722 family of network adapters is under review. This is an important step towards introducing full iWARP support on systems equipped with Intel C620 Series Chipsets. Currently, with the iw_ixl driver, only the kVerbs API is supported.

    Open tasks:

    1. Additional testing.

    pNFS Server Plan B

    Links
    Instructions for Testing URL: http://people.FreeBSD.org/~rmacklem/pnfs-planb-setup.txt

    Contact: Rick Macklem <rmacklem@FreeBSD.org>

    A pNFS server allows an NFS service to be spread over multiple servers, separating the MetaData operations from the Data operations (Read and Write). This project will add the ability to use FreeBSD systems to create a pNFS service consisting of a single MetaData Server plus a set of Data Servers. The Data Servers can be mirrored, so that redundant copies of the file data are maintained.

    The support for non-mirrored Data Servers is now believed to be complete. Support for mirrored Data Servers using the Flexible File Layout, which will soon be published as an RFC, is implemented. However, there is still significant work to be done, since the current implementation of mirrored Data Servers does not handle failed Data Servers or their resilvering/recovery. It is hoped that support for failure/recovery of Data Servers will be implemented in the next six months.

    The patched FreeBSD sources may now be accessed for testing via either Subversion or downloading a gzipped tarball. They consist of a patched kernel and nfsd and can be used on any FreeBSD 11 or later system. The installation procedure is covered in the linked document.

    Open tasks:

    1. Testing by others will be needed, now that the implementation is available.
    2. Implementation and testing of mirror failure/recovery.

    Architectures

    Updating platform-specific features and bringing in support for new hardware platforms.


    AMD Zen (family 17h) support

    Contact: Conrad Meyer cem@FreeBSD.org <>

    This quarter, a bit of work was done to enhance platform support for AMD Zen (Ryzen, Threadripper, Epyc) processors:

    • The CPU topology detection code was enhanced to properly detect Zen dies and CPU Complexes. This gives the scheduler more locality information to use when making scheduling decisions.
    • The x86 topology analysis was enhanced to report dies and CPU Complexes, in addition to the existing reporting on packages, cores, and threads. An example of the new output is FreeBSD/SMP: 1 package(s) x 2 groups x 2 cache groups x 4 core(s) x 2 hardware threads.
    • The amdsmn(4) driver for accessing SMN (System Management Network) registers was added.
    • CPU temperature monitoring support for Zen was added to amdtemp(4).
    • In cpufreq(4):
      • Added support for decoding Zen P-state information from Machine State Registers (which is usually not necessary, since it is largely redundant with ACPI P-state information, but is potentially useful)
      • Work around the apparent Ryzen inability to achieve the P1 state by not busying cores waiting to transition to it
    • The intpm(4) smbus driver was fixed to attach to the AMD FCH (Fusion Controller Hub).
    • All MCA banks are now enabled and monitored on Zen CPUs.
    • Feature-bit decoding was added for: CLZERO, SVM features, and RAS capabilities.
    • SHA intrinsic support was added to the aesni(4) driver. Ryzen is currently the only desktop processor to feature these intrinsics. Support for these intrinsics is also present in Intel's Goldmont line of low-end SoCs.

    Overall, Zen is now a very usable platform for x86 workstations and servers.

    This project was sponsored by Dell EMC Isilon.

    Open tasks:

    1. Add HWPMC support for the new performance counters avilable on the Zen architecture.
    2. Add support for the CCP (Crypto Co-Processor).

    Userland Programs

    Changes affecting the base system and programs in it.


    Updates to GDB

    Contact: John Baldwin <jhb@FreeBSD.org>
    Contact: Luca Pizzamiglio <pizzamig@FreeBSD.org>

    The devel/gdb port has been updated to GDB 8.0.1.

    Support for FreeBSD/aarch64 userland binaries has been committed upstream. These patches, along with support for debugging FreeBSD/aarch64 kernels, have been committed to the port.

    Upstream patches adding improved support for FreeBSD/arm userland binaries are currently in review. FreeBSD 12 has recently grown support for debugging VFP registers via ptrace() and core dumps as part of this work. Support for FreeBSD/arm kernels will be added to the port after the upstream patches are added to the port.

    Support for $_siginfo has been committed upstream. This uses the recently added NT_LWPINFO note to extract signal information from process cores.

    -

    Hangs that occured when GDB's kill command was used +

    Hangs that occurred when GDB's kill command was used were fixed in FreeBSD in r313992.

    Open tasks:

    1. Figure out why the powerpc kgdb targets are not able to unwind the stack past the initial frame.
    2. Test support for sparc64 binaries and kernels.
    3. Add support for debugging powerpc vector registers.
    4. Implement info proc commands.
    5. Implement info os commands.

    Ports

    Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.


    FreeBSDDesktop

    Links
    FreeBSDDesktop on GitHub URL: https://github.com/FreeBSDDesktop/

    Contact: Johannes Dieterich <jmd@freebsd.org>
    Contact: Mark Johnston <markj@freebsd.org>
    Contact: Hans Petter Selasky <hselasky@freebsd.org>
    Contact: Matthew Macy <mmacy@nextbsd.org>

    The FreeBSDDesktop team is happy to announce the availability of graphics/drm-next-kmod. This port for FreeBSD-CURRENT (amd64) provides support for the amdgpu, i915, and radeon DRM modules using the linuxkpi compatibility framework. The port currently corresponds to the DRM from Linux 4.9 and is in an experimental state. It works reliably for many testers with modern GPU hardware (AMD HD7000 series/Tahiti to Polaris and Intel HD3000/Sandy Bridge to Skylake). Broader testing and reporting/fixing of bugs is appreciated.

    Open tasks:

    1. Resolve issues that cause radeonkms and amdgpu to fail with EFI boot (though there is a workaround available).
    2. Upgrade to Linux 4.10 and higher DRM versions.
    3. Get feedback from broader testing.

    OpenJFX 8

    Links
    OpenJFX Wiki URL: https://wiki.openjdk.java.net/display/OpenJFX/Main
    java/openjfx8-devel URL: https://www.freshports.org/java/openjfx8-devel
    java/openjfx8-scenebuilder URL: https://www.freshports.org/java/openjfx8-scenebuilder
    AsciidocFX URL: https://github.com/asciidocfx/AsciidocFX

    Contact: Tobias Kortkamp <tobik@FreeBSD.org>

    OpenJFX is an open source, next generation, client application platform for desktop and embedded systems, based on JavaSE. This quarter, the OpenJFX port was reworked and has received some significant improvements.

    More modules are being built. With the new web module we gain support for applications that have their own builtin web browser such as AsciidocFX. The new media module allows JavaFX applications to play audio and video files.

    A port of the JavaFX scenebuilder, a RAD tool for building JavaFX scenes, was added to the ports tree.

    The OpenGL Prism backend for GPU acceleration was enabled by default.

    From a mainainer's and contributor's perspective, the port was simplified by moving all FreeBSD-local patches to the ports tree and fetching the upstream sources directly, instead of using a separate repository for them.

    Open tasks:

    1. Upstream some of the patches in the Ports Collection.

    Puppet

    Links
    Puppetlab's FreeBSD Slack Channel URL: https://puppetcommunity.slack.com/messages/C6CK0UGB1/

    Contact: Puppet Team <puppet@FreeBSD.org>

    This summer has seen the creation of a puppet@ team to help maintain the approximately 30 Puppet-related ports in the FreeBSD Ports Collection. These ports were previously maintained by various committers, and from time to time the distributed maintainership introduced some delays when updating a port, due to the need to wait for a maintainer's approval for a related change to a different port.

    Puppet 5 is now in the ports tree (as sysutils/puppet5). The C++ version of Facter (sysutils/facter) got a lot of attention and is now a drop-in replacement for the previous Ruby version (sysutils/rubygem-facter); it is the default facts source for the Puppet 5 port.

    Work continues on bringing in Puppetserver 5 to the ports tree, and on keeping all the ports up-to-date.

    Open tasks:

    1. The pkg package provider has some minor issues (it breaks things when no repos are configured, and is not working properly from the context of the MCollective package agent).
    2. The databases/puppetdb[345] and sysutils/puppetserver[45] ports rely on Clojure and Java, and download compiled jar files instead of building them from source.

    Documentation

    Noteworthy changes in the documentation tree or new external books/documents.


    Absolute FreeBSD, 3rd Edition

    Links
    Official Announcement URL: https://blather.michaelwlucas.com/archives/3020

    Contact: Michael Lucas <mwlucas@michaelwlucas.com>

    The first draft of the third edition of Absolute FreeBSD is finished. It is 220,200 words, or roughly enough to stun a medium-sized ox. It's on target to be in print before BSDCan 2018.

    Open tasks:

    1. Stare at the wall blankly for a few days.
    2. Fix all the problems pointed out by dozens of community reviewers.
    3. Fix all the problems pointed out by John Baldwin, tech reviewer extraordinaire.
    4. Editing. Copyediting. Page layout. Page editing. Re-editing. Indexing. Edits discovered by indexer.
    5. Pre-orders should open some time next year.
    6. Restrain myself from strangling people who ask when the fourth edition is coming.

    Manual Pages

    Links
    FreeBSD Documentation Project Primer URL: https://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/

    Contact: Warren Block <wblock@FreeBSD.org>

    Over the last year, interest has increased in manual pages, in large part due to excellent infrastructure work by Baptiste Daroussin and others, and promotion by George Neville-Neil and others. This increased interest has been both gratifying and problematic. Our man pages are underappreciated gems, but we have sadly lacked any substantial documentation on how to write new ones.

    In September, I added a new chapter to the FreeBSD Documentation Project Primer describing the basics of creating a man page. It includes descriptions of the markup, section structure, recommended optional material such as examples, and sample templates for the most common types of man pages. The Resources section includes links to several external resources, including the excellent Practical UNIX Manuals: mdoc.

    While this chapter is not a full tutorial, it does begin to fill in a large gap in our documentation resources and provide a starting point from which to grow.

    Open tasks:

    1. Add more explanation and examples of markup usage.
    2. Expand the sample templates with additional desired standard features, like an EXAMPLES section.
    3. Add more sample templates.

    Third-Party Projects

    Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.


    The nosh Project

    Links
    Introduction URL: http://jdebp.eu./Softwares/nosh/
    FreeBSD Binary Packages URL: http://jdebp.eu./Softwares/nosh/freebsd-binary-packages.html
    Installation How-To URL: http://jdebp.eu./Softwares/nosh/timorous-admin-installation-how-to.html
    Roadmap URL: http://jdebp.eu./Softwares/nosh/roadmap.html
    A Slightly Outdated User Guide URL: http://jdebp.eu./Softwares/nosh/guide/index.html
    Archnosh URL: http://framagit.org/taca/archnosh

    Contact: Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM>

    The nosh project is a suite of system-level utilities for initializing, running, and shutting down BSD systems; and for managing daemons, terminals, and logging. It attempts to supersede BSD init, the Mewburn rc.d system, and OpenRC as used on FreeBSD and TrueOS, drawing inspiration from Solaris SMF for named milestones, daemontools-encore for service control/status mechanisms, UCSPI, and IBM AIX for separated service and system management. It includes a range of compatibility mechanisms, including shims for familiar commands from other systems, and an automatic import mechanism that takes existing configuration data from /etc/fstab, /etc/rc.conf{,.local}, /etc/ttys, and elsewhere, applying them to its native service definitions and creating additional native services. It is portable (including to Linux) and composable, it provides a migration path from the world of systemd Linux, and it does not require new kernel APIs. It provides clean service environments, orderings and dependencies between services, parallelized startup and shutdown (including fsck), strictly size-capped and autorotated logging, the service manager as a "subreaper", and uses kevent(2) for event-driven parallelism.

    Since the last status report, in December 2015, the project has seen: restructured and finer-grained packaging that has fewer conflicts with other toolsets; the addition of zsh completion files; improvements to the virtual terminal subsystem, keyboard map, mouse support, and ugen and DECSCUSR support; RFC 5424/5426 remote logging support; replacement of libkqueue and the C library's environment handling functions; several new helper commands; support for Java VM autolocation; improved socket-passing code; an extended status API and "one-shot" service support; additional pre-supplied service bundles; support for service aliases; improved handling of per-user D-Bus services; improved importing of MySQL, MariaDB, Percona, and OpenVPN services; improved configuration import support; and extensive additions to the nosh Guide.

    On the recently updated roadmap you can see plans for even more documentation, continuing the work to extend the capabilities of the networking subsystem, and the scant handful of rc.d-related items remaining. There are also some ideas still in the speculative or planning phases, including work that may depend on incorporating nosh support into other software.

    Open tasks:

    1. Improve Ansible and SaltStack integration (the maintainer of the Arch Linux nosh integration has some ideas).
    2. Command-line completions are still needed for bash, csh, and fish.
    3. Document convert-systemd-units for use by port maintainers in making packaged service bundles from systemd unit files.
    4. nosh could take advantage of several proposed features for the base system:
      • the boot loader signaling "emergency" and "rescue" modes of operation
      • adding machine-readable status output to fsck
      • adding runtime support for more clang-compilable languages in the early bootstrap stage
      • adding hooks for invoking external configuration import mechanisms
    News Home | Status Home

    diff --git a/website/content/en/status/report-2019-07-2019-09.html b/website/content/en/status/report-2019-07-2019-09.html index 190fad2639..f2e48e4fd7 100644 --- a/website/content/en/status/report-2019-07-2019-09.html +++ b/website/content/en/status/report-2019-07-2019-09.html @@ -1,2683 +1,2683 @@ FreeBSD Quarterly Status Report
    Skip site navigation (1) Skip section navigation (2)

    Introduction

    Here is the third quarterly status report for 2019.

    This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work).

    Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner.

    • Send a first draft of your report when you receive the first call for reports (1 month before the deadline).
    • Update your report, if needed, when you receive reminders: you will normaly receive two (2 weeks and 1 week before the deadline).
    • If after the deadline you still have some more updates ask the team (either on IRC via #freebsd-wiki or send an email at monthly@) to wait for you if you feel that they are urgent, otherwise start putting them in a draft for the next quarter.

    Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October.

    Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January.

    -- Lorenzo Salvadore

    FreeBSD Team Reports

    Projects

    Kernel

    Architectures

    Userland Programs

    Ports

    Third-Party Projects


      FreeBSD Team Reports

      Entries from the various official and semi-official teams, as found in the Administration Page.


      Cluster Administration Team

      Contact: Cluster Administration Team <clusteradm@FreeBSD.org>

      The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the Project relies on for its distributed work and communications to be synchronised. In this quarter, the team has worked on the following:

      • Change IPv6 address in TWN site.
      • Solved hardware issues in KWC site (with hrs@).
      • Moved remaining infrastructure from the YSV (Yahoo!) site to NYI (New York Internet) (peter@).
        • YSV hosted most of FreeBSD.org between 2000 and 2019.
      • Installed new machines for portmgr@ courtesy of the FreeBSD Foundation.
      • Resolved outtages (thanks uqs@) with GitHub exporter, Bugzilla and hg-beta (thanks bapt@).
      • PowerPC64 servers are online (power8) building pkgs and reference hosts.
      • Ongoing systems administration work:
        • Creating accounts for new committers.
        • Backups of critical infrastructure.
        • Keeping up with security updates in 3rd party software.

      Work in progress:

      • Review the service jails and service administrators operation.
      • South Africa Mirror (JINX) in progress.
      • NVME issues on PowerPC64 Power9 blocking dual socket machine from being used as pkg builder.
      • Drive upgrade test for pkg builders (SSDs) courtesy of the FreeBSD Foundation.
      • Boot issues with Aarch64 reference machines.
      • New NYI.net sponsored colocation space in Chicago-land area.
      • Setup new host for CI staging environment.

      Continuous Integration

      Links
      FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org
      FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org/
      FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins
      freebsd-testing Mailing List URL: https://lists.FreeBSD.org/mailman/listinfo/freebsd-testing
      FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci
      Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maauwg
      Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI
      FreeBSD CI weekly report URL: https://hackmd.io/@FreeBSD-CI

      Contact: Jenkins Admin <jenkins-admin@FreeBSD.org>
      Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>

      The FreeBSD CI team maintains continuous integration system and related tasks for the FreeBSD project. The CI system regularly checks the committed changes can be successfully built, then performs various tests and analysis of the results. The results from build jobs are archived in an artifact server, for the further testing and debugging needs. The CI team members examine the failing builds and unstable tests, and work with the experts in that area to fix the code or adjust test infrastructure. The details are of these efforts are available in the weekly CI reports.

      We had a testing working group at the 201909 DevSummit lwhsu@ has presented the Testing/CI project status and "how to work with the FreeBSD CI system", slides are available at the DevSummit page. Some contents have been migrated to https://wiki.freebsd.org/Jenkins/Debug , extending is welcomed.

      We continue publishing CI Weekly Report and moved the archive to https://hackmd.io/@FreeBSD-CI

      Work in progress:

      • Collecting and sorting CI tasks and ideas at https://hackmd.io/bWCGgdDFTTK_FG0X7J1Vmg
      • Setup the CI stage environment and put the experimental jobs on it
      • Extending and publishing the embedded boards testbed
      • Implementing automatic tests on bare metal hardware
      • Adding drm ports building test against -CURRENT
      • Testing and merging pull requests at https://github.com/freebsd/freebsd-ci/pulls
      • Planning for running ztest and network stack tests
      • Help more 3rd software get CI on FreeBSD through a hosted CI solution

      Please see freebsd-testing@ related tickets for more WIP information.

      This project was sponsored by The FreeBSD Foundation.

      FreeBSD Core Team

      Contact: FreeBSD Core Team <core@FreeBSD.org>

      The FreeBSD Core Team is the governing body of FreeBSD.

      • Core has provisionally accepted the BSD+patent license for use in some cases. The Core Team must approve the import of new BSD+Patent licensed components or the change of license of existing components to the BSD+Patent License.
        https://opensource.org/licenses/BSDplusPatent
      • Kernel Pseudo Random Number Generator (PRNG) maintainership was updated to reduce the contribution barrier for committers who have demonstrated competence in this part of the tree.
      • Core approved a source commit bit for Paweł Biernacki. Konstantin Belousov <kib@> will mentor Paweł and Mateusz Guzik <mjg@> will be co-mentor.
      • The Core-initiated Git Transition Working Group met over the last quarter, however a report is still forthcoming. Discussions will continue in the fourth quarter of 2019. There are many issues to resolve including how to deal with contrib/, whether to re-generate hashes in the current Git repository, and how to best implement commit testing.

      FreeBSD Foundation

      Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

      The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resources to improve security and quality assurance efforts; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.

      Here are some highlights of what we did to help FreeBSD last quarter:

      Partnerships and Commercial User Support We help facilitate collaboration between commercial users and FreeBSD developers. We also meet with companies to discuss their needs and bring that information back to the Project. In Q3, Ed Maste and Deb Goodkin met with a few commercial users in the US. It is not only beneficial for the above, but it also helps us understand some of the applications where FreeBSD is used. We were also able to meet with a good number of commercial users at vBSDCon and EuroBSDCon. These venues provide an excellent opportunity to meet with commercial and individual users and contributors to FreeBSD.

      Fundraising Efforts Our work is 100% funded by your donations. We are continuing to work hard to get more commercial users to give back to help us continue our work supporting FreeBSD. More importantly, we'd like to thank our individual donors for making $10-$1,000 donations last quarter, for more than $16,000!

      Please consider making a donation to help us continue and increase our support for FreeBSD!

      We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information at www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share with your companies.

      OS Improvements

      The Foundation supports software development projects to improve the FreeBSD operating system through our full time technical staff, contractors, and project grant recipients. They maintain and improve critical kernel subsystems, add new features and functionality, and fix problems.

      Over the last quarter there were 345 commits to the FreeBSD base system repository sponsored by the FreeBSD Foundation - this represents about one fifth of all commits during this period. Many of these projects have their own entries in this quarterly report (and are not repeated here).

      Foundation staff member Konstantin Belousov committed many improvements to multiple kernel subsystems, as well as low-level 32-bit and 64-bit x86 infrastructure. These included fixes for robust mutexes, unionfs, the out of memory (OOM) handler, and per-cpu allocators.

      Additional work included fixes for security issues and introduction and maintenance of vulnerability mitigations, and improving POSIX conformance.

      Ed Maste committed a number of minor security bug fixes and improvements, as well as the first iteration of a tool for editing the mitigation control ELF note. Additional work included effort on build infrastructure and the tool chain.

      Clang's integrated assembler (IAS) is now used more widely, as part of the path to retiring the assembler from GNU binutils 2.17.50. The readelf tool now decodes some additional ELF note information.

      Ed also enabled the Linuxulator (Linux binary support layer) on arm64, and added a trivial implementation of the renameat2 system call (handling common options).

      Mark Johnston added Capsicum support to a number of ELF Tool Chain utilities, and committed a number of other Capsicum kernel and userland fixes.

      Mark worked on a number of changes related to security improvements, including integration and support of the Syzkaller automated system call fuzzer, and fixing issues identified by Syzkaller. Other changes included addressing failures caused by refcount wraparound, improvements to the prot_max memory protection. Other work included NUMA, locking, kernel debugging, RISC-V and arm64 kernel improvements.

      Edward Napierala continued working on Linuxulator improvements over the quarter. The primary focus continued to be tool improvements - strace is now more usable for diagnosing issues with Linux binaries running under the Linuxulator. That said, as with previous work a number of issues have been fixed along the way. These are generally minor issues with a large impact - for example, every binary linked against up-to-date glibc previously segfaulted on startup. This is now fixed.

      Continuous Integration and Quality Assurance The Foundation provides a full-time staff member who is working on improving our automated testing, continuous integration, and overall quality assurance efforts.

      During the third quarter of 2019, Foundation staff continued to improve the project's CI infrastructure, worked with contributors to fix the failing build and test cases, and worked with other teams in the Project for their testing needs. We added several new CI jobs and worked on getting the hardware regression testing lab ready.

      Li-Wen Hsu gave presentations "Testing/CI status update" and "How to work with the FreeBSD CI system" at the 201909 DevSummit. Slides are available at the DevSummit page.

      We continue publishing the CI weekly report on the freebsd-testing@. mailing list, and an archive is available.

      See the FreeBSD CI section of this report for completed work items and detailed information.

      Supporting FreeBSD Infrastructure The Foundation provides hardware and support to improve the FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware located around the world.

      FreeBSD Advocacy and Education A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations.

      The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project.

      Check out some of the advocacy and education work we did last quarter:

      • Sponsored USENIX 2019 Annual Technical Conference as an Industry Partner
      • Represented FreeBSD at OSCON 2019 in Portland, OR
      • Represented FreeBSD at COSCUP 2019 in Taiwan
      • Presented at the Open Source Summit, North American in San Diego, CA
      • Executive Director Deb Goodkin was interviewed by TFiR https://www.freebsdfoundation.org/news-and-events/latest-news/tfir-interview-freebsd-meets-linux-at-the-open-source-summit/
      • Sponsored FreeBSD Hackathon at vBSDcon 2019 in Reston, VA
      • Sponsored the attendee bags and attended vBSDcon 2019 in Reston VA
      • Represented FreeBSD at APNIC-48 in Chiang Mai, Thailand
      • Represented FreeBSD at MNNOG-1 in Ulaanbaatar, Mongolia
      • Served as an administrator for the Project’s Google Summer of Code Session. See the Google Summer of Code section of this report for more information.
      • Sponsored FreeBSD Developers Summit at EuroBSDCon in Lillehammer, Norway
      • Sponsored and attended EuroBSDcon 2019 in Lillehammer, Norway
      • Applied and was accepted for a FreeBSD Miniconf at linux.conf.au, in Gold Coast, Australia, Jan 14, 2020
      • Our FreeBSD talk was accepted at seaGL, Seattle, WA, November 15 and 16.

      We continued producing FreeBSD advocacy material to help people promote FreeBSD. Learn more about our recent efforts to advocate for FreeBSD around the world: https://www.freebsdfoundation.org/blog/freebsd-around-the-world/

      Our Faces of FreeBSD series is back. Check out the latest post: Roller Angel.

      Read more about our conference adventures in the conference recaps and trip reports in our monthly newsletters: https://www.freebsdfoundation.org/news-and-events/newsletter/

      We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.FreeBSDfoundation.org/journal/.

      You can find out more about events we attended and upcoming events.

      We opened our official FreeBSD Swag Store. Get stickers, shirts, mugs and more at ShopFreeBSD.

      We have continued our work with a new website developer to help us improve our website. Work has begun to make it easier for community members to find information and to make the site more efficient.

      Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.

      Go to http://www.FreeBSDfoundation.org to find out how we support FreeBSD and how we can help you!

      FreeBSD Graphics Team status report

      Links
      Project GitHub page URL: https://github.com/FreeBSDDesktop

      Contact: FreeBSD Graphics Team <x11@freebsd.org>
      Contact: Niclas Zeising <zeising@freebsd.org>

      The FreeBSD X11/Graphics team maintains the lower levels of the FreeBSD graphics stack. This includes graphics drivers, graphics libraries such as the MESA OpenGL implementation, the X.org xserver with related libraries and applications, and Wayland with related libraries and applications.

      During the last period, several changes have been made, but most of them has been behind the scene. We have also worked on general clean up of old xorg ports that have been deprecated upstream.

      The ports infrastructure for xorg ports and ports that depend on xorg ports have been updated. We have switched USE_XORG and XORG_CAT to use the USES framework, instead of the old way of including bsd.xorg.mk from bsd.port.mk. This infrastructure work has been fairly substantial, and new ports depending on xorg ports should add USES=xorg to their makefiles. As part of this bsd.xorg.mk was split up, and the XORG_CAT part was split out to USES=xorg-cat. This is used for the xorg ports themselves, and sets up a common environment for building all xorg ports. In addition, framework for pulling xorg ports directly from freedesktop.org gitlab was added, which will make improve development and testing, since it makes it possible to create ports of unreleased versions. Further improvements in this area includes framework for using meson instead of autotools for building xorg ports. This is still a work in progress.

      We have also worked to clean up and deprecate several old xorg ports and libraries. Some of these ports have already been removed, and some are still waiting on removal after a sufficient deprecation period. Most notably amongst the deprecations are x11/libXp, which required to fix several dependencies. Several other old libraries have also been deprecated, such as x11/Xxf86misc, x11-fonts/libXfontcache and graphics/libGLw. Some applications and drivers have also been deprecated during the period. With the remaining removals in this area, we should be up to speed with deprecations upstream. We are currently investigating if there are new software added upstream that we need to port to FreeBSD.

      We have also continued our regularly scheduled bi-weekly meetings.

      People who are interested in helping out can find us on the x11@FreeBSD.org mailing list, or on our gitter chat: https://gitter.im/FreeBSDDesktop/Lobby. We are also available in #freebsd-xorg on EFNet.

      We also have a team area on GitHub where our work repositories can be found: https://github.com/FreeBSDDesktop

      FreeBSD Release Engineering Team

      Links
      FreeBSD 11.3-RELEASE announcement URL: https://www.freebsd.org/releases/11.3R/announce.html
      FreeBSD 12.1-RELEASE schedule URL: https://www.freebsd.org/releases/12.1R/schedule.html
      FreeBSD 12.1-RELEASE BETA/RC builds URL: https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.1/
      FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/

      Contact: FreeBSD Release Engineering Team <re@FreeBSD.org>

      The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.

      During the third quarter of 2019, the FreeBSD Release Engineering team finished the 11.3-RELEASE cycle, with the final release build started on July 5th and the official announcement sent on July 9th.

      FreeBSD 11.3-RELEASE is the fourth release from the stable/11 branch, building on the stability and reliability of 11.2-RELEASE.

      The FreeBSD Release Engineering Team also started work on the upcoming 12.1-RELEASE, which started September 6th. This release cycle is the first "freeze-less" release from the Subversion repository, and the test bed for eliminating the requirement of a hard code freeze on development branches. Commits to the releng/12.1 branch still require explicit approval from the Release Engineering Team, however.

      At present, there have been three BETA builds, and so far, two RC builds, with the final 12.1-RELEASE build scheduled for November 4th.

      Additionally throughout the quarter, several development snapshots builds were released for the head and stable/11 branches; snapshots for stable/12 were released as well although not during the 12.1-RELEASE cycle.

      Much of this work was sponsored by Rubicon Communications, LLC (Netgate) and the FreeBSD Foundation.

      FreeBSD Security Team

      Links
      FreeBSD security information URL: https://www.freebsd.org/security/

      Contact: Security Team <secteam@FreeBSD.org>

      Several members of the security team met at the Vendor Summit in October to formalize team structure dedicated for architecture and crypto engineering in addition to the existing product security incident response function.

      Since June we have started having fortnightly conference calls to discuss important issues and to collaborate closely on advisories and errata notices in the pipeline.

      • Security advisories sent out in 2019-Q3: 7
      • Errata Notices sent out in 2019-Q3: 5

      Projects

      Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.


      FAT / msdosfs support for makefs(8)

      Contact: Ed Maste <emaste@FreeBSD.org>

      In order to streamline the process of creating install or virtual machine system images we needed FAT filesystem support in makefs(8). Makefs was originally developed in NetBSD, and FAT support was added there not much later, but after the tool was ported to FreeBSD.

      Siva Mahadevan, one of the FreeBSD Foundation's interns from the University of Waterloo, worked on porting FAT support from NetBSD. I rebased and updated Siva's work, and committed it during this quarter. After a few follow-up fixes we are able to build FAT filesystem images without using md(4) and without requiring root.

      This project was sponsored by The FreeBSD Foundation.

      FUSE

      Contact: Alan Somers <asomers@FreeBSD.org>

      FUSE (File system in USErspace) allows a userspace program to implement a file system. It is widely used to support out-of-tree file systems like NTFS, as well as for exotic pseudo file systems like sshfs. FreeBSD's fuse driver was added as a GSoC project in 2012. Since that time, it has been largely neglected. The FUSE software is buggy and out-of-date. Our implementation is about 11 years behind.

      I completed this work during Q3. I fixed a few newly-introduced bugs, fixed a long-standing sendfile bug that affects FUSE ([236466](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=236466)) and merged everything to head and stable/12. Then I fixed the resulting Coverity CIDs. There have been no new FUSE-related bug reports, so I can only assume that everything is working great. Report any problems to asomers@FreeBSD.org.

      This project was sponsored by The FreeBSD Foundation.

      Google Summer of Code 2019

      Links
      2019 Summer of Code Project Wikis URL: https://wiki.freebsd.org/SummerOfCode2019Projects
      2019 Summer of Code Projects URL: https://summerofcode.withgoogle.com/archive/2019/organizations/6504969929228288/

      Contact: Summer of Code Admins <soc-admins@freebsd.org>

      The FreeBSD Project is pleased to have participated in Google Summer of Code 2019 marking our 14th year of participation. This year we had six successful projects:

      • Dual-stack ping command by Jn Sučan
      • Firewall test suite by Ahsan Barkati
      • Kernel sanitizers by Costin Carabaș
      • MAC policy on IP addresses for FreeBSD Jail by Shivank Garg
      • Separation of ports build process from local installation by Theron Tarigo
      • Virtual memory compression by Paavo-Einari Kaipila

      We thank Google for the opportunity to work with these students and hope they continue to work with FreeBSD in the future.

      This project was sponsored by Google Summer of Code.

      GSoC'19 Project - MAC policy on IP addresses in Jail: mac_ipacl

      Links
      FreeBSD's Phabricator Differential Link URL: https://reviews.freebsd.org/D20967
      Github Diff Link URL: https://github.com/freebsd/freebsd/compare/master...shivankgarg98:shivank_MACPolicyIPAddressJail
      Project Wiki Page URL: https://wiki.freebsd.org/SummerOfCode2019Projects/MACPolicyIPAddressJail

      Contact: Shivank Garg <shivank@FreeBSD.org>

      About - With the introduction of VNET(9) in FreeBSD, Jails are free to set their IP addresses. However, this privilege may need to be limited by the host as per its need for multiple security reasons. This project uses mac(9) for an access control framework to impose restrictions on FreeBSD jails according to rules defined by the root of the host using sysctl(8). It involves the development of a dynamically loadable kernel module (mac_ipacl) based on The TrustedBSD MAC Framework to implement a security policy for configuring the network stack. This project allows the root of the host to define the policy rules to limit the root of a jail to a set of IP (v4 or v6) addresses and/or subnets for a set of interfaces.

      Features this new MAC policy module are:

      • The host can define one or more lists of IP addresses/subnets for the jail to choose from.
      • The host can restrict the jail from setting certain IP addresses or prefixes (subnets).
      • The host can restrict this privilege to a few network interfaces.

      Implementation - The mac_ipacl module is a loadable kernel module. It implements mac checks in netinet/in.c and netinet6/in6.c to check the IP addresses requested by jail. The idea to implement these checks at these places comes from the fact that SIOCAIFADDR (for IPv4) and SIOCAIFADDR_IN6 (for IPv6) ioctl handlers are defined for adding the IP addresses to an interface. This is used by ifconfig (in userspace) for setting the IP address. The MAC Framework acts as multiplexer between the netinet and the module. The requested IP and the credentials are checked with the rules in mac_ipacl and output is returned accordingly to netinet. The module can be tuned with various sysctl and similarly, policy rules are also be defined with sysctl.

      TestSuite - Test scripts integrated with kyua and ATF are included with the module.

      Using the module - I have written a man page for the module. Please refer to the mac_ipacl(4) for using the new MAC module and various examples.

      Final Deliverables -

      This is a new project, developed as part of Google Summer of Code'19 under the guidance of Bjoern A. Zeeb <bz@FreeBSD.org>. The module is reviewed and Revision for this project is accepted and ready to land. It is yet to be merged with FreeBSD HEAD, and waiting to be tested by few more hands in the industry.

      I'll be very thankful if you can give this module a try and share your valuable experience about it. Please be free to share your ideas and feedback on this module and please do not hesitate to send me an email.

      Improving laptop support

      Contact: Ed Maste <emaste@FreeBSD.org>

      The FreeBSD Foundation would like to ensure that running FreeBSD on contemporary hardware, including laptops, remains viable. To that end we plan to purchase the latest generation of one or more of a family of laptops preferred by members of the FreeBSD community, evaluate the existing state of hardware support, and implement missing hardware support where possible.

      As the first laptop for this project we have selected a 7th Generation Lenovo X1 Carbon.

      This project was sponsored by The FreeBSD Foundation.

      NFS Version 4.2 implementation

      Contact: Rick Macklem <rmacklem@freebsd.org>

      RFC-7862 describes a new minor revision to the NFS Version 4 protocol. This project implements this new minor revision.

      The NFS Version 4 Minor Version 2 protocol adds several optional features to NFS, such as support for SEEK_DATA/SEEK_HOLE, file copying done on the server that avoids data transfer over the wire and support for posix_fallocate(), posix_fadvise(). Hopefully these features can improve performance for certain applications.

      The implementation is now nearing completion and recent work has been mostly testing. A cycle of interoperability testing with Linux has just been completed. The main area that still needs testing is use of the pNFS server with NFSv4.2 and that should start soon. Once testing of pNFS is completed, I believe the code is ready to be incorporated into FreeBSD head/current.

      Testing by others would be appreciated. The modified kernel can be found at https://svn.freebsd.org/base/projects/nfsv42/sys and should run with a recent FreeBSD head/current system. Client mounts need the "minorversion=2" mount option to enable this protocol.

      Rockchip RK3399 SoC's eMMC support

      Contact: Ganbold Tsagaankhuu <ganbold@FreeBSD.org>

      The followings features have been added to support RK3399 SoC eMMC on FreeBSD:

      • Extended simple_mfd driver to expose a syscon interface if that node is also compatible with syscon. For instance, Rockchip RK3399's GRF (General Register Files) is compatible with simple-mfd as well as syscon and has devices like usb2-phy, emmc-phy and pcie-phy etc. under it.
      • Made Rockchip's General Register Files driver the subclass of Simple MFD driver
      • Added driver for Rockchip RK3399 eMMC PHY.
      • Added eMMC support codes for Rockchip RK3399 SoC.
      • All above was tested on NanoPC-T4 board.

      syzkaller on FreeBSD

      Contact: Andrew Turner <andrew@FreeBSD.org>
      Contact: Mark Johnston <markj@FreeBSD.org>
      Contact: Michael Tuexen <tuexen@FreeBSD.org>
      Contact: Samy Al Bahra <sbahra@freebsd.org>

      See the syzkaller entry in the 2019q1 quarterly report for an introduction to syzkaller.

      Work has continued on fixing bugs uncovered by syzkaller. Over a dozen kernel bugs fixed in the past three months have been directly attributed to syzkaller, and a number of syzkaller reproducers have been incorporated into our test suite.

      backtrace.io, via Samy, has graciously provided a large server for running a dedicated syzkaller instance to fuzz FreeBSD under bhyve. Though syzbot, the public syzkaller instance run by Google, already fuzzes FreeBSD, it has proven fruitful to run multiple syzkaller instances: different instances find different bugs, and syzkaller.backtrace.io allows us to experiment with FreeBSD-specific extensions. In particular, this instance currently uploads data about each crash to backtrace.io, making it much easier to triage and analyze crashes. We plan to make this service generally available to FreeBSD developers in the near future.

      Going forward we will continue to extend syzkaller coverage and make it simpler for users and developers to run private instances, and optionally collect kernel crash information for debugging or for submission.

      This project was sponsored by backtrace.io, and The FreeBSD Foundation.

      TPM2 Software Stack (TSS2)

      Links
      tpm2-tss Documentation URL: https://tpm2-tss.readthedocs.io/en/latest/index.html
      tpm2 Source Repository URL: https://github.com/tpm2-software/
      tpm2 mailing list URL: https://lists.01.org/postorius/lists/tpm2.lists.01.org/
      tpm2 irc channel URL: ircs://chat.freenode.net:6697/tpm2.0-tss

      Contact: D Scott Phillips <scottph@FreeBSD.org>

      Intel has contributed ports of the TPM2 Software Stack to the ports tree, with the new ports security/tpm2-tss, security/tpm2-tools, security/tpm2-abrmd. tpm2-tss contains a set of libraries which expose various TPM2 APIs for using a TPM conforming to the TCG TPM 2.0 specification. tpm2-tools provides a set of command line tools which use the tpm2-tss libraries to perform tpm operations. Finally, tpm2-abrmd is a userspace daemon which acts as a TPM Access Broker and Resource Manager, multiplexing many TPM users onto a single TPM device.

      Sponsored by: Intel Corporation


      Kernel

      Updates to kernel subsystems/features, driver support, filesystems, and more.


      Casueword(9) livelock

      Contact: Konstantin Belousov <kib@FreeBSD.org>

      The Compare-And-Swap (CAS) is one of the fundamental building blocks for hardware-assisted atomic read/modify/write operations. Some architectures support it directly, for instance x86 and sparc. Others provide different building blocks, usually the pair of Load Linked/Store Conditional instructions (ll/sc) which can be used to construct CAS or other atomic operations like Fetch-And-Add or any atomic arithmetic ops using plain arithmetic instructions. An example is the LDXR/STXR pair on AArch64.

      The ll/sc operation is performed by first using the load linked instruction to load a value from memory and simultaneously mark the cache line for exclusive access. The value is then updated by the store conditional instruction, but only if there were not any writes to the marked cache line. Any write by another CPU makes the store instruction fail. So typically atomic primitives on ll/sc architectures retry the whole operation if only store failed, because it just means that this CPU either lost a race, or even the failure was spurious. This is so called strong version of CAS and atomics. If primitive returns failure instead, the CAS variant is called weak by C standard.

      For the FreeBSD threading library lock implementation, when the fast path mode in userspace was not possible, the kernel often needs to do a CAS operation on user memory location. In addition to all guarantees of normal CAS, it also must ensure the safety and tolerance to paging. In FreeBSD, the casueword32(9) primitive provides CAS on usermode 32bit words for kernel. Casueword32(9) was implemented as strong CAS, similarly to the mode of operation of hardware CAS instructions on x86.

      Using the strong implementation for casueword may be dangerous, since the same address is potentially accessible to other, potentially malicious, threads in the same or other processes. If such a thread constantly dirties the cache line used by a ll/sc loop, it could practically force the kernel-mode thread to remain stuck in the loop forever. Since the loop is tight, and it does not check for signals, the thread cannot be stopped or killed.

      The solution is to make the casueword implementation weak, which means that the interface of the primitive must be changed to allow notifying the caller about spurious failures. Also, it is now the caller's responsibility to retry as appropriate.

      The change to casueword was made for all architectures. Even on x86, the PSL.ZF value after the CMPXCHG instruction was returned directly for the new casueword. All two dozens uses of the primitive, all located in kern_umtx.c, were inspected and retry added as needed.

      As a somewhat related note, in-kernel atomic_cmpset(9) operations are strong, while atomic_fcmpset(9) should be weak, unless broken by a specific architecture. The general attitude seems to be that retry is the duty of the primitive's caller.

      This project was sponsored by The FreeBSD Foundation.

      Kernel Mapping Protections

      Contact: Mark Johnston <markj@FreeBSD.org>

      Modern CPU architectures have the ability to flag memory mappings as "no-execute" (NX), which prevents that memory from being executed by a processor. NX mappings are an important security mitigation since they help segregate code and data, blocking unintentional execution of memory whose contents is controlled by an attacker. W^X (write XOR execute) is a security policy which disallows the creation of mappings that are simultaneously writeable and executable. Under this policy, memory whose contents can be modified must be mapped with the NX flag. This policy makes it harder to exploit bugs that permit an attacker to arbitrarily overwrite data.

      FreeBSD has long made use of the NX flag for userspace mappings whenever possible, but only in the past several years has it been applied to kernel mappings. A recent project has sought to implement a W^X-by-default policy for the amd64 kernel by completing an audit of the remaining executable mappings in the kernel, and making modifications to either apply the NX bit to those mappings or to make them read-only. This work has landed in HEAD and will be available in FreeBSD 13.0 and 12.2. Similar work for other CPU architectures is also planned.

      To help audit kernel mapping protections, the vm.pmap.kernel_maps sysctl was added; it dumps a snapshot of the kernel's page table entries and their attributes. This way, mappings violating the W^X policy can easily be discovered and investigated, and the sysctl provides information useful to anyone interested in kernel memory layout.

      With a few rare exceptions, the only kernel mappings which require execute permission are those of the kernel executable itself, and loadable kernel modules. A number of other regions of memory were unnecessarily being mapped without NX, and these were identified and corrected first. To address the kernel code mappings, the amd64 kernel linker script was modified to pad the .text segment to a 2MB boundary. To improve performance, the kernel creates superpage mappings of its .text segment, but this means that any data cohabiting the final 2MB .text mapping is mapped with execute permissions. The padding allows executable code to be segregated from data which follows in the kernel image, avoiding this problem and maintaining the superpage optimization at the expense of some wasted RAM.

      Enforcing W^X turned out to be somewhat trickier. Unlike other CPU architectures supported by FreeBSD, amd64 kernel modules are linked as relocatable object files, i.e., .o files. (On other architectures, they are dynamically shared objects (DSOs, or .so files), as one might naively expect.) The use of .o files means that amd64 kernel modules contain more efficient code than they would if linked as DSOs, since DSOs inherently make use of certain types of indirection which allow shared libraries to be loaded at arbitrary addresses, and this indirection is useless in the kernel. As part of this project an attempt was made to switch amd64 to using DSOs as well, since the cost of this indirection can largely be mitigated with modern toolchains, but it was found that the use of DSOs would also force a change to the code model used when compiling amd64 kernel code, resulting in a further performance penalty.

      The main obstacle with the use of .o files is that sections are not page-aligned by default; the segregation of sections with differing mapping protection requirements is done by the static linker when linking a DSO or executable file. Since mapping protections are applied at the granularity of the page size (4KB on amd64), the overlap of sections within a page causes problems since, for example, the end of the read-only .text section may overlap with the beginning of the read-write .data section. One possible solution is to perform any required section reordering and padding at kernel module load time, but this approach breaks debugging tools such as dtrace(1) and kgdb which assume that the kernel linker does not modify the layout of loadable modules. As a result, an amd64 kernel module linker script is now used to insert padding for specific sections. Finally, the kernel linker was modified to restrict mapping protections based on section flags.

      As a result of all of this, amd64 kernels now boot without any writeable, executable mappings. Though some of the work was architecture-specific, much of it can and will be leveraged to provide the same policy for our other supported architectures.

      This project was sponsored by Netflix.

      Kernel ZLIB Update

      Contact: Xin Li <delphij@FreeBSD.org>
      Contact: Yoshihiro Ota <ota@j.email.ne.jp>

      The ZLIB is a compression library is widely used in various software. The FreeBSD system had used an ancient (over 20 year-old) version of zlib (version 1.0.4) and total of 3 versions, one in user-land, one in ZFS, and one in kernel. Xin and Yoshihiro upgraded zlib to the latest and eliminated 2 extra copies. Along the efforts, zlib version was upgraded to 1.2.11, netgraph's ppp is re-implemented to use the standard zlib, and removed unmaintained code. We now only have one and the latest version of zlib in FreeBSD code base, new compress, compress2, and uncompress APIs exposed in the kernel, and importing changes from zlib will be simple.

      Kernel zlib changes

      Kernel zlib user updates

      Legacy code removals

      PROT_MAX mmap/mprotect maximum protections API

      Links
      PROT_MAX commit URL: https://reviews.freebsd.org/rS349240

      Contact: Brooks Davis <brooks@freebsd.org>

      Unix-like systems provide the mmap(2) system call to allocate memory or map files or devices into memory with specified access protection, and the mprotect(2) system call to change protections on mapped memory. Protection flags specify whether the memory may be read, written, and/or executed (PROT_READ, PROT_WRITE, PROT_EXEC respectively). Traditionally, mprotect(2) can be used to set any desired protections (except that a shared mapping of a file opened read-only cannot have PROT_WRITE set).

      A new macro PROT_MAX() adds a facility for specifying the maximum possible protection flags for mmap(2) and mprotect(2) calls. The program can then specify whether a mapping may be changed in the future to allow a given access protection. For example, a memory mapping can be set such that it can have read and write protections set, but may never be made executable.

      Maximum protection values are provided to the PROT_MAX() macro, and are OR'd with regular protection values.

      This change allows (e.g.) a region that must be writable during run-time linking or JIT code generation to be made permanently read+execute after writes are complete. This complements Write-XOR-Execute (W^X) protections available on other operating systems, allowing more precise control by the programmer.

      For example, to request memory that cannot be made executable: mmap(NULL, size, PROT_READ | PROT_WRITE | PROT_MAX(PROT_READ | PROT_WRITE), MAP_ANON, -1, 0);

      and to request memory that may have execute permission enabled later on, but is not currently executable:

      mmap(NULL, size, PROT_READ | PROT_WRITE | PROT_MAX(PROT_READ | PROT_WRITE | PROT_EXECUTE), MAP_ANON, -1, 0);

      This change alters mprotect argument checking and returns an error when unhandled protection flags are set. This differs from POSIX (in that POSIX only specifies an error if a valid permission can not be set), but is the documented behavior on Linux and more closely matches historical mmap behavior.

      In addition to explicit setting of the maximum permissions, an experimental sysctl vm.imply_prot_max causes mmap to assume that the initial permissions requested should be the maximum when the sysctl is set to 1. This behavior is known to break code that uses PROT_NONE reservations before mapping contents into part of the reservation. A later version of this work is expected to provide per-binary and per-process opt-in/out options and this sysctl may be removed in its current form. As such it is undocumented.

      While these flags are non-portable, they can be used in portable code with simple ifdefs to expand PROT_MAX() to 0.

      Sponsors: DARPA, AFRL

      Randomized Top of Stack pointer

      Contact: Konstantin Belousov <kib@FreeBSD.org>

      After the ASLR so useful addition, next in the series of the buzzword-compliant checkboxes is the stack addresses randomization.

      The main userspace thread stack on FreeBSD is traditionally allocated from the top of the user address space and grows down. Besides the initial pointer for the stack on userspace entry, this area also contains structures for program arguments and environment (so called strings), and aux vector data for ELF images.

      Considering the goal of randomizing the addresses of strings and main thread initial frame, moving the main stack area in the address space is not feasible. It would cause significant ABI breakage, invalidates the knowledge already coded into many introspection tools, and causes unneeded additional fragmentation of the user address space.

      Instead a typical approach of adding a gap of randomized size between top of stack and a place for strings was used. It is done in a way which preserves the stack alignment requirement. Stack gap is only enabled if ASLR is enabled (not default) and stack gap itself is enabled (default if ASLR is enabled). Stack gap is specified in percentage of the total stack size that can be used for maximum gap.

      The main drawback of the gap approach was shortly identified. Since gap is cut from the normal stack area, attempts of the programs to reduce stack size using rlimit(RLIMIT_STACK) could cut the active stack region if new limit happens to be smaller than the gap. E.g. on amd64 with its default 512M main thread stack, even one percent of the max gap gives approximately 5M of unused stack, that can blow up the limit of several KBs.

      Typical reason for using rlimit(2) this way is for programs that wire all of its address space with mlockall(2), trying to reduce potential wired stack size to avoid exceeding RLIMIT_MEMLOCK.

      First victim of that issue was ntpd, which resets the stack limit after start for a really small value. Currently the wiring was removed from ntpd, because apparently it does not make the timekeeping better by any means, contrary to popular belief.

      My opinion is that the problem is more in the user interface area than in the gap approach itself. We should make it easy to specify small gap sizes, which cannot be done with integral percentage interface. So far I did not formulated a way to do this which I would like, and since nobody looked for a good solution because after ntpd was fixed, the severity of the issue seems very low.

      This project was sponsored by The FreeBSD Foundation.

      Signals delivered on unhandled Page Faults

      Contact: Konstantin Belousov <kib@FreeBSD.org>

      By necessity, handling of page faults is separated into two pieces. The first is the architecture-dependent low-level machine exception handler, and the second is the architecture-independent vm_fault() function in sys/vm/vm_fault.c.

      Machine-dependent code for page faults typically consists of three components: a trampoline written in assembly which creates the C execution environment and gathers hardware-supplied data about page fault reason, a trap() function which is common C-level entry point to dispatch all exceptions processing, and the trap_pfault() C function to specifically handle page faults. trap_pfault() calls vm_fault() when help from VM is needed to resolve the situation.

      If the fault was handled either by trap()/trap_pfault() or vm_fault(), the faulting instruction is restarted. If fault cannot be handled for any reason, the next action depends on the mode in which - the fault occured. + the fault occurred. If it was in kernel, and the kernel installed a helper, then the helper is called instead of returning to the faulting instruction. Otherwise, a kernel-mode page fault causes a panic.

      -

      If the unhandled fault occured in usermode, then all +

      If the unhandled fault occurred in usermode, then all Unixes send a signal to the thread whose execution caused the exception. POSIX (or Single Unix Specification) lists several cases where a signal should be sent, and specifies the signal number and si_code from siginfo that must be supplied with the signal.

      Unfortunately, FreeBSD was rather non-compliant in this regard. A long time ago, to improve compliance, we changed the signal sent on access to a page with permissions incompatible with the access mode. That caused multiple problems with garbage collection (GC)-based runtimes which incorporated knowledge of the FreeBSD quirks, so the machdep.prot_fault_translation sysctl knob was added. More cases of incompatibility were identified recently.

      Part of the problem is that code to calculate the signal and si_code from the Mach error returned by vm_fault() was located in trap_pfault(). In other words, each architecture did that on its own, with a specific set of bugs and non-compliance. Sometimes code even mis-interpreted returned Mach errors as errno.

      A new helper function vm_fault_trap() was added, that does several things for trap handlers: it creates ktrace points for faults, calls vm_fault(), and then interprets the result in terms of the user-visible error condition, returning precalculated signal number and si_code to the caller. Now trap_pfault() only needs to provide signal numbers for truly machine-dependent errors. For amd64, an example of such a case is a protection key violation.

      Besides compliance and bug fixes, we also provided some refinements to userspace about the reason of the delivered signal. For instance, on SIGBUS caused by copy-on-write failure due to exceeding swap reservation limit, we provide specific si_code BUS_OOMERR.

      This project was sponsored by The FreeBSD Foundation.


      Architectures

      Updating platform-specific features and bringing in support for new hardware platforms.


      Broadcom ARM64 SoC support

      Contact: Michal Stanek <mst@semihalf.com>
      Contact: Kornel Duleba <mindal@semihalf.com>
      Contact: Marcin Wojtas <mw@semihalf.com>

      The Semihalf team continued working on FreeBSD support for the Broadcom BCM5871X SoC series

      BCM5871X are quad-core 64-bit ARMv8 Cortex-A57 communication processors targeted for networking applications such as 10G routers, gateways, control plane processing and NAS.

      Completed since the last update:

      • iProc PCIe root complex (internal and external buses): fixes and improvements,
      • Crypto engine acceleration for IPsec offloading.

      Todo:

      • Upstreaming of work. This work is expected to be submitted/merged to HEAD in the Q4 of 2019.

      This project was sponsored by Juniper Networks, Inc.

      FreeBSD support for the forthcoming Arm Morello CPU, SoC, and board

      Contact: Robert Watson <rwatson@FreeBSD.org>
      Contact: Andrew Turner <andrew@FreeBSD.org>
      Contact: Brooks Davis <brooks@FreeBSD.org>

      In September 2019, Arm announced Morello, an experimental multicore superscalar ARMv8-A CPU, SoC, and prototype board extended to implement the CHERI protection model. Morello will become available in 2021. More details can be found in Arm's blog, a Light Blue Touchpaper blog, and the main CHERI website.

      We have updated CheriBSD, a CHERI-adapted version of FreeBSD originally targeted at the MIPS-based SRI/Cambridge CHERI processor prototype, to support the current draft architecture. This includes full userspace spatial and referential memory safety CheriABI, as well as backwards compatibility to support running existing ARMv8-A userspace binaries.

      We will continue to update CheriBSD/Morello as the ISA is finalised. Will also closely track the public CheriBSD/MIPS trunk, picking up new software features utilizing CHERI as they mature, as well as to pick up changes from the baseline FreeBSD development trunk.

      We currently anticipate releasing CheriBSD/Morello as open source once the ISA and toolchain are published in 2020.

      Sponsors: DARPA, AFRL

      FreeBSD/powerpc Project

      Links
      Status of FreeBSD ports on PowerPC URL: https://wiki.freebsd.org/powerpc/ports
      Status of FreeBSD ports on PowerPC built using gcc URL: https://wiki.freebsd.org/powerpc/ports/PortsOnGcc
      Status of FreeBSD ports on PowerPC built using clang URL: https://wiki.freebsd.org/powerpc/ports/PortsOnClang
      Bringing LLVM to PowerPC64 target, using OpenPower ELF v2 ABI URL: https://wiki.freebsd.org/powerpc/llvm-elfv2

      Contact: Mark Linimon (ports) <linimon@FreeBSD.org>
      Contact: Justin Hibbits (src) <jhibbits@FreeBSD.org>
      Contact: Piotr Kubaj (ports) <pkubaj@FreeBSD.org>

      The FreeBSD/powerpc project continues to make great strides. However, as we discovered at BSDCan 2019, we have not done a great job of letting people know.

      Key points:

      • powerpc64 src on recent hardware has been production-quality for over a year now.
      • powerpc64 ports has been developer-quality for over 18 months now.

      The main targeted platforms:

      • powerpc64 on IBM POWER8 and POWER9 chips (the most recent available). This is the primary focus going forward. FreeBSD 12 is required; FreeBSD 13 is recommended.
      • powerpc64 on older Apple Power Macs, on a continuing but secondary basis. Any FreeBSD version should work.
      • powerpc64 on x5000. However, this is still developer-only quality. FreeBSD 13 is recommended.
      • powerpcspe on Amiga A1222. However, this is still developer-only quality. FreeBSD 13 is recommended.

      The software status:

      • powerpc*/12 and earlier still remain on the antiquated gcc4.2.1 in base.
      • powerpc*/13 will soon be switched to llvm90 in base. A great deal of work has been undertaken to ensure as few regressions as possible. Once that switch has occurred (see llvm-elfv2 link above), powerpc*/13 support on gcc4.2.1 will no longer be a priority.
      • FreeBSD.org package sets are available for powerpc64/12 (quarterly) and powerpc64/13 (default) through the usual method.
      • Firefox works on powerpc64 using experimental (not-yet committed) patches,
      • As of the most recent package build on powerpc64/13 (still gcc4.2.1), the following statistics apply:
        Queued Built Failed Skipped Ignored
        33306 30514 245 1686 861
      • More ports fixes are being committed every day.

      Also, Piotr would like to thank the FreeBSD Foundation for funding his personal Talos, and Raptor (via its IntegriCloud subsidiary) for loaning a server on which talos.anongoth.pl runs.

      The team would like to thank IBM for the loan of two POWER8 machines, and Oregon State University (OSU) for providing the hosting. As well, we would like to thank the clusteradm team for keeping the Tyan POWER8 machines online that are hosted at NYI.

      NXP ARM64 SoC support

      Contact: Marcin Wojtas <mw@semihalf.com>
      Contact: Artur Rojek <ar@semihalf.com>

      The Semihalf team initiated working on FreeBSD support for the NXP LS1046A SoC

      LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with integrated packet processing acceleration and high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications.

      Completed since the last update:

      • DPAA Network interface support
      • SD/MMC
      • USB3.0
      • I2C
      • GPIO

      In progress:

      • QSPI
      • Network performance improvements

      Todo:

      • Upstreaming of developed features. This work is expected to be submitted/merged to HEAD in the Q4 of 2019.

      This project was sponsored by Alstom Group.


      Userland Programs

      Changes affecting the base system and programs in it.


      gets(3) retirement

      Contact: Ed Maste <emaste@FreeBSD.org>

      gets is an obsolete C library routine for reading a string from standard input. It was removed from the C standard as of C11 because there was no way to use it safely. Prompted by a comment during Paul Vixie's talk at vBSDCon 2017 I started investigating what it would take to remove gets from libc.

      The patch was posted to Phabricator and refined several times, and the portmgr team performed several exp-runs to identify ports broken by the removal. Symbol versioning is used to preserve binary compatibility for existing software that uses gets.

      The change was committed in September, and will be in FreeBSD 13.0.

      This project was sponsored by The FreeBSD Foundation.


      Ports

      Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.


      FreshPorts

      Links
      FreshPorts URL: https://www.FreshPorts.org/
      git_proc_commit code URL: https://github.com/FreshPorts/git_proc_commit
      Things you didn’t know FreshPorts can do URL: https://news.freshports.org/2019/09/03/things-you-didnt-know-freshports-can-do/

      Contact: Dan Langille <dvl@FreeBSD.org>

      FreshPorts consolidates commits into an easy-to-follow format so you can track changes to your favorite ports. It also processes src, doc, and www commit. FreshPorts parses incoming emails and refreshes the database with what it finds.

      In early September I started looking at how FreshPorts could use a git repository for processing commits. The result was an approach for identifying new commits and for iterating through them.

      The next idea was commit hooks but the most interesting bit of that exercise was commit iteration.

      At the EuroBSDCon 2019 FreeBSD Developer summit, I wrote up a small requirements section and then received great help from two sources:

      • Jan-Piet MENS recommended a Python library and it turned out to be great
      • Sergey Kozlov wrote Python code to create xml using that Python library

      On the trip home, I was able to get the code to parse a git commit into XML and loaded into a FreshPorts database.

      Returning home from the conference, I created a new FreshPorts instance for processing git based on the above. The website has needed no changes related to git.

      The remaining tasks:

      • automate the script (git pull, etc)
      • detect new commits (for later iteration)
      • design a light-weight git hook

      Event: EuroBSDCon 2019 Hackathon Sponsored by: Intel Corporation (work done by Sergey Kozlov)

      Java on FreeBSD

      Links
      OpenJDK 11 repository at FreeBSD GitHub URL: https://github.com/freebsd/openjdk-jdk11u

      Contact: Greg Lewis <glewis@FreeBSD.org>

      Over the last few quarters there has been considerable work in improving support for Java 11 and higher, with some work being backported to Java 8.

      Starting with the initial release in March on amd64, over the intervening months FreeBSD support was added for features such as:

      • Java Flight Recorder
      • HotSpot Serviceability Agent
      • HotSpot Debugger
      • DTrace
      • Javac Server
      • Java Sound
      • SCTP

      In addition, support for the aarch64, i386 and powerpc64 architectures was also added.

      With most features supported, attention turned to compliance, using the internal JDK test suite as a method of measuring this. Most work during the third quarter has focused on this, with test failures dropping from 50+ failures to only 2 tier1 test failures (which don't appear to impact functionality at all). Some significant fixes include:

      • A stack overflow crash
      • Errors in external process handling
      • The unpack200 utility (on little endian systems)
      • HotSpot Debugger functionality such as thread enumeration
      • Networking functionality

      Finally, this work has been forward ported to Java 12 and 13, with FreeBSD gaining support for these versions on or just after the day of release.

      Note that this work has been a collaboration with many others. While there are too many contributors to list here (please take a look at the commit history of the GitHub repository), I'd like to recognise Kurt Miller of OpenBSD for his tireless work as a co-collaborator on Java for BSD through many years.

      I'm also grateful to the sponsorship of the FreeBSD Foundation, which has allowed me to focus on this work for Q3.

      This project was sponsored by FreeBSD Foundation.

      KDE on FreeBSD

      Links
      KDE FreeBSD URL: https://freebsd.kde.org/
      KDE Community FreeBSD URL: https://community.kde.org/FreeBSD

      Contact: Adriaan de Groot <kde@FreeBSD.org>

      The KDE on FreeBSD project packages the software produced by the KDE Community for FreeBSD. The software includes a full desktop environment, the art application https://kdenlive.org and hundreds of other applications that can be used on any FreeBSD desktop machine.

      Along with KDE itself, the team maintains the Qt libraries, the CMake build system, and a handful of other C++ libraries used in the KDE stack.

      The upstream releases KDE Frameworks (libraries) on a monthly cycle, KDE Plasma (the desktop environment) quarterly and a collection of KDE Applications (usable everywhere) twice a year. The KDE on FreeBSD team chased a dozen updates to these components so that FreeBSD remains one of the most up-to-date systems with KDE software (and Qt).

      A large (and possibly breaking, still needs more investigation) change came with the release to KDE's Digikam 6.3.0, which stopped using its previous plugins system (the "old" plugins are still used by other KDE software).

      A new entry in the net-im category was added for Ruqola, a Rocket.chat client for rich instant-messaging.

      CMake was updated twice. This forces the rebuild of several thousand C++-based ports. The KDE on FreeBSD team then fixes those ports, regardless of whether the error is in the CMake update, or in the upstream code. These updates tend to take a large amount of effort, since they touch unfamiliar (and often very-very-legacy) code.

      The open bugs list grew to 24 this quarter. It tends to hover around 20 items as new things come in and old ones are resolved. We welcome detailed bug reports and patches. KDE packaging updates are prepared in a copy of the ports repository on GitHub and then merged in SVN. We welcome pull requests there as well.

      Ports Collection

      Links
      About FreeBSD Ports URL: https://www.FreeBSD.org/ports/
      Contributing to Ports URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html
      FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/index.html
      Ports Management Team URL: https://www.freebsd.org/portmgr/index.html

      Contact: Ren Ladan <portmgr-secretary@FreeBSD.org>
      Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

      The FreeBSD Ports Management Team, tasked with overseeing the Ports Tree and its committers, reports that the following events happened during 2019Q3:

      The number of ports grew to just under 38,000 during the last quarter. We have just over 2,000 open ports PRs, of which 400 are unassigned. In this period, 169 committers made 7,340 commits to HEAD and 561 commits to the quarterly branch. This shows that the trend of last quarter of increased activity is continuing.

      During the last quarter, we welcomed Santhosh Raju (fox@) and Dmitri Goutnik (dmgk@) to the team, and said goodbye to gabor@. During the last quarter, feld@ decided to step down from the portmgr@ and ports-secteam@ teams. We would like to thank them for their past services.

      In the last three months, bapt@ put on his engineering hat and released a new version of pkg (1.12), added support for overlays to the Ports Tree, fixed two Make targets (deinstall-depends and reinstall), and cleaned up bsd.sites.mk.

      On the infrastructure side, USES=pure became obsolete and has therefore been removed, and two new USES, xorg and xorg-cat have been added and replace the old bsd.xorg.mk. Two new keywords, ldconfig and ldconfig-linux, were added to simplify formatting of package lists.

      A number of default versions changed: Lazarus to 2.0.4, Linux to CentOS 7, LLVM to 9.0, Perl to 5.30, PostgreSQL to 11, and Ruby to 2.6. Of the big user-visible ports, Firefox got updated to 69.0.1, Firefox-esr to 68.1.0, Chromium to 76.0.3809.132, and Xfce to 4.14.

      During the last quarter, antoine@ ran 48 exp-runs to test package updates, test updating bsd.java.mk, test the new ldconfig and ldconfig-linux keywords, test default version updates, test the new xorg and xorg-cat USES, test base updates, and test various improvements to Go and Ruby.

      XFCE 4.14 update

      Links
      XFCE 4.14 announce URL: https://xfce.org/about/news/?post=1565568000

      Contact: Guido Falsi <xfce@FreeBSD.org>

      On September 19th the XFCE desktop environment ports have been updated to the recently released XFCE 4.14 version.

      These updates include upgrades of all the main XFCE components to the latest versions which have been migrated to GTK3, with few exceptions. Base components still require and link to GTK2 in addition to GTK3 to allow older GTK2 XFCE applications, for example panel plugins, to keep working.

      Due to this change the gtk-xfce-engine theme is deprecated since it only supports GTK2. The XFCE metaport by default installs the greybird theme, but it is not enabled by default.

      This new version also includes now it's own xfce4-screensaver program which is installed by default.

      Finally the default Display Manager on which XFCE depends has been changed to LightDM.

      Some regressions were reported in bugzilla. In particular the one affecting most users is a regression in the window manager: on specific graphic display hardware the window manager fails to properly draw window decorations, which appear black and blocky on affected systems.

      This problem has also been reported in the upstream bug tracker and a solution is being sought.

      If anyone is experiencing this display glitch and is able to test, please contact xfce@freebsd.org to help trying to figure out a solution.


      Third-Party Projects

      Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.


      ClonOS: virtualization platform on top of FreeBSD Operating System

      Links
      ClonOS 19.09 URL: https://clonos.tekroutine.com/download.html
      CBSD URL: https://www.bsdstore.ru/

      Contact: Oleg Ginzburg <olevole@olevole.ru>

      What is ClonOS?

      ClonOS is a turnkey open-source platform based on FreeBSD and the CBSD framework. ClonOS offers a complete web UI for an easy control, deployment and management of FreeBSD jails containers and bhyve/Xen hypervisor virtual environments.

      ClonOS is currently the only available platforms which allow both Xen and bhyve hypervisors to coexist on the same host. Since ClonOS is a FreeBSD-based platform, it has the ability to create and manage jails natively, allowing you to run FreeBSD applications without losing performance.

      ClonOS/CBSD 2019Q3 Status Report

      • Added support for cloud-init (Linux/BSD VMs) and cloudbase-init (Windows VMs). It gives the ability to use FreeBSD as IaaS platform for instant deployment and usage of virtual machines based on bhyve hypervisor.
      • Project started to use own cloud images for better stability and resiliency.
      • New mirrors in France, US and Canada were added for distributing ISO/cloud-init images in addition to Russia, Latvia and Ukraine.
      • Now we're using Jenkins CI for testing regular ClonOS builds: Update clonos packages (Thanks to Daniel Shafer)
      • New pkg repository was deployed to support ClonOS installation from packages (at this moment only FreeBSD-12 packages are available) ClonOS package repo (Thanks to Daniel Shafer)

      Open issues and tasks:

      • Volunteers, contributors and testers are urgently needed to distribute ClonOS on FreeBSD environments.
      • We'd like to expand our mirrors number geographically, your help and contribution will be much appriciated.
      • We're urgently looking for hosting sponsorship for various developing and testing activities.

      ENA FreeBSD Driver Update

      Links
      ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README

      Contact: Michal Krawczyk <mk@semihalf.com>
      Contact: Maciej Bielski <mba@semihalf.com>
      Contact: Marcin Wojtas <mw@semihalf.com>

      ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used.

      ENAv2 has been under development for FreeBSD, similar to Linux and DPDK. Since the last update internal review and improvements of the patches were done, followed by validation on various AWS instances.

      Completed since the last update:

      • Verification and review of the NETMAP support
      • Mapping of the memory as WC on A1 instances in order to enable LLQ mode

      Todo:

      • Upstream of NETMAP support
      • Upstream of the fix for LLQ mode on A1 instances

      Amazon.com Inc

      Nomad pot driver - Orchestrating jails via nomad

      Links
      Nomad pot driver URL: https://github.com/trivago/nomad-pot-driver
      Pot project URL: https://github.com/pizzamig/pot

      Contact: Luca Pizzamiglio <pizzamig@FreeBSD.org>
      Contact: Esteban Barrios <esteban.barrios@trivago.com>

      An experimental project has started to provide jail orchestration based on nomad and the jail framework pot, similarly to how orchestration works with docker.

      This model allows us to split the jail creation and the jail deployment. Jail images can be created and exported using the pot framework. The images can be deployed and orchestrated using nomad. A driver has been developed to allow nomad to interact with pot.

      One of the goals of this project is to use non-persistent jails as containers, allowing us to:

      • define containers similar to Docker (but not identical, because the underlaying OS is different)
      • identify potential missing features in FreeBSD to support such a computational model

      In the next quarter, we will launch the first service based on this project.

      Next steps are:

      • provide more guides and howtos
      • improve stability, extending the tests suite
      • improving tooling to create/manage images

      This project was sponsored by trivago N.V..

      sysctlinfo

      Links
      gitlab.com/alfix/sysctlinfo URL: https://gitlab.com/alfix/sysctlinfo

      Contact: Alfonso Sabato Siciliano <alfonso.siciliano@email.com>

      The FreeBSD kernel maintains a Management Information Base (MIB) where a component (object) represents a parameter of the system. The sysctl system call explores the MIB to find an object by its Object Identifier (OID) and calls its handler to get or set the value of the parameter. It is often necessary to find an object not to call its handler but to get its info (description, type, name, next object, etc.), so the kernel provides an undocumented interface implemented in kern_sysctl.c.

      sysctlinfo is a new interface to explore the sysctl MIB and to pass the info of an object to the userland. The project provides: a README, a manual, helper macros, examples, tests and converted tools.

      Primarily sysctlinfo provides a new set of sysctl nodes (corresponding to the kernel interface) to handle an object up to CTL_MAXNAME levels: sysctl.entryfakename, sysctl.entrydesc, sysctl.entrylabel, sysctl.entrykind, sysctl.entryfmt and sysctl.entrynextleaf. Moreover new features have been implemented: the support for the capability mode, sysctl.entryname, sysctl.entryidbyname and sysctl.entrynextnode. To get all the info about an object the kernel needs to find it many times, then the new sysctl.entryallinfo* nodes were written, they are 30% more efficient. Finally, *byname nodes were added: they search the object by its name avoiding to call sysctl.name2oid (or similar) to explore the MIB just to find the corresponding OID.

      sysctlinfo can be installed via sysutils/sysctlinfo-kmod or by applying the sysctlinfo.diff patch (more efficient than the module because uses a shared lock). The interface is used by deskutils/sysctlview 1.5, sysutils/nsysctl 1.2 and the converted version of sysctl(8), sysctlbyname(3), sysctlnametomib(3), they should be used to get the value of an object with 23/24 levels or if some level-name has only the '\0' character. In the future a new byname node will be added to allow sysctlbyname() to manage a CTLTYPE_NODE with a no-NULL handler, example sysctlbyname("kern.proc.pid.\<pid\>").

      News Home | Status Home

      diff --git a/website/content/en/status/report-2021-10-2021-12/membarrier-rseq.adoc b/website/content/en/status/report-2021-10-2021-12/membarrier-rseq.adoc index a0362524e1..814598c6aa 100644 --- a/website/content/en/status/report-2021-10-2021-12/membarrier-rseq.adoc +++ b/website/content/en/status/report-2021-10-2021-12/membarrier-rseq.adoc @@ -1,103 +1,103 @@ === sched_getcpu(2), membarrier(2), and rseq(2) syscalls Contact: Konstantin Belousov Links: + link:https://kib.kiev.ua/kib/membarrier.pdf[Linux manpage for membarrier(2)] URL: link:https://kib.kiev.ua/kib/membarrier.pdf[https://kib.kiev.ua/kib/membarrier.pdf] + link:https://reviews.freebsd.org/D32360[membarrier(2) implementation] URL: link:https://reviews.freebsd.org/D32360[https://reviews.freebsd.org/D32360] + link:https://kib.kiev.ua/kib/rseq.pdf[Linux manpage for rseq(2)] URL: link:https://kib.kiev.ua/kib/rseq.pdf[https://kib.kiev.ua/kib/rseq.pdf] + link:https://reviews.freebsd.org/D32505[rseq(2) and userspace bindings implementation] URL: link:https://reviews.freebsd.org/D32505[https://reviews.freebsd.org/D32505] Linux provides a set of syscalls that allow to develop mostly syscall-less scalable algorithms in userspace. The mechanisms are based on optimistic execution using CPU-local data with the assumption that rare events like context switches or signal delivery do not occur for the given calculation, and if they do occur, rollback and restart is performed. This very high-level approach is used, as I understand, for implementation of tools like URCU, fast malloc allocators (tcmalloc) and other userspace infrastructure projects aimed at large partitioned machines. For instance, sched_getcpu(2) syscall returns the CPU id of the CPU where the current thread is currently executing. On amd64, if available, we use a RDTSCP or RDPID instruction to query the CPU id without changing CPU mode, otherwise this is a light-weight syscall. Of course, the answer provided is obsolete the moment it is created, even before it is returned to userspace. But it allows seeding values in some structures that are valid for a long time (at the CPU speed scale) and are automatically corrected on exceptional control flow events like context switches, and userspace can either detect and rollback or sync and rollback with the exceptions. There are two cornerstone syscalls that allow userspace to implement these efficient algorithms: membarrier(2) and rseq(2). Membarrier is a facility that helps implementing fast CPU ordering barriers, typically used for asymmetric/biased locking. In these lock implementation schemes, the owner of the object often assumes that there are contenders/parallel threads that need coordinating with. If some thread starts accessing the same resource, then it is its duty to ensure correctness. Examples of 'traps' that fast code path utilize are reads from a dedicated page that is unmapped by contenders, to switch the fast path to the slow one. Or we could send a signal to all threads that potentially have access to that object, to insert a barrier. Or we can use the membarrier(2) facility, which incurs significantly less overhead than signalling all threads. Membarrier(2) inserts a barrier, which is the typical underlying hardware operation to ensure ordering, into the specified set of CPUs, if these CPUs are executing the specified thread. If these CPUs are not executing the targeted threads, it is assumed that sequential consistency guarantees from the context switch are enough to fulfill the requirement of membarrier(2). Overall, the fast path can be implemented without slow instructions, and the slow path injects required fences into the fast path at the cost of IPI. The facility to detect exceptional conditions in the userspace thread execution was developed in Linux and called rseq(2). It is a feature often called Restartable Atomic Sequences, which explains the acronym. The ability to cheaply do that allows code longer than a single instruction to execute atomically, without the need to propose and implement unsafe operations like disabling preemption, which is not feasible for userspace. For instance, code might use CPU-local resources, which otherwise does not cope well with context switches. There cannot be an analog of critical_enter(9) in userspace. (A facility to cheaply block signal delivery exists in FreeBSD, see sigfastblock(2), but correctly using it is provably too hard to implement in general-purpose code, esp. because it requires version-dependent coordination with rtdl and libthr.) rseq(2) takes per-thread block of memory, where the thread writes the current CPU id (see sched_getcpu(2)) and specifies the block of critical code that must be unwound if an exceptional situation like a context switch occurred while the block was executing. The fast code path uses per-cpu data and typically does not need any corrections, but would a context switch occur, transfer of control to the abort handler informs userspace about the event. So instead of disabling context switches, code can cheaply check for one after the calculation and retry if needed. An interesting rseq(2) implementation detail is that it is impossible (and not needed) to access/update rseq structures from kernel during the actual context switch, because we cannot access userspace from under a spinlock. In other words, threads using rseq do not incur any performance cost from system-global context switches. Instead, if the process registered for rseq(2), on any return to user mode we check if any exceptional events happened while the thread was in the kernel (context switches may happen only while the thread is in kernel mode), and if a context switch indeed -occured, we fire an ast to check whether the program counter is inside the +occurred, we fire an ast to check whether the program counter is inside the critical section and jump to the abort handler if it is. The implementations of membarrier(2) and rseq(2) are clean-room: I used Linux manual pages as the reference and public discussions of the features for clarifying corner cases. On Linux/glibc, there was no stable glibc interface to the rseq facility. One proposed integration was committed then reverted from glibc. It might be prudent to wait some more for the rseq(2) interface to stabilize in glibc before providing it in our libc or to rely on tight integration between kernel and userspace in our base system, and use ABI tricks like symbol versioning to evolve the interface. There is no goal to be 100% compatible with Linux anyway. Sponsor: The FreeBSD Foundation