diff --git a/en_US.ISO8859-1/articles/casestudy-argentina.com/article.sgml b/en_US.ISO8859-1/articles/casestudy-argentina.com/article.sgml index 9382fb4d51..25d586fc19 100644 --- a/en_US.ISO8859-1/articles/casestudy-argentina.com/article.sgml +++ b/en_US.ISO8859-1/articles/casestudy-argentina.com/article.sgml @@ -1,329 +1,329 @@ %articles.ent; ]>
Argentina.com : A Case Study Carlos Horowicz
ch@argentina.com
$FreeBSD$ &tm-attrib.freebsd; &tm-attrib.cvsup; &tm-attrib.intel; &tm-attrib.xfree86; &tm-attrib.general; Overview Argentina.Com is an Argentine ISP with a small infrastructure of fewer than 15 employees and whose primary source of income originates in the free dialup business. It began operation in the year 2000 with barely one server for mail and chat. It has since grown to a market presence in the Argentine free dialup market of 4.5 billion minutes annually. Its most popular product provides nearly half a million users with free e-mail with webmail, POP3 and SMTP access, and 300M disk space. Towards the end of 2002 there were around 50,000 mail users. After two and a half years of re-engineering and consistent technical improvements this ISP has grown by a factor of 3 in terms of billing, and by a factor of 10 with regard to the mail user base. Our competitors in the Argentine market of free dialup include Fullzero which is owned by the Clarin Media Group, Alternativa Gratis, and Tutopia which is funded by IFX and promoted by Hotmail. Some of these large corporate competitors started their free dialup business with multi-million dollar investments and aggressive television and Internet ad campaigns. Argentina.Com does not rely on advertising like these other larger corporations. It has climbed to the fourth position and to an 8% market share during the last two years thanks to superior quality of service. In Argentina and Latin America in general people who do not have computers at home go to so called Locutorios (Internet Centers), where for a few pesos they can use a computer connected to the Internet and usually read and write emails through popular webmails like Hotmail, Yahoo or Argentina.Com. Due to limited financial resources, Argentina.Com made the decision to invest in a new email system instead of publicity in the media. This strategic decision opens the door to a future business in the corporate and paid email arena. The Challenge The main challenge for Argentina.Com is to achieve a dialup uptime of at least 99.95%, or less than 5 hours yearly downtime. Due to the high rotation and volatility in this business, things have to work correctly so the user does not switch -voluntarily or not- the dialup provider or the number he calls to connect. The dialup business involves a support structure to deal with the Telcos about telephony problems and quality of service, plus a technical structure where latency and packet-loss should be minimized due to the UDP nature of Radius and DNS, and where recursive DNS should always be available. This also implies having a high uptime in the POP3 and SMTP services, and in the webmail. For POP3 and SMTP we estimated the need for an uptime equal to the one for dialup, whereas for the webmail we could live with 99.5% which means around two days of yearly downtime. We decided to migrate the email to a proprietary, opensource architecture which should be horizontally scalable, and whose antivirus and antispam infrastructure should support more than just one type of mailstore or back-end. The rough competition in the free email market, mostly due to the recent improvements introduced by Hotmail, Yahoo and Gmail, made it necessary to design the new system with at least 300M user disk space, but at a cost lower than 3 US dollars per GB with some degree of redundancy. Bear in mind that rackmountable hardware is hard to find in Argentina, and is between 30 and 40% more expensive than in the US. Our total budget for equipment acquisition in two years was 75,000 USD, which is only a fraction of our direct competitors' investments. With regard to the antispam service, it became necessary to develop a product that could compete with the systems offered by the big ones. Given the hostile conditions imposed by the existence of spam (dictionary attacks, spams with high degree of obfuscation and refinement, phishing, trojans, mail-bombs, etc.) it becomes very difficult to achieve an excellent uptime while repelling attacks. One must also be careful that the user does not lose mails because of false positives in the classification strategy, that he does not become flooded with spam or spam notifications, and dangerous mails do not make it through to his mailbox. In addition, the technical infrastructure for spam classification should not introduce noticeable delays in the delivery of mails. Finally, the mail system has to be protected from spammers who might misuse it to send spam. The opensource paradigm tends to require hiring large teams of system administrators, operators and programmers who apply patches, correct bugs and integrate platforms. The opposed paradigm is also costly because of expensive software licences, the need for increasingly expensive hardware and a large support staff. So the challenge was to find the right mixture for scarce human and monetary resources, high stability and predictability, and quick and reliable deployment. In Buenos Aires, well-trained Computer Science professionals are hard to find, most of them live and work abroad, while the remaining have stable jobs either at the government or big companies. The FreeBSD solution Introduction At the beginning of 2003 we had a CriticalPath mail system running on Solaris x86 plus a Redhat box for SMTP, Radius and DNS. The DNS and Radius services were constantly down and we were struggling with huge mail queues. There was an attempt to install CriticalPath for Linux into Redhat on an Intel box with a Megaraid card, but the disk latency was enormous and the mail application never really worked. The first step depicted towards the "FreeBSD solution" consisted in migrating this hardware and commercial software to FreeBSD 4.8 with Linux emulation. The choice of FreeBSD The FreeBSD operating system is well-known for its great stability, plus its pragmatism and common sense to put applications on-line thanks to its excellent Ports System. We consider its release engineering process to be easily understandable, while the users' community at the official mailing lists keeps a polite and civilized style when it comes to asking for support or reading other people's problems and solutions. Another important feature is quick deployment. Fortunately, we could state our OS install policy around FreeBSD's great out-of-the-box capability. In a small company you sometimes need to run to a Datacenter and quickly setup a server for some service. In the last two years, Argentina.Com acquired around forty servers, most of them Pentium IV but also several double-Xeons and a few double-Opterons to be co-located in the Datacenters where we have dialup and hosting operation contracts. All of them run FreeBSD, ranging from 4.8 (there are a couple with two years uptime and zero trouble) til currently 6.0-BETA2. The general policy for the operating system is to try to bring all servers periodically to the stable code branch by using RELENG_4, RELENG_5 and now RELENG_6. This regularity lets us be more prepared regarding possible exploits at the operating system or base software level, especially in web servers. Basic re-engineering The first re-engineering step was to put in place two FreeBSD 4.8 boxes whose unique task was to be authoritative DNS for all our domains. The chosen software was Bind9. Those boxes were co-located in different datacenters, taking care that there was good latency between them to avoid zone transfer problems, and making it possible to deal with TTLs between 60 and 600 seconds to have quicker response in case of trouble. Second step was to deploy two more boxes of the same class, again in different Datacenters, to only deal with Radius and recursive DNS. The Network Access Servers at the Telcos were configured to send Radius Authorization and Accounting to those servers, and to assign these recursive DNSs to dialup users. The third golden rule never to put SMTP incoming and outgoing in the same servers. We deployed separate FreeBSD boxes with postfix for incoming and outgoing mail. Email migration The email migration required careful planning due to the fact that we were going to migrate both mail front and back-ends. We first built a perimetral antispam and antivirus system in FreeBSD 4.x and 5.x based on postfix, amavisd-new, clamav and SpamAssassin. These systems were to deliver mails to both the old and the new system until the new back-end was in place. In the meantime, we added small FreeBSD NFS boxes to increase CriticalPath's mailspool, without any problem. At the frontline of incoming mail, we put in place several MXs of the Argentina.com domain to filter dictionary attacks (attempts to forward mail to nonexistent users) as well as a black-list derived from SURBL that resulted in almost no false positives. The mails are then multiplexed to a cluster of double-Xeons and double-Opterons where we run amavisd-new with MySQL based white and black-listing. We discarded the use of Bayes and Autowhitelisting at the global level because of great quantities of false positives and false negatives. We instead defined a few spam levels going from the least to the most tolerant, each one with cutoff or discard levels. Every email with a score below the one associated with the selected spam tolerance goes to the user's Inbox. Emails between this level and the cutoff level go to a user's folder named Spam, and those above the cutoff level get discarded because it is a very obvious spam. For the sake of simplicity, we transparently associated the use of the Address Book with the antispam system, so that every personal contact gets automatically whitelisted. With the introduction of Spamassassin 3.x, the DNS traffic to query global blacklists grew considerably, so we signed agreements with SpamCop, Spamhaus and SURBL to install public mirrors of their databases in our FreeBSD equipment. Thanks to these mirrors that cost us between 1 and 2Mbps in traffic, we were able to dramatically cut down Spamassassin latency. At the 3rd level there is the delivery to the maildrops. As soon as we started building a new Cyrus-Imap back-end with MySQL authentication, we needed to multiplex incoming mail to users in both old and new maildrop formats. Finally, we managed to migrate hundreds of thousands of mailspools to the new Cyrus architecture using a great tool named imapsync, which is directly installable from ports. We also put perdition, a POP3 and IMAP proxy, in the middle to assure a transparent migration and distribution of mailboxes across several servers. Briefly, all information of where a user's maildrop is located resides in MySQL, and is being used by all software pieces in the chain. With regard to the hardware for disk space, we currently use seven Cyrus-Imap loaded FreeBSD boxes with diverse hardware. The biggest are Pentium IV with 4G of RAM and 3ware cards in chassis with 12 hotswappable bays, organized in 3 RAID-5 units of 1 Terabyte each. The 3ware software sends you en email whenever the RAID is degraded -mostly because of a failing disk- and lets you rebuild the RAID with everything up and running. We use smartmontools in the cases where we have less redundancy, to have immediate alerts of disks with temperature problems or failing selftests. As webmail software, we chose a commercial product named Atmail, which is available with perl sources and utilizes mod_perl. Under FreeBSD it is extremely easy to deal with perl modules, you do not even need to use the CPAN shell, you just have to choose the right port and run "make install". After several months of integration work, we integrated the Client-only version of Atmail that talks IMAP with our back-ends. We had to modify some parts of the code to adapt the product to our massive free environment, and to our antispam and antivirus perimeter, in addition to our specific customizations and translations. Web migration With the adoption of FreeBSD, there was almost no additional effort necessary to setup a working Apache, PHP and MySQL environment in minutes. Even the upgrades from PHP4 to PHP5 were - painless. The ports system was again extemely useful in these + painless. The ports system was again extremely useful in these cases, and permitted us to do things like compress text and html contents in Apache with just a few lines of documentation. In addition, we have experienced excellent performance and rock-solid stability and uptime. Results We managed to deploy a FreeBSD based email architecture that is horizontally scalable, using 3 Terabyte Intel based storage servers at a current cost of 3 dollars per Gigabyte with redundancy. The great stability achieved enabled Argentina.Com to explore other fields like hosting for resellers and housing with presence in three Argentine Datacenters. We offer now also corporate dialup for roaming users in Argentina and Peru thanks to our presence and contracts with most Telcos. Among our indirect customers, there are major American companies like Ford, Exxon and Reuters. We now run the free dialup business in Brazil, Chile, Colombia and Panama as well.
diff --git a/en_US.ISO8859-1/articles/cups/article.sgml b/en_US.ISO8859-1/articles/cups/article.sgml index 1139237759..e611c06e3d 100644 --- a/en_US.ISO8859-1/articles/cups/article.sgml +++ b/en_US.ISO8859-1/articles/cups/article.sgml @@ -1,385 +1,385 @@ %articles.ent; ]>
CUPS on FreeBSD $FreeBSD$ Chess Griffin
chess@chessgriffin.com
 &tm-attrib.freebsd; &tm-attrib.general; An article about configuring CUPS on &os;. An Introduction to the Common Unix Printing System (CUPS) printing CUPS CUPS, the Common UNIX Printing System, provides a portable printing layer for &unix;-based operating systems. It has been developed by Easy Software Products to promote a standard printing solution for all &unix; vendors and users. CUPS uses the Internet Printing Protocol (IPP) as the basis for managing print jobs and queues. The Line Printer Daemon (LPD), Server Message Block (SMB), and AppSocket (a.k.a. JetDirect) protocols are also supported with reduced functionality. CUPS adds network printer browsing and PostScript Printer Description (PPD) based printing options to support real-world printing under &unix;. As a result, CUPS is ideally-suited for sharing and accessing printers in mixed environments of &os;, &linux;, &macos; X, or &windows;. The main site for CUPS is . Installing the CUPS Print Server CUPS can be installed from ports or by using a precompiled binary package. To install CUPS from ports, issue the following command from a root terminal: &prompt.root; cd /usr/ports/print/cups && make install clean To install CUPS using a precompiled binary, issue the following command from a root terminal: &prompt.root; pkg_add -r cups Other optional, but recommended, ports or packages are print/gutenprint-cups and print/hplip, both of which add drivers and utilities for a variety of printers. Once installed, the CUPS configuration files can be found in the directory /usr/local/etc/cups. Configuring the CUPS Print Server After installation, a few files must edited in order to configure the CUPS server. First, create or modify, as the case may be, the file /etc/devfs.rules and add the following information to set the proper permissions on all potential printer devices and to associate printers with the cups user group: [system=10] add path 'unlpt*' mode 0660 group cups add path 'ulpt*' mode 0660 group cups add path 'lpt*' mode 0660 group cups Next, add two lines to /etc/rc.conf as follows: cupsd_enable="YES" devfs_system_ruleset="system" These two entries will start the CUPS print server on boot and invoke the local devfs rule created above, respectively. In order to enable CUPS printing under certain µsoft.windows; clients, the line below should be uncommented in /usr/local/etc/cups/mime.types and /usr/local/etc/cups/mime.convs: application/octet-stream Once these changes have been made, the &man.devfs.8; and CUPS systems must both be restarted, either by rebooting the computer or issuing the following two commands in a root terminal: &prompt.root; /etc/rc.d/devfs restart &prompt.root; /usr/local/etc/rc.d/cupsd restart Configuring Printers on the CUPS Print Server After the CUPS system has been installed and configured, the administrator can begin configuring the local printers attached to the CUPS print server. This part of the process is very similar, if not identical, to configuring CUPS printers on other &unix;-based operating systems, such as a &linux; distribution. The primary means for managing and administering the CUPS server is through the web-based interface, which can be found by launching a web browser and entering in the browser's URL bar. If the CUPS server is on another machine on the network, substitute the server's - local IP addresss for + local IP address for localhost. The CUPS web interface is fairly self-explanatory, as there are sections for managing printers and print jobs, authorizing users, and more. Additionally, on the right-hand side of the Administration screen are several check-boxes allowing easy access to commonly-changed settings, such as whether to share published printers connected to the system, whether to allow remote administration of the CUPS server, and whether to allow users additional access and privileges to the printers and print jobs. Adding a printer is generally as easy as clicking Add Printer at the Administration screen of the CUPS web interface, or clicking one of the New Printers Found buttons also at the Administration screen. When presented with the Device drop-down box, simply select the desired locally-attached printer, and then continue through the process. If one has added the print/gutenprint-cups or print/hplip ports or packages as referenced above, then additional print drivers will be available in the subsequent screens that might provide more stability or features. Configuring CUPS Clients Once the CUPS server has been configured and printers have been added and published to the network, the next step is to configure the clients, or the machines that are going to access the CUPS server. If one has a single desktop machine that is acting as both server and client, then much of this information may not be needed. &unix; Clients CUPS will also need to be installed on your &unix; clients. Once CUPS is installed on the clients, then CUPS printers that are shared across the network are often automatically discovered by the printer managers for various desktop environments such as GNOME or KDE. Alternatively, one can access the local CUPS interface on the client machine at and click on Add Printer in the Administration section. When presented with the Device drop-down box, simply select the networked CUPS printer, if it was automatically discovered, or select ipp or http and enter the IPP or HTTP URI of the networked CUPS printer, usually in one of the two following syntaxes: ipp://server-name-or-ip/printers/printername http://server-name-or-ip:631/printers/printername If the CUPS clients have difficulty finding other CUPS printers shared across the network, sometimes it is helpful to add or create a file /usr/local/etc/cups/client.conf with a single entry as follows: ServerName server-ip In this case, server-ip would be replaced by the local IP address of the CUPS server on the network. &windows; Clients Versions of &windows; prior to XP did not have the capability to natively network with IPP-based printers. However, &windowsxp; and later versions do have this capability. Therefore, to add a CUPS printer in these versions of &windows; is quite easy. Generally, the &windows; administrator will run the &windows; Add Printer wizard, select Network Printer and then enter the URI in the following syntax: http://server-name-or-ip:631/printers/printername If one has an older version of &windows; without native IPP printing support, then the general means of connecting to a CUPS printer is to use net/samba3 and CUPS together, which is a topic outside the scope of this chapter. CUPS Troubleshooting Difficulties with CUPS often lies in permissions. First, double check the &man.devfs.8; permissions as outlined above. Next, check the actual permissions of the devices created in the file system. It is also helpful to make sure your user is a member of the cups group. If the permissions check boxes in the Administration section of the CUPS web interface do not seem to be working, another fix might be to manually backup the main CUPS configuration file located at /usr/local/etc/cups/cupsd.conf and edit the various configuration options and try different combinations of configuration options. One sample /usr/local/etc/cups/cupsd.conf to test is listed below. Please note that this sample cupsd.conf file sacrifices security for easier configuration; once the administrator successfully - connnects to the CUPS server and + connects to the CUPS server and configures the clients, it is advisable to revisit this configuration file and begin locking down access. # Log general information in error_log - change "info" to "debug" for # troubleshooting... LogLevel info # Administrator user group... SystemGroup wheel # Listen for connections on Port 631. Port 631 #Listen localhost:631 Listen /var/run/cups.sock # Show shared printers on the local network. Browsing On BrowseOrder allow,deny #BrowseAllow @LOCAL BrowseAllow 192.168.1.* # change to local LAN settings BrowseAddress 192.168.1.* # change to local LAN settings # Default authentication type, when authentication is required... DefaultAuthType Basic DefaultEncryption Never # comment this line to allow encryption # Allow access to the server from any machine on the LAN <Location /> Order allow,deny #Allow localhost Allow 192.168.1.* # change to local LAN settings </Location> # Allow access to the admin pages from any machine on the LAN <Location /admin> #Encryption Required Order allow,deny #Allow localhost Allow 192.168.1.* # change to local LAN settings </Location> # Allow access to configuration files from any machine on the LAN <Location /admin/conf> AuthType Basic Require user @SYSTEM Order allow,deny #Allow localhost Allow 192.168.1.* # change to local LAN settings </Location> # Set the default printer/job policies... <Policy default> # Job-related operations must be done by the owner or an adminstrator... <Limit Send-Document Send-URI Hold-Job Release-Job Restart-Job Purge-Jobs \ Set-Job-Attributes Create-Job-Subscription Renew-Subscription Cancel-Subscription \ Get-Notifications Reprocess-Job Cancel-Current-Job Suspend-Current-Job Resume-Job \ CUPS-Move-Job> Require user @OWNER @SYSTEM Order deny,allow </Limit> # All administration operations require an adminstrator to authenticate... <Limit Pause-Printer Resume-Printer Set-Printer-Attributes Enable-Printer \ Disable-Printer Pause-Printer-After-Current-Job Hold-New-Jobs Release-Held-New-Jobs \ Deactivate-Printer Activate-Printer Restart-Printer Shutdown-Printer Startup-Printer \ Promote-Job Schedule-Job-After CUPS-Add-Printer CUPS-Delete-Printer CUPS-Add-Class \ CUPS-Delete-Class CUPS-Accept-Jobs CUPS-Reject-Jobs CUPS-Set-Default> AuthType Basic Require user @SYSTEM Order deny,allow </Limit> # Only the owner or an administrator can cancel or authenticate a job... <Limit Cancel-Job CUPS-Authenticate-Job> Require user @OWNER @SYSTEM Order deny,allow </Limit> <Limit All> Order deny,allow </Limit> </Policy> Fine Tuning CUPS-Related Ports If CUPS is going to serve as the primary printing system, then one may choose to optionally add certain knobs to /etc/make.conf that will emphasize CUPS over other printing options. Some of these knobs that one may want to add are: WITH_CUPS=YES CUPS_OVERWRITE_BASE=YES WITHOUT_LPR=YES The first knob, WITH_CUPS, adds CUPS support to ports where applicable. The second knob, CUPS_OVERWRITE_BASE, will fix certain symlinks and paths that would otherwise apply to the default &os; printing system, LPR, and will prevent these fixes from being reverted upon the next buildworld system upgrade. The third knob, WITHOUT_LPR, will prevent LPR support from being added to ports where applicable.
diff --git a/en_US.ISO8859-1/articles/ldap-auth/article.sgml b/en_US.ISO8859-1/articles/ldap-auth/article.sgml index bb3f4da8ee..6d9c248380 100644 --- a/en_US.ISO8859-1/articles/ldap-auth/article.sgml +++ b/en_US.ISO8859-1/articles/ldap-auth/article.sgml @@ -1,909 +1,909 @@ %articles.ent; ]>
LDAP Authentication Toby Burress
kurin@causa-sui.net
 $FreeBSD$ 2007 2008 The FreeBSD Documentation Project &tm-attrib.freebsd; &tm-attrib.general; - This document is indended as a guide for the configuration + This document is intended as a guide for the configuration of an LDAP server (principally an OpenLDAP server) for authentication on &os;. This is useful for situations where many servers need the same user accounts, for example as a replacement for NIS. Preface This document is intended to give the reader enough of an understanding of LDAP to configure an LDAP server. This document will attempt to provide an explanation of net/nss_ldap and security/pam_ldap for use with client machines services for use with the LDAP server. When finished, the reader should be able to configure and deploy a &os; server that can host an LDAP directory, and to configure and deploy a &os; server which can authenticate against an LDAP directory. This article is not intended to be an exhaustive account of the security, robustness, or best practice considerations for configuring LDAP or the other services discussed herein. While the author takes care to do everything correctly, he does not address security issues beyond a general scope. This article should be considered to lay the theoretical groundwork only, and any actual implementation should be accompanied by careful requirement analysis. Configuring LDAP LDAP stands for Lightweight Directory Access Protocol and is a subset of the X.500 Directory Access Protocol. Its most recent specifications are in RFC4510 and friends. Essentially it is a database that expects to be read from more often than it is written to. The LDAP server OpenLDAP will be used in the examples in this document; while the principles here should be generally applicable to many different servers, most of the concrete administration is OpenLDAP-specific. There are several server versions in ports, for example net/openldap23-server. Client servers will need the corresponding net/openldap23-client libraries. There are (basically) two areas of the LDAP service which need configuration. The first is setting up a server to receive connections properly, and the second is adding entries to the server's directory so that &os; tools know how to interact with it. Setting Up the Server for Connections This section is specific to OpenLDAP. If you are using another server, you will need to consult that server's documentation. Installing <application>OpenLDAP</application> First, install OpenLDAP: Installing <application>OpenLDAP</application> &prompt.root; cd /usr/ports/net/openldap24-server &prompt.root; make install clean This installs the slapd and slurpd binaries, along with the required OpenLDAP libraries. Configuring <application>OpenLDAP</application> Next we must configure OpenLDAP. You will want to require encryption in your connections to the LDAP server; otherwise your users' passwords will be transferred in plain text, which is considered insecure. The tools we will be using support two very similar kinds of encryption, SSL and TLS. TLS stands for Transportation Layer Security. Services that employ TLS tend to connect on the same ports as the same services without TLS; thus an SMTP server which supports TLS will listen for connections on port 25, and an LDAP server will listen on 389. SSL stands for Secure Sockets Layer, and services that implement SSL do not listen on the same ports as their non-SSL counterparts. Thus SMTPS listens on port 465 (not 45), HTTPS listens on 443, and LDAPS on 636. The reason SSL uses a different port than TLS is because a TLS connection begins as plain text, and switches to encrypted traffic after the STARTTLS directive. SSL connections are encrypted from the beginning. Other than that there are no substantial differences between the two. We will adjust OpenLDAP to use TLS, as SSL is considered deprecated. Once OpenLDAP is installed via ports, the following configuration parameters in /usr/local/etc/openldap/slapd.conf will enable TLS: security ssf=128 TLSCertificateFile /path/to/your/cert.crt TLSCertificateKeyFile /path/to/your/cert.key TLSCACertificateFile /path/to/your/cacert.crt Here, ssf=128 tells OpenLDAP to require 128-bit encryption for all connections, both search and update. This parameter may be configured based on the security needs of your site, but rarely you need to weaken it, as most LDAP client libraries support strong encryption. The cert.crt, cert.key, and cacert.crt files are necessary for clients to authenticate you as the valid LDAP server. If you simply want a server that runs, you can create a self-signed certificate with OpenSSL: Generating an RSA key &prompt.user; openssl genrsa -out cert.key 1024 Generating RSA private key, 1024 bit long modulus ....................++++++ ...++++++ e is 65537 (0x10001) &prompt.user; openssl req -new -key cert.key -out cert.csr At this point you should be prompted for some values. You may enter whatever values you like; however, it is important the Common Name value be the fully qualified domain name of the OpenLDAP server. In our case, and the examples here, the server is server.example.org. Incorrectly setting this value will cause clients to fail when making connections. This can the cause of great frustration, so ensure that you follow these steps closely. Finally, the certificate signing request needs to be signed: Self-signing the certificate &prompt.user; openssl x509 -req -in cert.csr -days 365 -signkey cert.key -out cert.crt Signature ok subject=/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd Getting Private key This will create a self-signed certificate that can be used for the directives in slapd.conf, where cert.crt and cacert.crt are the same file. If you are going to use many OpenLDAP servers (for replication via slurpd) you will want to see to generate a CA key and use it to sign individual server certificates. Once this is done, put the following in /etc/rc.conf: slapd_enable="YES" Then run /usr/local/etc/rc.d/slapd start. This should start OpenLDAP. Confirm that it is listening on 389 with &prompt.user; sockstat -4 -p 389 ldap slapd 3261 7 tcp4 *:389 *:* Configuring the Client Install the net/openldap23-client port for the OpenLDAP libraries. The client machines will always have OpenLDAP libraries since that is all security/pam_ldap and net/nss_ldap support, at least for the moment. The configuration file for the OpenLDAP libraries is /usr/local/etc/openldap/ldap.conf. Edit this file to contain the following values: base dc=example,dc=org uri ldap://server.example.org/ ssl start_tls tls_cacert /path/to/your/cacert.crt It is important that your clients have access to cacert.crt, otherwise they will not be able to connect. There are two files called ldap.conf. The first is this file, which is for the OpenLDAP libraries and defines how to talk to the server. The second is /usr/local/etc/ldap.conf, and is for pam_ldap. At this point you should be able to run ldapsearch -Z on the client machine; means use TLS. If you encounter an error, then something is configured wrong; most likely it is your certificates. Use &man.openssl.1;'s s_client and s_server to ensure you have them configured and signed properly. Entries in the Database Authentication against an LDAP directory is generally - accomplished by attempting to bind to the directory as the connectin user. + accomplished by attempting to bind to the directory as the connecting user. This is done by establishing a simple bind on the directory with the user name supplied. If there is an entry with the uid equal to the user name and that entry's userPassword attribute matches the password supplied, then the bind is successful. The first thing we have to do is figure out is where in the directory our users will live. The base entry for our database is dc=example,dc=org. The default location for users that most clients seem to expect is something like ou=people,base, so that is what will be used here. However keep in mind that this is configurable. So the ldif entry for the people organizational unit will look like: dn: ou=people,dc=example,dc=org objectClass: top objectClass: organizationalUnit ou: people All users will be created as subentries of this organizational unit. Some thought might be given to the object class your users will belong to. Most tools by default will use people, which is fine if you simply want to provide entries against which to authenticate. However, if you are going to store user information in the LDAP database as well, you will probably want to use inetOrgPerson, which has many useful attributes. In either case, the relevant schemas need to be loaded in slapd.conf. For this example we will use the person object class. If you are using inetOrgPerson, the steps are basically identical, except that the sn attribute is required. To add a user testuser, the ldif would be: dn: uid=tuser,ou=people,dc=example,dc=org objectClass: person objectClass: posixAccount objectClass: shadowAccount objectClass: top uidNumber: 10000 gidNumber: 10000 homeDirectory: /home/tuser loginShell: /bin/csh uid: tuser cn: tuser - I start my LDAP users' UIDs at 10000 to avoid colisions with + I start my LDAP users' UIDs at 10000 to avoid collisions with system accounts; you can configure whatever number you wish here, as long as it's less than 65536. We also need group entries. They are as configurable as user entries, but we will use the defaults below: dn: ou=groups,dc=example,dc=org objectClass: top objectClass: organizationalUnit ou: groups dn: cn=tuser,ou=groups,dc=example,dc=org objectClass: posixGroup objectClass: top gidNumber: 10000 cn: tuser To enter these into your database, you can use slapadd or ldapadd on a file containing these entries. Alternatively, you can use sysutils/ldapvi. The ldapsearch utility on the client machine should now return these entries. If it does, your database is properly configured to be used as an LDAP authentication server. Client Configuration The client should already have OpenLDAP libraries from , but if you are installing several client machines you will need to install net/openldap23-client on each of them. &os; requires two ports to be installed to authenticate against an LDAP server, security/pam_ldap and net/nss_ldap. Authentication security/pam_ldap is configured via /usr/local/etc/ldap.conf. This is a different file than the OpenLDAP library functions' configuration file, /usr/local/etc/openldap/ldap.conf; however, it takes many of the same options; in fact it is a superset of that file. For the rest of this section, references to ldap.conf will mean /usr/local/etc/ldap.conf. Thus, we will want to copy all of our original configuration parameters from openldap/ldap.conf to the new ldap.conf. Once this is done, we want to tell security/pam_ldap what to look for on the directory server. We are identifying our users with the uid attribute. To configure this (though it is the default), set the pam_login_attribute directive in ldap.conf: Setting <literal>pam_login_attribute</literal> pam_login_attribute uid With this set, security/pam_ldap will search the entire LDAP directory under base for the value uid=username. If it finds one and only one entry, it will attempt to bind as that user with the password it was given. If it binds correctly, then it will allow access. Otherwise it will fail. PAM PAM, which stands for Pluggable Authentication Modules, is the method by which &os; authenticates most of its sessions. To tell &os; we wish to use an LDAP server, we will have to add a line to the appropriate PAM file. Most of the time the appropriate PAM file is /etc/pam.d/sshd, if you want to use SSH (remember to set the relevant options in /etc/ssh/sshd_config, otherwise SSH will not use PAM). To use PAM for authentication, add the line auth sufficient /usr/local/lib/pam_ldap.so no_warn Exactly where this line shows up in the file and which options appear in the fourth column determine the exact behavior of the authentication mechanism; see &man.pam.d.5; With this configuration you should be able to authenticate a user against an LDAP directory. PAM will perform a bind with your - credentails, and if successful will tell + credentials, and if successful will tell SSH to allow access. However it is not a good idea to allow every user in the directory into every client machine. With the current configuration, all that a user needs to log into a machine is an LDAP entry. Fortunately there are a few ways to restrict user access. ldap.conf supports a pam_groupdn directive; every account that connects to this machine needs to be a member of the group specified here. For example, if you have pam_groupdn cn=servername,ou=accessgroups,dc=example,dc=org in ldap.conf, then only members of that group will be able to log in. There are a few things to bear in mind, however. Members of this group are specified in one or more memberUid attributes, and each attribute must have the full distinguished name of the member. So memberUid: someuser will not work; it must be: memberUid: uid=someuser,ou=people,dc=example,dc=org Additionally, this directive is not checked in PAM during authentication, it is checked during account management, so you will need a second line in your PAM files under account. This will require, in turn, every user to be listed in the group, which is not necessarily what we want. To avoid blocking users that are not in LDAP, you should enable the ignore_unknown_user attribute. Finally, you should set the ignore_authinfo_unavail option so that you are not locked out of every computer when the LDAP server is unavailable. Your pam.d/sshd might then end up looking like this: Sample <filename>pam.d/sshd</filename> auth required pam_nologin.so no_warn auth sufficient pam_opie.so no_warn no_fake_prompts auth requisite pam_opieaccess.so no_warn allow_local auth sufficient /usr/local/lib/pam_ldap.so no_warn auth required pam_unix.so no_warn try_first_pass account required pam_login_access.so account required /usr/local/lib/pam_ldap.so no_warn ignore_authinfo_unavail ignore_unknown_user Since we are adding these lines specifically to pam.d/sshd, this will only have an effect on SSH sessions. LDAP users will be unable to log in at the console. To change this behavior, examine the other files in /etc/pam.d and modify them accordingly. Name Service Switch NSS is the service that maps attributes to names. So, for example, if a file is owned by user 1001, an application will query NSS for the name of 1001, and it might get bob or ted or whatever the user's name is. Now that our user information is kept in LDAP, we need to tell NSS to look there when queried. The net/nss_ldap port does this. It uses the same configuration file as security/pam_ldap, and should not need any extra parameters once it is installed. Instead, what is left is simply to edit /etc/nsswitch.conf to take advantage of the directory. Simply replace the following lines: group: compat passwd: compat with group: files ldap passwd: files ldap This will allow you to map usernames to UIDs and UIDs to usernames. Congratulations! You should now have working LDAP authentication. Caveats Unfortunately, as of the time this was written &os; did not support changing user passwords with &man.passwd.1;. Because of this, most administrators are left to implement a solution themselves. I provide some examples here. Note that if you write your own password change script, there are some security issues you should be made aware of; see Shell script for changing passwords This script does hardly any error checking, but more important it is very cavalier about how it stores your passwords. If you do anything like this, at least adjust the security.bsd.see_other_uids sysctl value: &prompt.root; sysctl security.bsd.see_other_uids=0. A more flexible (and probably more secure) approach can be used by writing a custom program, or even a web interface. The following is part of a Ruby library that can change LDAP passwords. It sees use both on the command line, and on the web. Ruby script for changing passwords Although not guaranteed to be free of security holes (the password is kept in memory, for example) this is cleaner and more flexible than a simple sh script. Security Considerations Now that your machines (and possibly other services) are authenticating against your LDAP server, this server needs to be protected at least as well as /etc/master.passwd would be on a regular server, and possibly even more so since a broken or cracked LDAP server would break every client service. Remember, this section is not exhaustive. You should continually review your configuration and procedures for improvements. Setting attributes read-only Several attributes in LDAP should be read-only. If left writable by the user, for example, a user could change his uidNumber attribute to 0 and get root access! To begin with, the userPassword attribute should not be world-readable. By default, anyone who can connect to the LDAP server can read this attribute. To disable this, put the following in slapd.conf: Hide passwords access to dn.subtree="ou=people,dc=example,dc=org" attrs=userPassword by self write by anonymous auth by * none access to * by self write by * read This will disallow reading of the userPassword attribute, while still allowing users to change their own passwords. Additionally, you'll want to keep users from changing some of their own attributes. By default, users can change any attribute (except for those which the LDAP schemas themselves deny changes), such as uidNumber. To close this hole, modify the above to Read-only attributes access to dn.subtree="ou=people,dc=example,dc=org" attrs=userPassword by self write by anonymous auth by * none access to attrs=homeDirectory,uidNumber,gidNumber by * read access to * by self write by * read This will stop users from being able to masquerade as other users. <username>Root</username> account definition Often the root or manager account for the LDAP service will be defined in the configuration file. OpenLDAP supports this, for example, and it works, but it can lead to trouble if slapd.conf is compromised. It may be better to use this only to bootstrap yourself into LDAP, and then define a root account there. Even better is to define accounts that have limited permissions, and omit a root account entirely. For example, users to can add or remove user accounts are added to one group, but they cannot themselves change the membership of this group. Such a security policy would help mitigate the effects of a leaked password. Creating a management group Say you want your IT department to be able to change home directories for users, but you don't want all of them to be able to add or remove users. The way to do this is to add a group for these admins: Creating a management group dn: cn=homemanagement,dc=example,dc=org objectClass: top objectClass: posixGroup cn: homemanagement gidNumber: 121 # required for posixGroup memberUid: uid=tuser,ou=people,dc=example,dc=org memberUid: uid=user2,ou=people,dc=example,dc=org And then change the permissions attributes in slapd.conf: ACLs for a home directory management group access to dn.subtree="ou=people,dc=example,dc=org" attr=homeDirectory by dn="cn=homemanagement,dc=example,dc=org" dnattr=memberUid write Now tuser and user2 can change other users' home directories. In this example we've given a subset of administrative power to certain users without giving them power in other domains. The idea is that soon no single user account has the power of a root account, but every power root had is had by at least one user. The root account then becomes unnecessary and can be removed. Password storage By default OpenLDAP will store the value of the userPassword attribute as it stores any other data: in the clear. Most of the time it is base 64 encoded, which provides enough protection to keep an honest administrator from knowing your password, but little else. It is a good idea, then, to store passwords in a more secure format, such as SSHA (salted SHA). This is done by whatever program you use to change users' passwords. Useful Aids There are a few other programs that might be useful, particularly if you have many users and do not want to configure everything manually. security/pam_mkhomedir is a PAM module that always succeeds; its purpose is to create home directories for users which do not have them. If you have dozens of client servers and hundreds of users, it is much easier to use this and set up skeleton directories than to prepare every home directory. sysutils/cpu is a &man.pw.8;-like utility that can be used to manage users in the LDAP directory. You can call it directly, or wrap scripts around it. It can handle both TLS (with the flag) and SSL (directly). sysutils/ldapvi is a great utility for editing LDAP values in an LDIF-like syntax. The directory (or subsection of the directory) is presented in the editor chosen by the EDITOR environment variable. This makes it easy to enable large-scale changes in the directory without having to write a custom tool. security/openssh-portable has the ability to contact an LDAP server to verify SSH keys. This is extremely nice if you have many servers and do not want to copy your public keys across all of them. <application>OpenSSL</application> Certificates For LDAP If you are hosting two or more LDAP servers, you will probably not want to use self-signed certificates, since each client will have to be configured to work with each certificate. While this is possible, it is not nearly as simple as creating your own certificate authority, and signing your servers' certificates with that. The steps here are presented as they are with very little attempt at explaining what is going on—further explanation can be found in &man.openssl.1; and its friends. To create a certificate authority, we simply need a self-signed certificate and key. The steps for this again are Creating a certificate &prompt.user; openssl genrsa -out root.key 1024 &prompt.user; openssl req -new -key root.key -out root.csr &prompt.user; openssl x509 -req -days 1024 -in root.csr -signkey root.key -out root.crt These will be your root CA key and certificate. You will probably want to encrypt the key and store it in a cool, dry place; anyone with access to it can masquerade as one of your LDAP servers. Next, using the first two steps above create a key ldap-server-one.key and certificate signing request ldap-server-one.csr. Once you sign the signing request with root.key, you will be able to use ldap-server-one.* on your LDAP servers. Do not forget to use the fully qualified domain name for the common name attribute when generating the certificate signing request; otherwise clients will reject a connection with you, and it can be very tricky to diagnose. To sign the key, use and instead of : - Signing as a certificate authorty + Signing as a certificate authority &prompt.user; openssl x509 -req -days 1024 \ -in ldap-server-one.csr -CA root.crt -CAkey root.key \ -out ldap-server-one.crt The resulting file will be the certificate that you can use on your LDAP servers. Finally, for clients to trust all your servers, distribute root.crt (the certificate, not the key!) to each client, and specify it in the TLSCACertificateFile directive in ldap.conf.
diff --git a/en_US.ISO8859-1/articles/linux-emulation/article.sgml b/en_US.ISO8859-1/articles/linux-emulation/article.sgml index 09028436bb..03c46148dd 100644 --- a/en_US.ISO8859-1/articles/linux-emulation/article.sgml +++ b/en_US.ISO8859-1/articles/linux-emulation/article.sgml @@ -1,2377 +1,2377 @@ %articles.ent; ]>
&linux; emulation in &os; Roman Divacky
rdivacky@FreeBSD.org
 &tm-attrib.adobe; &tm-attrib.ibm; &tm-attrib.freebsd; &tm-attrib.linux; &tm-attrib.netbsd; &tm-attrib.realnetworks; &tm-attrib.oracle; &tm-attrib.sun; &tm-attrib.general; This masters thesis deals with updating the &linux; emulation layer (the so called Linuxulator). The task was to update the layer to match the functionality of &linux; 2.6. As a reference implementation, the &linux; 2.6.16 kernel was chosen. The concept is loosely based on the NetBSD implementation. Most of the work was done in the summer of 2006 as a part of the Google Summer of Code students program. The focus was on bringing the NPTL (new &posix; thread library) support into the emulation layer, including TLS (thread local storage), futexes (fast user space mutexes), PID mangling, and some other minor things. Many small problems were identified and fixed in the process. My work was integrated into the main &os; source repository and will be shipped in the upcoming 7.0R release. We, the emulation development team, are working on making the &linux; 2.6 emulation the default emulation layer in &os;. Introduction In the last few years the open source &unix; based operating systems started to be widely deployed on server and client machines. Among these operating systems I would like to point out two: &os;, for its BSD heritage, time proven code base and many interesting features and &linux; for its wide user base, enthusiastic open developer community and support from large companies. &os; tends to be used on server class machines serving heavy duty networking tasks with less usage on desktop class machines for ordinary users. While &linux; has the same usage on servers, but it is used much more by home based users. This leads to a situation where there are many binary only programs available for &linux; that lack support for &os;. Naturally, a need for the ability to run &linux; binaries on a &os; system arises and this is what this thesis deals with: the emulation of the &linux; kernel in the &os; operating system. During the Summer of 2006 Google Inc. sponsored a project which focused on extending the &linux; emulation layer (the so called Linuxulator) in &os; to include &linux; 2.6 facilities. This thesis is written as a part of this project. A look inside… In this section we are going to describe every operating system in question. How they deal with syscalls, trapframes etc. all the low-level stuff. We also describe the way they understand common &unix; primitives like what a PID is, what a thread is, etc. In the third subsection we talk about how &unix; on &unix; emulation could be done in general. What is &unix; &unix; is an operating system with a long history that has influenced almost every other operating system currently in use. Starting in the 1960s, its development continues to this day (although in different projects). &unix; development soon forked into two main ways: the BSDs and System III/V families. They mutually influenced themselves by growing a common &unix; standard. Among the contributions originated in BSD we can name virtual memory, TCP/IP networking, FFS, and many others. The System V branch contributed to SysV interprocess communication primitives, copy-on-write, etc. &unix; itself does not exist any more but its ideas have been used by many other operating systems world wide thus forming the so called &unix;-like operating systems. These days the most influential ones are &linux;, Solaris, and possibly (to some extent) &os;. There are in-company &unix; derivatives (AIX, HP-UX etc.), but these have been more and more migrated to the aforementioned systems. Let us summarize typical &unix; characteristics. Technical details Every running program constitutes a process that represents a state of the computation. Running process is divided between kernel-space and user-space. Some operations can be done only from kernel space (dealing with hardware etc.), but the process should spend most of its lifetime in the user space. The kernel is where the management of the processes, hardware, and low-level details take place. The kernel provides a standard unified &unix; API to the user space. The most important ones are covered below. Communication between kernel and user space process Common &unix; API defines a syscall as a way to issue commands from a user space process to the kernel. The most common implementation is either by using an interrupt or specialized instruction (think of SYSENTER/SYSCALL instructions for ia32). Syscalls are defined by a number. For example in &os;, the syscall number 85 is the &man.swapon.2; syscall and the syscall number 132 is &man.mkfifo.2;. Some syscalls need parameters, which are passed from the user-space to the kernel-space in various ways (implementation dependant). Syscalls are synchronous. Another possible way to communicate is by using a trap. Traps occur asynchronously after some event occurs (division by zero, page fault etc.). A trap can be transparent for a process (page fault) or can result in a reaction like sending a signal (division by zero). Communication between processes There are other APIs (System V IPC, shared memory etc.) but the single most important API is signal. Signals are sent by processes or by the kernel and received by processes. Some signals can be ignored or handled by a user supplied routine, some result in a predefined action that cannot be altered or ignored. Process management Kernel instances are processed first in the system (so called init). Every running process can create its identical copy using the &man.fork.2; syscall. Some slightly modified versions of this syscall were introduced but the basic semantic is the same. Every running process can morph into some other process using the &man.exec.3; syscall. Some modifications of this syscall were introduced but all serve the same basic purpose. Processes end their lives by calling the &man.exit.2; syscall. Every process is identified by a unique number called PID. Every process has a defined parent (identified by its PID). Thread management Traditional &unix; does not define any API nor implementation for threading, while &posix; defines its threading API but the implementation is undefined. Traditionally there were two ways of implementing threads. Handling them as separate processes (1:1 threading) or envelope the whole thread group in one process and managing the threading in userspace (1:N threading). Comparing main features of each approach: 1:1 threading - heavyweight threads - the scheduling cannot be altered by the user (slightly mitigated by the &posix; API) + no syscall wrapping necessary + can utilize multiple CPUs 1:N threading + lightweight threads + scheduling can be easily altered by the user - syscalls must be wrapped - cannot utilize more than one CPU What is &os;? The &os; project is one of the oldest open source operating systems currently available for daily use. It is a direct descendant of the genuine &unix; so it could be claimed that it is a true &unix; although licensing issues do not permit that. The start of the project dates back to the early 1990's when a crew of fellow BSD users patched the 386BSD operating system. Based on this patchkit a new operating system arose named &os; for its liberal license. Another group created the NetBSD operating system with different goals in mind. We will focus on &os;. &os; is a modern &unix;-based operating system with all the features of &unix;. Preemptive multitasking, multiuser facilities, TCP/IP networking, memory protection, symmetric multiprocessing support, virtual memory with merged VM and buffer cache, they are all there. One of the interesting and extremely useful features is the ability to emulate other &unix;-like operating systems. As of December 2006 and 7-CURRENT development, the following emulation functionalities are supported: &os;/i386 emulation on &os;/amd64 &os;/i386 emulation on &os;/ia64 &linux;-emulation of &linux; operating system on &os; NDIS-emulation of Windows networking drivers interface NetBSD-emulation of NetBSD operating system PECoff-support for PECoff &os; executables SVR4-emulation of System V revision 4 &unix; Actively developed emulations are the &linux; layer and various &os;-on-&os; layers. Others are not supposed to work properly nor be usable these days. &os; development happens in a central CVS repository where only - a selected team of so called commiters can write. This repository - posseses several branches; the most interesting are the HEAD branch, + a selected team of so called committers can write. This repository + possesses several branches; the most interesting are the HEAD branch, in &os; nomenclature called -CURRENT, and RELENG_X branches, where X stands for a number indicating a major version of &os;. As of December 2006, there are development branches for 6.X development (RELENG_6) and for the 5.X development (RELENG_5). Other branches are closed and not actively maintained or only fed with security patches by the Security Officer of the &os; project. Historically the active development was done in the HEAD branch so it was considered extremely unstable and supposed to happen to break at any time. This is not true any more as the Perforce (commercial version control system) repository was introduced so that active development happen there. There are many branches in Perforce where development of certain parts of the system happens and these branches are from time to time merged back to the main CVS repository thus effectively putting the given feature to the &os; operating system. The same happened with the rdivacky_linuxolator branch where development of this thesis code was going on. More info about the &os; operating system can be found at [2]. Technical details &os; is traditional flavor of &unix; in the sense of dividing the run of processes into two halves: kernel space and user space run. There are two types of process entry to the kernel: a syscall and a trap. There is only one way to return. In the subsequent sections we will describe the three gates to/from the kernel. The whole description applies to the i386 architecture as the Linuxulator only exists there but the concept is similar on other architectures. The information was taken from [1] and the source code. System entries &os; has an abstraction called an execution class loader, which is a wedge into the &man.execve.2; syscall. This employs a structure sysentvec, which describes an executable ABI. It contains things like errno translation table, signal translation table, various functions to serve syscall needs (stack fixup, coredumping, etc.). Every ABI the &os; kernel wants to support must define this structure, as it is used later in the syscall processing code and at some other places. System entries are handled by trap handlers, where we can access both the kernel-space and the user-space at once. Syscalls Syscalls on &os; are issued by executing interrupt 0x80 with register %eax set to a desired syscall number with arguments passed on the stack. When a process issues an interrupt 0x80, the int0x80 syscall trap handler is issued (defined in sys/i386/i386/exception.s), which prepares arguments (i.e. copies them on to the stack) for a call to a C function &man.syscall.2; (defined in sys/i386/i386/trap.c), which processes the passed in trapframe. The processing consists of preparing the syscall (depending on the sysvec entry), determining if the syscall is 32-bit or 64-bit one (changes size of the parameters), then the parameters are copied, including the syscall. Next, the actual syscall function is executed with processing of the return code (special cases for ERESTART and EJUSTRETURN errors). Finally an userret() is scheduled, switching the process back to the users-pace. The parameters to the actual syscall handler are passed in the form of struct thread *td, struct syscall args * arguments where the second parameter is a pointer to the copied in structure of parameters. Traps Handling of traps in &os; is similar to the handling of syscalls. Whenever a trap occurs, an assembler handler is called. It is chosen between alltraps, alltraps with regs pushed or calltrap depending on the type of the trap. This handler prepares arguments for a call to a C function trap() (defined in sys/i386/i386/trap.c), which then processes the occurred trap. After the processing it might send a signal to the process and/or exit to userland using userret(). Exits Exits from kernel to userspace happen using the assembler routine doreti regardless of whether the kernel was entered via a trap or via a syscall. This restores the program status from the stack and returns to the userspace. &unix; primitives &os; operating system adheres to the traditional &unix; scheme, where every process has a unique identification number, the so called PID (Process ID). PID numbers are allocated either linearly or randomly ranging from 0 to PID_MAX. The allocation of PID numbers is done using linear searching of PID space. Every thread in a process receives the same PID number as result of the &man.getpid.2; call. There are currently two ways to implement threading in &os;. The first way is M:N threading followed by the 1:1 threading model. The default library used is M:N threading (libpthread) and you can switch at runtime to 1:1 threading (libthr). The plan is to switch to 1:1 library by default soon. Although those two libraries use the same kernel primitives, they are accessed through different API(es). The M:N library uses the kse_* family of syscalls while the 1:1 library uses the thr_* family of syscalls. Because of this, there is no general concept of thread ID shared between kernel and userspace. Of course, both threading libraries implement the pthread thread ID API. Every kernel thread (as described by struct thread) has td tid identifier but this is not directly accessible from userland and solely serves the kernel's needs. It is also used for 1:1 threading library as pthread's thread ID but handling of this is internal to the library and cannot be relied on. As stated previously there are two implementations of threading in &os;. The M:N library divides the work between kernel space and userspace. Thread is an entity that gets scheduled in the kernel but it can represent various number of userspace threads. M userspace threads get mapped to N kernel threads thus saving resources while keeping the ability to exploit multiprocessor parallelism. Further information about the implementation can be obtained from the man page or [1]. The 1:1 library directly maps a userland thread to a kernel thread thus greatly simplifying the scheme. None of these designs implement a fairness mechanism (such a mechanism was implemented but it was removed recently because it caused serious slowdown and made the code more difficult to deal with). What is &linux; &linux; is a &unix;-like kernel originally developed by Linus Torvalds, and now being contributed to by a massive crowd of programmers all around the world. From its mere beginnings to todays, with wide support from companies such as IBM or Google, &linux; is being associated with its fast development pace, full hardware support and benevolent dictator model of organization. &linux; development started in 1991 as a hobbyist project at University of Helsinki in Finland. Since then it has obtained all the features of a modern &unix;-like OS: multiprocessing, multiuser support, virtual memory, networking, basically everything is there. There are also highly advanced features like virtualization etc. As of 2006 &linux; seems to be the most widely used open source operating system with support from independent software vendors like Oracle, RealNetworks, Adobe, etc. Most of the commercial software distributed for &linux; can only be obtained in a binary form so recompilation for other operating systems is impossible. Most of the &linux; development happens in a Git version control system. Git is a distributed system so there is no central source of the &linux; code, but some branches are considered prominent and official. The version number scheme implemented by &linux; consists of four numbers A.B.C.D. Currently development happens in 2.6.C.D, where C represents major version, where new features are added or changed while D is a minor version for bugfixes only. More information can be obtained from [4]. Technical details &linux; follows the traditional &unix; scheme of dividing the run of a process in two halves: the kernel and user space. The kernel can be entered in two ways: via a trap or via a syscall. The return is handled only in one way. The further description applies to &linux; 2.6 on the &i386; architecture. This information was taken from [3]. Syscalls Syscalls in &linux; are performed (in userspace) using syscallX macros where X substitutes a number representing the number of parameters of the given syscall. This macro translates to a code that loads %eax register with a number of the syscall and executes interrupt 0x80. After this syscall return is called, which translates negative return values to positive errno values and sets res to -1 in case of an error. Whenever the interrupt 0x80 is called the process enters the kernel in system call trap handler. This routine saves all registers on the stack and calls the selected syscall entry. Note that the &linux; calling convention expects parameters to the syscall to be passed via registers as shown here: parameter -> %ebx parameter -> %ecx parameter -> %edx parameter -> %esi parameter -> %edi parameter -> %ebp There are some exceptions to this, where &linux; uses different calling convention (most notably the clone syscall). Traps The trap handlers are introduced in arch/i386/kernel/traps.c and most of these handlers live in arch/i386/kernel/entry.S, where handling of the traps happens. Exits Return from the syscall is managed by syscall &man.exit.3;, which checks for the process having unfinished work, then checks whether we used user-supplied selectors. If this happens stack fixing is applied and finally the registers are restored from the stack and the process returns to the userspace. &unix; primitives In the 2.6 version, the &linux; operating system redefined some of the traditional &unix; primitives, notably PID, TID and thread. PID is defined not to be unique for every process, so for some processes (threads) &man.getppid.2; returns the same value. Unique identification of process is provided by TID. This is because NPTL (New &posix; Thread Library) defines threads to be normal processes (so called 1:1 threading). Spawning a new process in &linux; 2.6 happens using the clone syscall (fork variants are reimplemented using it). This clone syscall defines a set of flags that affect behaviour of the cloning process regarding thread implementation. The semantic is a bit fuzzy as there is no single flag telling the syscall to create a thread. Implemented clone flags are: CLONE_VM - processes share their memory space CLONE_FS - share umask, cwd and namespace CLONE_FILES - share open files CLONE_SIGHAND - share signal handlers and blocked signals CLONE_PARENT - share parent CLONE_THREAD - be thread (further explanation below) CLONE_NEWNS - new namespace CLONE_SYSVSEM - share SysV undo structures CLONE_SETTLS - setup TLS at supplied address CLONE_PARENT_SETTID - set TID in the parent CLONE_CHILD_CLEARTID - clear TID in the child CLONE_CHILD_SETTID - set TID in the child CLONE_PARENT sets the real parent to the parent of the caller. This is useful for threads because if thread A creates thread B we want thread B to be parented to the parent of the whole thread group. CLONE_THREAD does exactly the same thing as CLONE_PARENT, CLONE_VM and CLONE_SIGHAND, rewrites PID to be the same as PID of the caller, sets exit signal to be none and enters the thread group. CLONE_SETTLS sets up GDT entries for TLS handling. The CLONE_*_*TID set of flags sets/clears user supplied address to TID or 0. As you can see the CLONE_THREAD does most of the work and does not seem to fit the scheme very well. The original intention is unclear (even for authors, according to comments in the code) but I think originally there was one threading flag, which was then parcelled among many other flags but this separation was never fully finished. It is also unclear what this partition is good for as glibc does not use that so only hand-written use of the clone permits a programmer to access this features. For non-threaded programs the PID and TID are the same. For threaded programs the first thread PID and TID are the same and every created thread shares the same PID and gets assigned a unique TID (because CLONE_THREAD is passed in) also parent is shared for all processes forming this threaded program. The code that implements &man.pthread.create.3; in NPTL defines the clone flags like this: int clone_flags = (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SIGNAL | CLONE_SETTLS | CLONE_PARENT_SETTID | CLONE_CHILD_CLEARTID | CLONE_SYSVSEM #if __ASSUME_NO_CLONE_DETACHED == 0 | CLONE_DETACHED #endif | 0); The CLONE_SIGNAL is defined like #define CLONE_SIGNAL (CLONE_SIGHAND | CLONE_THREAD) the last 0 means no signal is sent when any of the threads exits. What is emulation According to a dictionary definition, emulation is the ability of a program or device to imitate another program or device. This is achieved by providing the same reaction to a given stimulus as the emulated object. In practice, the software world mostly sees three types of emulation - a program used to emulate a machine (QEMU, various game console emulators etc.), software emulation of a hardware facility (OpenGL emulators, floating point units emulation etc.) and operating system emulation (either in kernel of the operating system or as a userspace program). Emulation is usually used in a place, where using the original component is not feasible nor possible at all. For example someone might want to use a program developed for a different operating system than he uses. Then emulation comes in handy. Sometimes there is no other way but to use emulation - e.g. when the hardware device you try to use does not exist (yet/anymore) then there is no other way but emulation. This happens often when porting an operating system to a new (non-existent) platform. Sometimes it is just cheaper to emulate. Looking from an implementation point of view, there are two main approaches to the implementation of emulation. You can either emulate the whole thing - accepting possible inputs of the original object, maintaining inner state and emitting correct output based on the state and/or input. This kind of emulation does not require any special conditions and basically can be implemented anywhere for any device/program. The drawback is that implementing such emulation is quite difficult, time-consuming and error-prone. In some cases we can use a simpler approach. Imagine you want to emulate a printer that prints from left to right on a printer that prints from right to left. It is obvious that there is no need for a complex emulation layer but simply reversing of the printed text is sufficient. Sometimes the emulating environment is very similar to the emulated one so just a thin layer of some translation is necessary to provide fully working emulation! As you can see this is much less demanding to implement, so less time-consuming and error-prone than the previous approach. But the necessary condition is that the two environments must be similar enough. The third approach combines the two previous. Most of the time the objects do not provide the same capabilities so in a case of emulating the more powerful one on the less powerful we have to emulate the missing features with full emulation described above. This master thesis deals with emulation of &unix; on &unix;, which is exactly the case, where only a thin layer of translation is sufficient to provide full emulation. The &unix; API consists of a set of syscalls, which are usually self contained and do not affect some global kernel state. There are a few syscalls that affect inner state but this can be dealt with by providing some structures that maintain the extra state. No emulation is perfect and emulations tend to lack some parts but this usually does not cause any serious drawbacks. Imagine a game console emulator that emulates everything but music output. No doubt that the games are playable and one can use the emulator. It might not be that comfortable as the original game console but its an acceptable compromise between price and comfort. The same goes with the &unix; API. Most programs can live with a very limited set of syscalls working. Those syscalls tend to be the oldest ones (&man.read.2;/&man.write.2;, &man.fork.2; family, &man.signal.3; handling, &man.exit.3;, &man.socket.2; API) hence it is easy to emulate because their semantics is shared among all &unix;es, which exist todays. Emulation How emulation works in &os; As stated earlier, &os; supports running binaries from several other &unix;es. This works because &os; has an abstraction called the execution class loader. This wedges into the &man.execve.2; syscall, so when &man.execve.2; is about to execute a binary it examines its type. There are basically two types of binaries in &os;. Shell-like text scripts which are identified by #! as their first two characters and normal (typically ELF) binaries, which are a representation of a compiled executable object. The vast majority (one could say all of them) of binaries in &os; are from type ELF. ELF files contain a header, which specifies the OS ABI for this ELF file. By reading this information, the operating system can accurately determine what type of binary the given file is. Every OS ABI must be registered in the &os; kernel. This applies to the &os; native OS ABI, as well. So when &man.execve.2; executes a binary it iterates through the list of registered APIs and when it finds the right one it starts to use the information contained in the OS ABI description (its syscall table, errno translation table, etc.). So every time the process calls a syscall, it uses its own set of syscalls instead of some global one. This effectively provides a very elegant and easy way of supporting execution of various binary formats. The nature of emulation of different OSes (and also some other subsystems) led developers to invite a handler event mechanism. There are various places in the kernel, where a list of event handlers are called. Every subsystem can register an event handler and they are called accordingly. For example, when a process exits there is a handler called that possibly cleans up whatever the subsystem needs to be cleaned. Those simple facilities provide basically everything that is needed for the emulation infrastructure and in fact these are basically the only things necessary to implement the &linux; emulation layer. Common primitives in the &os; kernel Emulation layers need some support from the operating system. I am going to describe some of the supported primitives in the &os; operating system. Locking primitives Contributed by: &a.attilio; The &os; synchronization primitive set is based on the idea to supply a rather huge number of different primitives in a way that the better one can be used for every particular, appropriate situation. To a high level point of view you can consider three kinds of synchronization primitives in the &os; kernel: atomic operations and memory barriers locks scheduling barriers Below there are descriptions for the 3 families. For every lock, you should really check the linked manpage (where possible) for more detailed explanations. Atomic operations and memory barriers Atomic operations are implemented through a set of functions - performing simple aritmetics on memory operands in an atomic way + performing simple arithmetics on memory operands in an atomic way with respect to external events (interrupts, preemption, etc.). Atomic operations can guarantee atomicity just on small data types (in the magnitude order of the .long. architecture C data type), so should be rarely used directly in the end-level code, if not only for very simple operations (like flag setting in a bitmap, for example). In fact, it is rather simple and common to write down a wrong semantic based on just atomic operations (usually referred as lock-less). The &os; kernel offers a way to perform atomic operations in conjunction with a memory barrier. The memory barriers will guarantee that an atomic operation will happen following some specified ordering with respect to other memory accesses. For example, if we need that an atomic operation happen just after all other pending writes (in terms of instructions reordering buffers activities) are completed, we need to explicitly use a memory barrier in conjunction to this atomic operation. So it is simple to understand why memory barriers play a key role for higher-level locks building (just as refcounts, mutexes, etc.). For a detailed explanatory on atomic operations, please refer to &man.atomic.9;. It is far, however, noting that atomic operations (and memory barriers as well) should ideally only be used for building front-ending locks (as mutexes). Refcounts Refcounts are interfaces for handling reference counters. They are implemented through atomic operations and are intended to be used just for cases, where the reference counter is the only one thing to be protected, so even something like a spin-mutex is deprecated. Using the refcount interface for structures, where a mutex is already used is often wrong since we should probably close the reference counter in some already protected paths. A manpage discussing refcount does not exist currently, just check sys/refcount.h for an overview of the existing API. Locks &os; kernel has huge classes of locks. Every lock is defined by some peculiar properties, but probably the most important is the event linked to contesting holders (or in other terms, the behaviour of threads unable to acquire the lock). &os;'s locking scheme presents three different behaviours for contenders: spinning blocking sleeping numbers are not casual Spinning locks Spin locks let waiters to spin until they cannot acquire the lock. An important matter do deal with is when a thread contests on a spin lock if it is not descheduled. Since the &os; kernel is preemptive, this exposes spin lock at the risk of deadlocks that can be solved just disabling interrupts while they are acquired. For this and other reasons (like lack of priority propagation support, poorness in load balancing schemes between CPUs, etc.), spin locks are intended to protect very small paths of code, or ideally not to be used at all if not explicitly requested (explained later). Blocking Block locks let waiters to be descheduled and blocked until the lock owner does not drop it and wakes up one or more contenders. In order to avoid starvation issues, blocking locks do priority propagation from the waiters to the owner. Block locks must be implemented through the turnstile interface and are intended to be the most used kind of locks in the kernel, if no particular conditions are met. Sleeping Sleep locks let waiters to be descheduled and fall asleep until the lock holder does not drop it and wakes up one or more waiters. Since sleep locks are intended to protect large paths of code and to cater asynchronous events, they do not do any form of priority propagation. They must be implemented through the &man.sleepqueue.9; interface. The order used to acquire locks is very important, not only for the possibility to deadlock due at lock order reversals, but even because lock acquisition should follow specific rules linked to locks natures. If you give a look at the table above, the practical rule is that if a thread holds a lock of level n (where the level is the number listed close to the kind of lock) it is not allowed to acquire a lock of superior levels, since this would break the specified semantic for a path. For example, if a thread holds a block lock (level 2), it is allowed to acquire a spin lock (level 1) but not a sleep lock (level 3), since block locks are intended to protect smaller paths than sleep lock (these rules are not about atomic operations or scheduling barriers, however). This is a list of lock with their respective behaviours: spin mutex - spinning - &man.mutex.9; sleep mutex - blocking - &man.mutex.9; pool mutex - blocking - &man.mtx.pool.9; sleep family - sleeping - &man.sleep.9; pause tsleep msleep msleep spin msleep rw msleep sx condvar - sleeping - &man.condvar.9; rwlock - blocking - &man.rwlock.9; sxlock - sleeping - &man.sx.9; lockmgr - sleeping - &man.lockmgr.9; semaphores - sleeping - &man.sema.9; Among these locks only mutexes, sxlocks, rwlocks and lockmgrs are intended to handle recursion, but currently recursion is only supported by mutexes and lockmgrs. Scheduling barriers Scheduling barriers are intended to be used in order to drive scheduling of threading. They consist mainly of three different stubs: critical sections (and preemption) sched_bind sched_pin Generally, these should be used only in a particular context and even if they can often replace locks, they should be avoided because they do not let the diagnose of simple eventual problems with locking debugging tools (as &man.witness.4;). Critical sections The &os; kernel has been made preemptive basically to deal with interrupt threads. In fact, in order to avoid high interrupt latency, time-sharing priority threads can be preempted by interrupt threads (in this way, they do not need to wait to be scheduled as the normal path previews). Preemption, however, introduces new racing points that need to be handled, as well. Often, in order to deal with preemption, the simplest thing to do is to completely disable it. A critical section defines a piece of code (borderlined by the pair of functions &man.critical.enter.9; and &man.critical.exit.9;, where preemption is guaranteed to not happen (until the protected code is fully executed). This can often replace a lock effectively but should be used carefully in order to not lose the whole advantage that preemption brings. sched_pin/sched_unpin Another way to deal with preemption is the sched_pin() interface. If a piece of code is closed in the sched_pin() and sched_unpin() pair of functions it is guaranteed that the respective thread, even if it can be preempted, it will always be executed on the same CPU. Pinning is very effective in the particular case when we have to access at per-cpu datas and we assume other threads will not change those data. The latter condition will determine a critical section as a too strong condition for our code. sched_bind/sched_unbind sched_bind is an API used in order to bind a thread to a particular CPU for all the time it executes the code, until a sched_unbind function call does not unbind it. This feature has a key role in situations where you cannot trust the current state of CPUs (for example, at very early stages of boot), as you want to avoid your thread to migrate on inactive CPUs. Since sched_bind and sched_unbind manipulate internal scheduler structures, they need to be enclosed in sched_lock acquisition/releasing when used. Proc structure Various emulation layers sometimes require some additional per-process data. It can manage separate structures (a list, a tree etc.) containing these data for every process but this tends to be slow and memory consuming. To solve this problem the &os; proc structure contains p_emuldata, which is a void pointer to some emulation layer specific data. This proc entry is protected by the proc mutex. The &os; proc structure contains a p_sysent entry that identifies, which ABI this process is running. In fact, it is a pointer to the sysentvec described above. So by comparing this pointer to the address where the sysentvec structure for the given ABI is stored we can effectively determine whether the process belongs to our emulation layer. The code typically looks like: if (__predict_true(p->p_sysent != &elf_&linux;_sysvec)) return; As you can see, we effectively use the __predict_true modifier to collapse the most common case (&os; process) to a simple return operation thus preserving high performance. This code should be turned into a macro because currently it is not very flexible, i.e. we do not support &linux;64 emulation nor A.OUT &linux; processes on i386. VFS The &os; VFS subsystem is very complex but the &linux; emulation layer uses just a small subset via a well defined API. It can either operate on vnodes or file handlers. Vnode represents a virtual vnode, i.e. representation of a node in VFS. Another representation is a file handler, which represents an opened file from the perspective of a process. A file handler can represent a socket or an ordinary file. A file handler contains a pointer to its vnode. More then one file handler can point to the same vnode. namei The &man.namei.9; routine is a central entry point to pathname lookup and translation. It traverses the path point by point from the starting point to the end point using lookup function, which is internal to VFS. The &man.namei.9; syscall can cope with symlinks, absolute and relative paths. When a path is looked up using &man.namei.9; it is inputed to the name cache. This behaviour can - be supressed. This routine is used all over the kernel and its + be suppressed. This routine is used all over the kernel and its performance is very critical. vn_fullpath The &man.vn.fullpath.9; function takes the best effort to traverse VFS name cache and returns a path for a given (locked) vnode. This process is unreliable but works just fine for the most common cases. The unreliability is because it relies on VFS cache (it does not traverse the on medium structures), it does not work with hardlinks, etc. This routine is used in several places in the Linuxulator. Vnode operations fgetvp - given a thread and a file descripton number it returns the associated vnode &man.vn.lock.9; - locks a vnode vn_unlock - unlocks a vnode &man.VOP.READDIR.9; - reads a directory referenced by a vnode &man.VOP.GETATTR.9; - gets attributes of a file or a directory referenced by a vnode &man.VOP.LOOKUP.9; - looks up a path to a given directory &man.VOP.OPEN.9; - opens a file referenced by a vnode &man.VOP.CLOSE.9; - closes a file referenced by a vnode &man.vput.9; - decrements the use count for a vnode and unlocks it &man.vrele.9; - decrements the use count for a vnode &man.vref.9; - increments the use count for a vnode File handler operations fget - given a thread and a file descriptor number it returns associated file handler and references it fdrop - drops a reference to a file handler fhold - references a file handler &linux; emulation layer -MD part This section deals with implementation of &linux; emulation layer in &os; operating system. It first describes the machine dependent part talking about how and where interaction between userland and kernel is implemented. It talks about syscalls, signals, ptrace, traps, stack fixup. This part discusses i386 but it is written generally so other architectures should not differ very much. The next part is the machine independent part of the Linuxulator. This section only covers i386 and ELF handling. A.OUT is obsolete and untested. Syscall handling Syscall handling is mostly written in linux_sysvec.c, which covers most of the routines pointed out in the sysentvec structure. When a &linux; process running on &os; issues a syscall, the general syscall routine calls linux prepsyscall routine for the &linux; ABI. &linux; prepsyscall &linux; passes arguments to syscalls via registers (that is why it is limited to 6 parameters on i386) while &os; uses the stack. The &linux; prepsyscall routine must copy parameters from registers to the stack. The order of the registers is: %ebx, %ecx, %edx, %esi, %edi, %ebp. The catch is that this is true for only most of the syscalls. Some (most notably clone) uses a different order but it is luckily easy to fix by inserting a dummy parameter in the linux_clone prototype. Syscall writing Every syscall implemented in the Linuxulator must have its prototype with various flags in syscalls.master. The form of the file is: ... AUE_FORK STD { int linux_fork(void); } ... AUE_CLOSE NOPROTO { int close(int fd); } ... The first column represents the syscall number. The second column is for auditing support. The third column represents the syscall type. It is either STD, OBSOL, NOPROTO and UNIMPL. STD is a standard syscall with full prototype and implementation. OBSOL is obsolete and defines just the prototype. NOPROTO means that the syscall is implemented elsewhere so do not prepend ABI prefix, etc. UNIMPL means that the syscall will be substituted with the nosys syscall (a syscall just printing out a message about the syscall not being implemented and returning ENOSYS). From syscalls.master a script generates three files: linux_syscall.h, linux_proto.h and linux_sysent.c. The linux_syscall.h contains definitions of syscall names and their numerical value, e.g.: ... #define LINUX_SYS_linux_fork 2 ... #define LINUX_SYS_close 6 ... The linux_proto.h contains structure definitions of arguments to every syscall, e.g.: struct linux_fork_args { register_t dummy; }; And finally, linux_sysent.c contains structure describing the system entry table, used to actually dispatch a syscall, e.g.: { 0, (sy_call_t *)linux_fork, AUE_FORK, NULL, 0, 0 }, /* 2 = linux_fork */ { AS(close_args), (sy_call_t *)close, AUE_CLOSE, NULL, 0, 0 }, /* 6 = close */ As you can see linux_fork is implemented in Linuxulator itself so the definition is of STD type and has no argument, which is exhibited by the dummy argument structure. On the other hand close is just an alias for real &os; &man.close.2; so it has no linux arguments structure associated and in the system entry table it is not prefixed with linux as it calls the real &man.close.2; in the kernel. Dummy syscalls The &linux; emulation layer is not complete, as some syscalls are not implemented properly and some are not implemented at all. The emulation layer employs a facility to mark unimplemented syscalls with the DUMMY macro. These dummy definitions reside in linux_dummy.c in a form of DUMMY(syscall);, which is then translated to various syscall auxiliary files and the implementation consists of printing a message saying that this syscall is not implemented. The UNIMPL prototype is not used because we want to be able to identify the name of the syscall that was called in order to know what syscalls are more important to implement. Signal handling Signal handling is done generally in the &os; kernel for all binary compatibilities with a call to a compat-dependent layer. &linux; compatibility layer defines linux_sendsig routine for this purpose. &linux; sendsig This routine first checks whether the signal has been installed with a SA_SIGINFO in which case it calls linux_rt_sendsig routine instead. Furthermore, it allocates (or reuses an already existing) signal handle context, then it builds a list of arguments for the signal handler. It translates the signal number based on the signal translation table, assigns a handler, translates sigset. Then it saves context for the sigreturn routine (various registers, translated trap number and signal mask). Finally, it copies out the signal context to the userspace and prepares context for the actual signal handler to run. linux_rt_sendsig This routine is similar to linux_sendsig just the signal context preparation is different. It adds siginfo, ucontext, and some &posix; parts. It might be worth considering whether those two functions could not be merged with a benefit of less code duplication and possibly even faster execution. linux_sigreturn This syscall is used for return from the signal handler. It does some security checks and restores the original process context. It also unmasks the signal in process signal mask. Ptrace Many &unix; derivates implement the &man.ptrace.2; syscall in order to allow various tracking and debugging features. This facility enables the tracing process to obtain various information about the traced process, like register dumps, any memory from the process address space, etc. and also to trace the process like in stepping an instruction or between system entries (syscalls and traps). &man.ptrace.2; also lets you set various information in the traced process (registers etc.). &man.ptrace.2; is a &unix;-wide standard implemented in most &unix;es around the world. &linux; emulation in &os; implements the &man.ptrace.2; facility in linux_ptrace.c. The routines for converting registers between &linux; and &os; and the actual &man.ptrace.2; syscall emulation syscall. The syscall is a long switch block that implements its counterpart in &os; for every &man.ptrace.2; command. The &man.ptrace.2; commands are mostly equal between &linux; and &os; so usually just a small modification is needed. For example, PT_GETREGS in &linux; operates on direct data while &os; uses a pointer to the data so after performing a (native) &man.ptrace.2; syscall, a copyout must be done to preserve &linux; semantics. The &man.ptrace.2; implementation in Linuxulator has some known weaknesses. There have been panics seen when using strace (which is a &man.ptrace.2; consumer) in the Linuxulator environment. Also PT_SYSCALL is not implemented. Traps Whenever a &linux; process running in the emulation layer traps the trap itself is handled transparently with the only exception of the trap translation. &linux; and &os; differs in opinion on what a trap is so this is dealt with here. The code is actually very short: static int translate_traps(int signal, int trap_code) { if (signal != SIGBUS) return signal; switch (trap_code) { case T_PROTFLT: case T_TSSFLT: case T_DOUBLEFLT: case T_PAGEFLT: return SIGSEGV; default: return signal; } } Stack fixup The RTLD run-time link-editor expects so called AUX tags on stack during an execve so a fixup must be done to ensure this. Of course, every RTLD system is different so the emulation layer must provide its own stack fixup routine to do this. So does Linuxulator. The elf_linux_fixup simply copies out AUX tags to the stack and adjusts the stack of the user space process to point right after those tags. So RTLD works in a smart way. A.OUT support The &linux; emulation layer on i386 also supports &linux; A.OUT binaries. Pretty much everything described in the previous sections must be implemented for A.OUT support (beside traps translation and signals sending). The support for A.OUT binaries is no longer maintained, especially the 2.6 emulation does not work with it but this does not cause any problem, as the linux-base in ports probably do not support A.OUT binaries at all. This support will probably be removed in future. Most of the stuff necessary for loading &linux; A.OUT binaries is in imgact_linux.c file. &linux; emulation layer -MI part This section talks about machine independent part of the Linuxulator. It covers the emulation infrastructure needed for &linux; 2.6 emulation, the thread local storage (TLS) implementation (on i386) and futexes. Then we talk briefly about some syscalls. Description of NPTL One of the major areas of progress in development of &linux; 2.6 was threading. Prior to 2.6, the &linux; threading support was implemented in the linuxthreads library. The library was a partial implementation of &posix; threading. The threading was implemented using separate processes for each thread using the clone syscall to let them share the address space (and other things). The main weaknesses of this approach was that every thread had a different PID, signal handling was broken (from the pthreads perspective), etc. Also the performance was not very good (use of SIGUSR signals for threads synchronization, kernel resource consumption, etc.) so to overcome these problems a new threading system was developed and named NPTL. The NPTL library focused on two things but a third thing came along so it is usually considered a part of NPTL. Those two things were embedding of threads into a process structure and futexes. The additional third thing was TLS, which is not directly required by NPTL but the whole NPTL userland library depends on it. Those improvements yielded in much improved performance and standards conformance. NPTL is a standard threading library in &linux; systems these days. The &os; Linuxulator implementation approaches the NPTL in three main areas. The TLS, futexes and PID mangling, which is meant to simulate the &linux; threads. Further sections describe each of these areas. &linux; 2.6 emulation infrastructure These sections deal with the way &linux; threads are managed and how we simulate that in &os;. Runtime determining of 2.6 emulation The &linux; emulation layer in &os; supports runtime setting of the emulated version. This is done via &man.sysctl.8;, namely compat.linux.osrelease, which is set to 2.4.2 by default (as of April 2007) and with all &linux; versions up to 2.6 it just determined what &man.uname.1 outputs. It is different with 2.6 emulation where setting this &man.sysctl.8; affects runtime behaviour of the emulation layer. When set to 2.6.x it sets the value of linux_use_linux26 while setting to something else keeps it unset. This variable (plus per-prison variables of the very same kind) determines whether 2.6 infrastructure (mainly PID mangling) is used in the code or not. The version setting is done system-wide and this affects all &linux; processes. The &man.sysctl.8; should not be changed when running any &linux; binary as it might harm things. &linux; processes and thread identifiers The semantics of &linux; threading are a little confusing and uses entirely different nomenclature to &os;. A process in &linux; consists of a struct task embedding two identifier fields - PID and TGID. PID is not a process ID but it is a thread ID. The TGID identifies a thread group in other words a process. For single-threaded process the PID equals the TGID. The thread in NPTL is just an ordinary process that happens to have TGID not equal to PID and have a group leader not equal to itself (and shared VM etc. of course). Everything else happens in the same way as to an ordinary process. There is no separation of a shared status to some external structure like in &os;. This creates some duplication of information and possible data inconsistency. The &linux; kernel seems to use task -> group information in some places and task information elsewhere and it is really not very consistent and looks error-prone. Every NPTL thread is created by a call to the clone syscall with a specific set of flags (more in the next subsection). The NPTL implements strict 1:1 threading. In &os; we emulate NPTL threads with ordinary &os; processes that share VM space, etc. and the PID gymnastic is just mimiced in the emulation specific structure attached to the process. The structure attached to the process looks like: struct linux_emuldata { pid_t pid; int *child_set_tid; /* in clone(): Child.s TID to set on clone */ int *child_clear_tid;/* in clone(): Child.s TID to clear on exit */ struct linux_emuldata_shared *shared; int pdeath_signal; /* parent death signal */ LIST_ENTRY(linux_emuldata) threads; /* list of linux threads */ }; The PID is used to identify the &os; process that attaches this structure. The child_se_tid and child_clear_tid are used for TID address copyout when a process exits and is created. The shared pointer points to a structure shared among threads. The pdeath_signal variable identifies the parent death signal and the threads pointer is used to link this structure to the list of threads. The linux_emuldata_shared structure looks like: struct linux_emuldata_shared { int refs; pid_t group_pid; LIST_HEAD(, linux_emuldata) threads; /* head of list of linux threads */ }; The refs is a reference counter being used to determine when we can free the structure to avoid memory leaks. The group_pid is to identify PID ( = TGID) of the whole process ( = thread group). The threads pointer is the head of the list of threads in the process. The linux_emuldata structure can be obtained from the process using em_find. The prototype of the function is: struct linux_emuldata *em_find(struct proc *, int locked); Here, proc is the process we want the emuldata structure from and the locked parameter determines whether we want to lock or not. The accepted values are EMUL_DOLOCK and EMUL_DOUNLOCK. More about locking later. PID mangling Because of the described different view knowing what a process ID and thread ID is between &os; and &linux; we have to translate the view somehow. We do it by PID mangling. This means that we fake what a PID (=TGID) and TID (=PID) is between kernel and userland. The rule of thumb is that in kernel (in Linuxulator) PID = PID and TGID = shared -> group pid and to userland we present PID = shared -> group_pid and TID = proc -> p_pid. The PID member of linux_emuldata structure is a &os; PID. The above affects mainly getpid, getppid, gettid syscalls. Where we use PID/TGID respectively. In copyout of TIDs in child_clear_tid and child_set_tid we copy out &os; PID. Clone syscall The clone syscall is the way threads are created in &linux;. The syscall prototype looks like this: int linux_clone(l_int flags, void *stack, void *parent_tidptr, int dummy, void * child_tidptr); The flags parameter tells the syscall how exactly the processes should be cloned. As described above, &linux; can create processes sharing various things independently, for example two processes can share file descriptors but not VM, etc. Last byte of the flags parameter is the exit signal of the newly created process. The stack parameter if non-NULL tells, where the thread stack is and if it is NULL we are supposed to copy-on-write the calling process stack (i.e. do what normal &man.fork.2; routine does). The parent_tidptr parameter is used as an address for copying out process PID (i.e. thread id) once the process is sufficiently instantiated but is not runnable yet. The dummy parameter is here because of the very strange calling convention of this syscall on i386. It uses the registers directly and does not let the compiler do it what results in the need of a dummy syscall. The child_tidptr parameter is used as an address for copying out PID once the process has finished forking and when the process exits. The syscall itself proceeds by setting corresponding flags depending on the flags passed in. For example, CLONE_VM maps to RFMEM (sharing of VM), etc. The only nit here is CLONE_FS and CLONE_FILES because &os; does not allow setting this separately so we fake it by not setting RFFDG (copying of fd table and other fs information) if either of these is defined. This does not cause any problems, because those flags are always set together. After setting the flags the process is forked using the internal fork1 routine, the process is instrumented not to be put on a run queue, i.e. not to be set runnable. After the forking is done we possibly reparent the newly created process to emulate CLONE_PARENT semantics. Next part is creating the emulation data. Threads in &linux; does not signal their parents so we set exit signal to be 0 to disable this. After that setting of child_set_tid and child_clear_tid is performed enabling the functionality later in the code. At this point we copy out the PID to the address specified by parent_tidptr. The setting of process stack is done by simply rewriting thread frame %esp register (%rsp on amd64). Next part is setting up TLS for the newly created process. After this &man.vfork.2; semantics might be emulated and finally the newly created process is put on a run queue and copying out its PID to the parent process via clone return value is done. The clone syscall is able and in fact is used for emulating classic &man.fork.2; and &man.vfork.2; syscalls. Newer glibc in a case of 2.6 kernel uses clone to implement &man.fork.2; and &man.vfork.2; syscalls. Locking The locking is implemented to be per-subsystem because we do not expect a lot of contention on these. There are two locks: emul_lock used to protect manipulating of linux_emuldata and emul_shared_lock used to manipulate linux_emuldata_shared. The emul_lock is a nonsleepable blocking mutex while emul_shared_lock is a sleepable blocking sx_lock. Because of the per-subsystem locking we can coalesce some locks and that is why the em find offers the non-locking access. TLS This section deals with TLS also known as thread local storage. Introduction to threading Threads in computer science are entities within a process that can be scheduled independently from each other. The threads in the process share process wide data (file descriptors, etc.) but also have their own stack for their own data. Sometimes there is a need for process-wide data specific to a given thread. Imagine a name of the thread in execution or something like that. The traditional &unix; threading API, pthreads provides a way to do it via &man.pthread.key.create.3;, &man.pthread.setspecific.3; and &man.pthread.getspecific.3; where a thread can create a key to the thread local data and using &man.pthread.getspecific.3; or &man.pthread.getspecific.3; to manipulate those data. You can easily see that this is not the most comfortable way this could be accomplished. So various producers of C/C++ compilers introduced a better way. They defined a new modifier keyword thread that specifies that a variable is thread specific. A new method of accessing such variables was developed as well (at least on i386). The pthreads method tends to be implemented in userspace as a trivial lookup table. The performance of such a solution is not very good. So the new method uses (on i386) segment registers to address a segment, where TLS area is stored so the actual accessing of a thread variable is just appending the segment register to the address thus addressing via it. The segment registers are usually %gs and %fs acting like segment selectors. Every thread has its own area where the thread local data are stored and the segment must be loaded on every context switch. This method is very fast and used almost exclusively in the whole i386 &unix; world. Both &os; and &linux; implement this approach and it yields very good results. The only drawback is the need to reload the segment on every context switch which can slowdown context switches. &os; tries to avoid this overhead by using only 1 segment descriptor for this while &linux; uses 3. Interesting thing is that almost nothing uses more than 1 descriptor (only Wine seems to use 2) so &linux; pays this unnecessary price for context switches. Segments on i386 The i386 architecture implements the so called segments. A segment is a description of an area of memory. The base address (bottom) of the memory area, the end of it (ceiling), type, protection, etc. The memory described by a segment can be accessed using segment selector registers (%cs, %ds, %ss, %es, %fs, %gs). For example let us suppose we have a segment which base address is 0x1234 and length and this code: mov %edx,%gs:0x10 This will load the content of the %edx register into memory location 0x1244. Some segment registers have a special use, for example %cs is used for code segment and %ss is used for stack segment but %fs and %gs are generally unused. Segments are either stored in a global GDT table or in a local LDT table. LDT is accessed via an entry in the GDT. The LDT can store more types of segments. LDT can be per process. - Both tables define upto 8191 entries. + Both tables define up to 8191 entries. Implementation on &linux; i386 There are two main ways of setting up TLS in &linux;. It can be set when cloning a process using the clone syscall or it can call set_thread_area. When a process passes CLONE_SETTLS flag to clone, the kernel expects the memory pointed to by the %esi register a &linux; user space representation of a segment, which gets translated to the machine representation of a segment and loaded into a GDT slot. The GDT slot can be specified with a number or -1 can be used meaning that the system itself should choose the first free slot. In practice, the vast majority of programs use only one TLS entry and does not care about the number of the entry. We exploit this in the emulation and in fact depend on it. Emulation of &linux; TLS i386 Loading of TLS for the current thread happens by calling set_thread_area while loading TLS for a second process in clone is done in the separate block in clone. Those two functions are very similar. The only difference being the actual loading of the GDT segment, which happens on the next context switch for the newly created process while set_thread_area must load this directly. The code basically does this. It copies the &linux; form segment descriptor from the userland. The code checks for the number of the descriptor but because this differs between &os; and &linux; we fake it a little. We only support indexes of 6, 3 and -1. The 6 is genuine &linux; number, 3 is genuine &os; one and -1 means autoselection. Then we set the descriptor number to constant 3 and copy out this to the userspace. We rely on the userspace process using the number from the descriptor but this works most of the time (have never seen a case where this did not work) as the userspace process typically passes in 1. Then we convert the descriptor from the &linux; form to a machine dependant form (i.e. operating system independent form) and copy this to the &os; defined segment descriptor. Finally we can load it. We assign the descriptor to threads PCB (process control block) and load the %gs segment using load_gs. This loading must be done in a critical section so that nothing can interrupt us. The CLONE_SETTLS case works exactly like this just the loading using load_gs is not performed. The segment used for this (segment number 3) is shared for this use between &os; processes and &linux; processes so the &linux; emulation layer does not add any overhead over plain &os;. amd64 The amd64 implementation is similar to the i386 one but there was initially no 32bit segment descriptor used for this purpose (hence not even native 32bit TLS users worked) so we had to add such a segment and implement its loading on every context switch (when a flag signaling use of 32bit is set). Apart from this the TLS loading is exactly the same just the segment numbers are different and the descriptor format and the loading differs slightly. Futexes Introduction to synchronization Threads need some kind of synchronization and &posix; provides some of them: mutexes for mutual exclusion, read-write locks for mutual exclusion with biased ratio of reads and writes and condition variables for signaling a status change. It is interesting to note that &posix; threading API lacks support for semaphores. Those synchronization routines implementations are heavily dependant on the type threading support we have. In pure 1:M (userspace) model the implementation can be solely done in userspace and thus be very fast (the condition variables will probably end up being implemented using signals, i.e. not fast) and simple. In 1:1 model, the situation is also quite clear - the threads must be synchronized - using kernel facilites (which is very slow because a syscall must be + using kernel facilities (which is very slow because a syscall must be performed). The mixed M:N scenario just combines the first and second approach or rely solely on kernel. Threads synchronization is a vital part of thread-enabled programming and its performance can affect resulting program a lot. Recent benchmarks on &os; operating system showed that an improved sx_lock implementation yielded 40% speedup in ZFS (a heavy sx user), this is in-kernel stuff but it shows clearly how important the performance of synchronization primitives is. Threaded programs should be written with as little contention on locks as possible. Otherwise, instead of doing useful work the thread just waits on a lock. Because of this, the most well written threaded programs show little locks contention. Futexes introduction &linux; implements 1:1 threading, i.e. it has to use in-kernel synchronization primitives. As stated earlier, well written threaded programs have little lock contention. So a typical sequence could be performed as two atomic increase/decrease mutex reference counter, which is very fast, as presented by the following example: pthread_mutex_lock(&mutex); .... pthread_mutex_unlock(&mutex); 1:1 threading forces us to perform two syscalls for those mutex calls, which is very slow. The solution &linux; 2.6 implements is called futexes. Futexes implement the check for contention in userspace and call kernel primitives only in a case of contention. Thus the typical case takes place without any kernel intervention. This yields reasonably fast and flexible synchronization primitives implementation. Futex API The futex syscall looks like this: int futex(void *uaddr, int op, int val, struct timespec *timeout, void *uaddr2, int val3); In this example uaddr is an address of the mutex in userspace, op is an operation we are about to perform and the other parameters have per-operation meaning. Futexes implement the following operations: FUTEX_WAIT FUTEX_WAKE FUTEX_FD FUTEX_REQUEUE FUTEX_CMP_REQUEUE FUTEX_WAKE_OP FUTEX_WAIT This operation verifies that on address uaddr the value val is written. If not, EWOULDBLOCK is returned, otherwise the thread is queued on the futex and gets suspended. If the argument timeout is non-zero it specifies the maximum time for the sleeping, otherwise the sleeping is infinite. FUTEX_WAKE This operation takes a futex at uaddr and wakes up val first futexes queued on this futex. FUTEX_FD This operations associates a file descriptor with a given futex. FUTEX_REQUEUE This operation takes val threads queued on futex at uaddr, wakes them up, and takes val2 next threads and requeues them on futex at uaddr2. FUTEX_CMP_REQUEUE This operation does the same as FUTEX_REQUEUE but it checks that val3 equals to val first. FUTEX_WAKE_OP This operation performs an atomic operation on val3 (which contains coded some other value) and uaddr. Then it wakes up val threads on futex at uaddr and if the atomic operation returned a positive number it wakes up val2 threads on futex at uaddr2. The operations implemented in FUTEX_WAKE_OP: FUTEX_OP_SET FUTEX_OP_ADD FUTEX_OP_OR FUTEX_OP_AND FUTEX_OP_XOR There is no val2 parameter in the futex prototype. The val2 is taken from the struct timespec *timeout parameter for operations FUTEX_REQUEUE, FUTEX_CMP_REQUEUE and FUTEX_WAKE_OP. Futex emulation in &os; The futex emulation in &os; is taken from NetBSD and further extended by us. It is placed in linux_futex.c and linux_futex.h files. The futex structure looks like: struct futex { void *f_uaddr; int f_refcount; LIST_ENTRY(futex) f_list; TAILQ_HEAD(lf_waiting_paroc, waiting_proc) f_waiting_proc; }; And the structure waiting_proc is: struct waiting_proc { struct thread *wp_t; struct futex *wp_new_futex; TAILQ_ENTRY(waiting_proc) wp_list; }; futex_get / futex_put A futex is obtained using the futex_get function, which searches a linear list of futexes and returns the found one or creates a new futex. When releasing a futex from the use we call the futex_put function, which decreases a reference counter of the futex and if the refcount reaches zero it is released. futex_sleep When a futex queues a thread for sleeping it creates a working_proc structure and puts this structure to the list inside the futex structure then it just performs a &man.tsleep.9; to suspend the thread. The sleep can be timed out. After &man.tsleep.9; returns (the thread was woken up or it timed out) the working_proc structure is removed from the list and is destroyed. All this is done in the futex_sleep function. If we got woken up from futex_wake we have wp_new_futex set so we sleep on it. This way the actual requeueing is done in this function. futex_wake Waking up a thread sleeping on a futex is performed in the futex_wake function. First in this function we mimic the strange &linux; behaviour, where it wakes up N threads for all operations, the only exception is that the REQUEUE operations are performed on N+1 threads. But this usually does not make any difference as we are waking up all threads. Next in the function in the loop we wake up n threads, after this we check if there is a new futex for requeueing. If so, we requeue up to n2 threads on the new futex. This cooperates with futex_sleep. futex_wake_op The FUTEX_WAKE_OP operation is quite complicated. First we obtain two futexes at addresses uaddr and uaddr2 then we perform the atomic operation using val3 and uaddr2. Then val waiters on the first futex is woken up and if the atomic operation condition holds we wake up val2 (i.e. timeout) waiter on the second futex. futex atomic operation The atomic operation takes two parameters encoded_op and uaddr. The encoded operation encodes the operation itself, comparing value, operation argument, and comparing argument. The pseudocode for the operation is like this one: oldval = *uaddr2 *uaddr2 = oldval OP oparg And this is done atomically. First a copying in of the number at uaddr is performed and the operation is done. The code handles page faults and if no page fault occurs oldval is compared to cmparg argument with cmp comparator. Futex locking Futex implementation uses two lock lists protecting sx_lock and global locks (either Giant or another sx_lock). Every operation is performed locked from the start to the very end. Various syscalls implementation In this section I am going to describe some smaller syscalls that are worth mentioning because their implementation is not obvious or those syscalls are interesting from other point of view. *at family of syscalls During development of &linux; 2.6.16 kernel, the *at syscalls were added. Those syscalls (openat for example) work exactly like their at-less counterparts with the slight exception of the dirfd parameter. This parameter changes where the given file, on which the syscall is to be performed, is. When the filename parameter is absolute dirfd is ignored but when the path to the file is relative, it comes to the play. The - dirfd paramtere is a directory relative to which + dirfd parameter is a directory relative to which the relative pathname is checked. The dirfd parameter is a file descriptor of some directory or AT_FDCWD. So for example the openat syscall can be like this: file descriptor 123 = /tmp/foo/, current working directory = /tmp/ openat(123, /tmp/bah\, flags, mode) /* opens /tmp/bah */ openat(123, bah\, flags, mode) /* opens /tmp/foo/bah */ openat(AT_FDWCWD, bah\, flags, mode) /* opens /tmp/bah */ openat(stdio, bah\, flags, mode) /* returns error because stdio is not a directory */ This infrastructure is necessary to avoid races when opening files outside the working directory. Imagine that a process consists of two threads, thread A and thread B. Thread A issues open(./tmp/foo/bah., flags, mode) and before returning it gets preempted and thread B runs. Thread B does not care about the needs of thread A and renames or removes /tmp/foo/. We got a race. To avoid this we can open /tmp/foo and use it as dirfd for openat syscall. This also enables user to implement per-thread working directories. &linux; family of *at syscalls contains: linux_openat, linux_mkdirat, linux_mknodat, linux_fchownat, linux_futimesat, linux_fstatat64, linux_unlinkat, linux_renameat, linux_linkat, linux_symlinkat, linux_readlinkat, linux_fchmodat and linux_faccessat. All these are implemented using the modified &man.namei.9; routine and simple wrapping layer. Implementation The implementation is done by altering the &man.namei.9; routine (described above) to take additional parameter dirfd in its nameidata structure, which specifies the starting point of the pathname lookup instead of using the current working directory every time. The resolution of dirfd from file descriptor number to a vnode is done in native *at syscalls. When dirfd is AT_FDCWD the dvp entry in nameidata structure is NULL but when dirfd is a different number we obtain a file for this file descriptor, check whether this file is valid and if there is vnode attached to it then we get a vnode. Then we check this vnode for being a directory. In the actual &man.namei.9; routine we simply substitute the dvp vnode for dp variable in the &man.namei.9; function, which determines the starting point. The &man.namei.9; is not used directly but via a trace of different functions on various levels. For example the openat goes like this: openat() --> kern_openat() --> vn_open() -> namei() For this reason kern_open and vn_open must be altered to incorporate the additional dirfd parameter. No compat layer is created for those because there are not many users of this and the users can be easily converted. This general implementation enables &os; to implement their own *at syscalls. This is being discussed right now. Ioctl The ioctl interface is quite fragile due to its generality. We have to bear in mind that devices differ between &linux; and &os; so some care must be applied to do ioctl emulation work right. The ioctl handling is implemented in linux_ioctl.c, where linux_ioctl function is defined. This function simply iterates over sets of ioctl handlers to find a handler that implements a given command. The ioctl syscall has three parameters, the file descriptor, command and an argument. The command is a 16-bit number, which in theory is divided into high 8 bits determining class of the ioctl command and low 8 bits, which are the actual command within the given set. The emulation takes advantage of this division. We implement handlers for each set, like sound_handler or disk_handler. Each handler has a maximum command and a minimum command defined, which is used for determining what handler is used. There are slight problems with this approach because &linux; does not use the set division consistently so sometimes ioctls for a different set are inside a set they should not belong to (SCSI generic ioctls inside cdrom set, etc.). &os; currently does not implement many &linux; ioctls (compared to NetBSD, for example) but the plan is to port those from NetBSD. The trend is to use &linux; ioctls even in the native &os; drivers because of the easy porting of applications. Debugging Every syscall should be debuggable. For this purpose we introduce a small infrastructure. We have the ldebug facility, which tells whether a given syscall should be debugged (settable via a sysctl). For printing we have LMSG and ARGS macros. Those are used - for altering a printable string for uniform debuging messages. + for altering a printable string for uniform debugging messages. Conclusion Results As of April 2007 the &linux; emulation layer is capable of emulating the &linux; 2.6.16 kernel quite well. The remaining problems concern futexes, unfinished *at family of syscalls, problematic signals delivery, missing epoll and inotify and probably some bugs we have not discovered yet. Despite this we are capable of running basically all the &linux; programs included in &os; Ports Collection with Fedora Core 4 at 2.6.16 and there are some rudimentary reports of success with Fedora Core 6 at 2.6.16. The - Fedora Core 6 linux_base was recently commited enabling + Fedora Core 6 linux_base was recently committed enabling some further testing of the emulation layer and giving us some more hints where we should put our effort in implementing missing stuff. We are able to run the most used applications like www/linux-firefox, www/linux-opera, net-im/skype and some games from the Ports Collection. Some of the programs exhibit bad behaviour under 2.6 emulation but this is currently under investigation and hopefully will be fixed soon. The only big application that is known not to work is the &linux; &java; Development Kit and this is because of the requirement of epoll facility which is not directly related to the &linux; kernel 2.6. We hope to enable 2.6.16 emulation by default some time after &os; 7.0 is released at least to expose the 2.6 emulation parts for some wider testing. Once this is done we can switch to Fedora Core 6 linux_base, which is the ultimate plan. Future work Future work should focus on fixing the remaining issues with futexes, implement the rest of the *at family of syscalls, fix the signal delivery and possibly implement the epoll and inotify facilities. We hope to be able to run the most important programs flawlessly soon, so we will be able to switch to the 2.6 emulation by default and make the Fedora Core 6 the default linux_base because our currently used Fedora Core 4 is not supported any more. The other possible goal is to share our code with NetBSD and DragonflyBSD. NetBSD has some support for 2.6 emulation but its far from finished and not really tested. DragonflyBSD has expressed some interest in porting the 2.6 improvements. Generally, as &linux; develops we would like to keep up with their development, implementing newly added syscalls. Splice comes to mind first. Some already implemented syscalls are also heavily crippled, for example mremap and others. Some performance improvements can also be made, finer grained locking and others. Team I cooperated on this project with (in alphabetical order): &a.jhb; &a.kib; Emmanuel Dreyfus Scot Hetzel &a.jkim; &a.netchild; &a.ssouhlal; Li Xiao &a.davidxu; I would like to thank all those people for their advices, code reviews and general support. Literatures Marshall Kirk McKusick - George V. Nevile-Neil. Design and Implementation of the &os; operating system. Addison-Wesley, 2005.
diff --git a/en_US.ISO8859-1/articles/linux-users/article.sgml b/en_US.ISO8859-1/articles/linux-users/article.sgml index e565fe413e..416e87cef7 100644 --- a/en_US.ISO8859-1/articles/linux-users/article.sgml +++ b/en_US.ISO8859-1/articles/linux-users/article.sgml @@ -1,590 +1,590 @@ %articles.ent; ]>
FreeBSD Quickstart Guide for &linux; Users John Ferrell 2008 The FreeBSD Documentation Project $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.linux; &tm-attrib.intel; &tm-attrib.redhat; &tm-attrib.unix; &tm-attrib.general; This document is intended quickly familiarize intermediate to advanced &linux; users with the basics of FreeBSD. Introduction This document will highlight the differences between &os; and &linux; so that intermediate to advanced &linux; users can quickly familiarize themselves with the basics of &os;. This is just a technical quickstart, it does not attempt to design philosophical differences between the two operating systems. This document assumes that you have already installed &os;. If you have not installed &os; or need help with the installation process please refer to the Installing FreeBSD chapter of the &os; Handbook. Shells: No Bash? Those coming from &linux; are often surprised to find that Bash is not the default shell in &os;. In fact, Bash is not even in the default installation. Instead, &os; uses &man.tcsh.1; as the default shell. Although, Bash and your other favorite shells are available in &os;'s Packages and Ports Collection. If you do install other shells you can use &man.chsh.1; to set a user's default shell. It is, however, recommended that the root's default shell remain unchanged. The reason for this is that shells not included in the base distribution are normally installed in /usr/local/bin or /usr/bin. In the event of a problem the file systems where /usr/local/bin and /usr/bin are located may not be mounted. In this case root would not have access to its default shell, preventing root from logging in. For this reason a second root account, the toor account, was created for use with non-default shells. See the security FAQ for information regarding the toor account. Packages and Ports: Adding software in &os; In addition to the traditional &unix; method of installing software (download source, extract, edit source code, and compile), &os; offers two other methods for installing applications: packages and ports. A complete list of of all available ports and packages can be found here. Packages Packages are pre-compiled applications, the &os; equivalents of .deb files on Debian/Ubuntu based systems and .rpm files on Red Hat/Fedora based systems. Packages are installed using &man.pkg.add.1;. For example, the following command installs Apache 2.2: &prompt.root; pkg_add /tmp/apache-2.2.6_2.tbz Using the switch will tell &man.pkg.add.1; to automatically fetch a package and install it, as well as any dependencies: &prompt.root; pkg_add -r apache22 Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-6.2-release/Latest/apache22.tbz... Done. Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-6.2-release/All/expat-2.0.0_1.tbz... Done. Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-6.2-release/All/perl-5.8.8_1.tbz... Done. [snip] To run apache www server from startup, add apache22_enable="YES" in your /etc/rc.conf. Extra options can be found in startup script. If you are running a release version of &os; (6.2, 6.3, 7.0, etc., generally installed from CD-ROM) pkg_add -r will download packages built for that specific release. These packages may not be the most up-to-date version of the application. You can use the PACKAGESITE variable to override this default behavior. For example, set PACKAGESITE to to download the most recent packages built for the 6.X series. You can read more about the &os; versions in the article Choosing the &os; Version That Is Right For You. For more information on packages please refer to section 4.4 of the &os; Handbook: Using the Packages System. Ports &os;'s second method for installing applications is the Ports Collection. The Ports Collection is a framework of Makefiles and patches specifically customized for installing various software applications from source on &os;. When installing a port the system will fetch the source code, apply any required patches, compile the code, and install the application (and do the same for any dependencies). The Ports Collection, sometimes referred to as the ports tree, can be found in /usr/ports. That is assuming the Ports Collection was installed during the &os; installation process. If the Ports Collection has not been installed it can be added from the installation discs using &man.sysinstall.8;, or pulled from the &os; servers using &man.csup.1; or &man.portsnap.8;. Detailed instructions for installing the Ports Collection can be found in section 4.5.1 of the handbook. Installing a port is as simple (generally) as changing in to the port's directory and starting the build process. The following example installs Apache 2.2 from the Ports Collection: &prompt.root; cd /usr/ports/www/apache22 &prompt.root; make install clean A major benefit of using ports to install software is the ability to customize the installation options. For example, when installing Apache 2.2 from ports you can enable mod_ldap by setting the WITH_LDAP &man.make.1; variable: &prompt.root; cd /usr/ports/www/apache22 &prompt.root; make WITH_LDAP="YES" install clean Please see section 4.5 of the &os; Handbook, Using the Ports Collection, for more information about the Ports Collection. Ports or packages, which one should I use? Packages are just pre-compiled ports, so it is really a matter of installing from source (ports) versus installing from binary packages. Each method has its own benefits: Packages (binary) Faster installation (compiling large applications can take quite a while). You do not need to understand how to compile software. No need to install compilers on your system. Ports (source) Ability to customize installation options. (Packages are normally built with standard options. With ports you can customize various options, such as building additional modules or changing the default path.) You can apply your own patches if you are so inclined. If you do not have any special requirements, packages will probably suit your situation just fine. If you may ever need to customize, ports are the way to go. (And remember, if you need to customize but prefer packages, you can build a custom package from ports using make package and then copy the package to other servers.) System Startup: Where are the run-levels? &linux; uses the SysV init system, whereas &os; uses the traditional BSD-style &man.init.8;. Under the BSD-style &man.init.8; there are no run-levels and no /etc/inittab, instead startup is controlled by the &man.rc.8; utility. The /etc/rc script reads /etc/defaults/rc.conf and /etc/rc.conf to determine which services are to be started. The specified services are then started by running the corresponding service initialization scripts located in /etc/rc.d/ and /usr/local/etc/rc.d/. These scripts are similar to the scripts located in /etc/init.d/ on &linux; systems. Why are there two locations for service initialization scripts? The scripts found in /etc/rc.d/ are for applications that are part of the base system. (&man.cron.8;, &man.sshd.8;, &man.syslog.3;, and others.) The scripts in /usr/local/etc/rc.d/ are for user-installed applications such as Apache, Squid, etc. What is the difference between the base system and user-installed applications? FreeBSD is developed as a complete operating system. In other words, the kernel, system libraries, and userland utilities (such as &man.ls.1;, &man.cat.1;, &man.cp.1;, etc.) are developed and released together as one. This is what is referred to as the base system. The user-installed applications are applications that are not part of the base system, such as Apache, X11, Mozilla Firefox, etc. These user-installed applications are generally installed using &os;'s Packages and Ports Collection. In order to keep them separate from the base system, user-installed applications are normally installed under /usr/local/. Therefore the user-installed binaries reside in /usr/local/bin/, configuration files are in /usr/local/etc/, and so on. Services are enabled by specifying ServiceName_enable="YES" in /etc/rc.conf (&man.rc.conf.5;). Take a look at /etc/defaults/rc.conf for the system defaults, these default settings are overridden by settings in /etc/rc.conf. Also, when installing additional applications be sure to review the documentation to determine how to enable any associated services. The following snippet from /etc/rc.conf enables &man.sshd.8; and Apache 2.2. It also specifies that Apache should be started with SSL. # enable SSHD sshd_enable="YES" # enable Apache with SSL apache22_enable="YES" apache22_flags="-DSSL" Once a service has been enabled in /etc/rc.conf, the service can be started from the command line (without rebooting the system): &prompt.root; /etc/rc.d/sshd start If a service has not been enabled it can be started from the command line using : &prompt.root; /etc/rc.d/sshd forcestart Network configuration Network Interfaces Instead of a generic ethX identifier that &linux; uses to identify a network interface, &os; uses the driver name followed by a number as the identifier. The following output from &man.ifconfig.8; shows two &intel Pro 1000 network interfaces (em0 and em1): &prompt.user; ifconfig em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 options=b<RXCSUM,TXCSUM,VLAN_MTU> inet 10.10.10.100 netmask 0xffffff00 broadcast 10.10.10.255 ether 00:50:56:a7:70:b2 media: Ethernet autoselect (1000baseTX <full-duplex>) status: active em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 options=b<RXCSUM,TXCSUM,VLAN_MTU> inet 192.168.10.222 netmask 0xffffff00 broadcast 192.168.10.255 ether 00:50:56:a7:03:2b media: Ethernet autoselect (1000baseTX <full-duplex>) status: active IP Configuration An IP address can be assigned to an interface using &man.ifconfig.8;. However, to remain persistent across reboots the IP configuration must be included in /etc/rc.conf. The following example specifies the hostname, IP address, and default gateway: hostname="server1.example.com" ifconfig_em0="inet 10.10.10.100 netmask 255.255.255.0" defaultrouter="10.10.10.1" Use the following to configure an interface for DHCP: hostname="server1.example.com" ifconfig_em0="DHCP" Firewall Like IPTABLES in &linux;, &os; also offers a kernel level firewall; actually &os; offers three firewalls: IPFIREWALL IPFILTER PF IPFIREWALL or IPFW (the command to manage an IPFW ruleset is &man.ipfw.8;) is the firewall developed and maintained by the &os; developers. IPFW can be paired with &man.dummynet.4; to provide traffic shaping capabilities and simulate different types of network connections. Sample IPFW rule to allow SSH in: ipfw add allow tcp from any to me 22 in via $ext_if IPFILTER is the firewall application developed by Darren Reed. It is not specific to &os;, and has been ported to several operating systems including NetBSD, OpenBSD, SunOS, HP/UX, and Solaris. Sample IPFILTER command to allow SSH in: pass in on $ext_if proto tcp from any to any port = 22 The last firewall application, PF, is developed by the OpenBSD project. PF was created as a replacement for IPFILTER. As such, the PF syntax is very similar to that of IPFILTER. PF can be paired with &man.altq.4; to provide QoS features. Sample PF command to allow SSH in: pass in on $ext_if inet proto tcp from any to ($ext_if) port 22 Updating &os; There are three methods for updating a &os; system: from source, binary updates, and the installation discs. Updating from source is the most involved update method, but offers the greatest amount of flexibility. The process involves synchronizing a local copy of the FreeBSD source code with the &os; CVS (Concurrent Versioning System) servers. Once the local source code is up to date you can build new versions of the kernel and userland. For more information on source updates see the chapter on updating in the &os; Handbook. Binary updates are similar to using yum or apt-get to update a &linux; system. The command &man.freebsd-update.8; will fetch new updates and install them. The updates can be scheduled using &man.cron.8;. If you do use &man.cron.8; to schedule the updates, please be sure to use freebsd-update cron in your &man.crontab.1; to reduce the possibility of a large number of machines all pulling updates at the same time. 0 3 * * * root /usr/sbin/freebsd-update cron The last update method, updating from the installation discs, is a straight-forward process. Boot from the installation discs and select the option to upgrade. procfs: Gone But Not Forgotten In &linux;, you may have looked at /proc/sys/net/ipv4/ip_forward to determine if IP forwarding was enabled. Under &os; you should use &man.sysctl.8; to view this and other system settings, as &man.procfs.5; has been deprecated in current versions of &os;. (Although sysctl is available in &linux; as well.) In the IP forwarding example, you would use the following to determine if IP forwarding is enabled on your FreeBSD system: &prompt.user; sysctl net.inet.ip.forwarding net.inet.ip.forwarding: 0 The flag is used to list all the system settings: &prompt.user; sysctl -a kern.ostype: FreeBSD kern.osrelease: 6.2-RELEASE-p9 kern.osrevision: 199506 kern.version: FreeBSD 6.2-RELEASE-p9 #0: Thu Nov 29 04:07:33 UTC 2007 root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC kern.maxvnodes: 17517 kern.maxproc: 1988 kern.maxfiles: 3976 kern.argmax: 262144 kern.securelevel: -1 kern.hostname: server1 kern.hostid: 0 kern.clockrate: { hz = 1000, tick = 1000, profhz = 666, stathz = 133 } kern.posix1version: 200112 ... Some of these sysctl values are read-only. There are occasions where procfs is required, such as running older software, using &man.truss.1; to trace system calls, and &linux; Binary Compatibility. (Although, &linux; Binary Compatibility uses its own procfs, &man.linprocfs.5;.) If you need to mount procfs you can add the following to /etc/fstab: proc /proc procfs rw,noauto 0 0 will prevent /proc from being automatically mounted at boot. And then mount procfs with: &prompt.root; mount /proc Common Commands Package Management &linux; command (Red Hat/Debian) &os; equivalent Purpose yum install package / apt-get install package pkg_add -r package Install package from remote repository rpm -ivh package / dpkg -i package pkg_add -v package Install package rpm -qa / dpkg -l pkg_info List installed packages System Management &linux; command &os; equivalent Purpose lspci pciconf List PCI devices lsmod kldstat List loaded kernel modules modprobe kldload / kldunload Load/Unload kernel modules strace truss Trace system calls Conclusion Hopefully this document has provided you with enough to get started with &os;. Be sure to take a look at the &os; Handbook - for more indepth coverage of the topics touched on as well as + for more in depth coverage of the topics touched on as well as the many topics not covered in this document.
diff --git a/en_US.ISO8859-1/articles/portbuild/article.sgml b/en_US.ISO8859-1/articles/portbuild/article.sgml index 0149573adc..84fe36567a 100644 --- a/en_US.ISO8859-1/articles/portbuild/article.sgml +++ b/en_US.ISO8859-1/articles/portbuild/article.sgml @@ -1,2233 +1,2233 @@ %articles.ent; ]>
Package Building Procedures The &os; Ports Management Team $FreeBSD$ 2003 2004 2005 2006 2007 2008 2009 2010 The &os; Ports Management Team &tm-attrib.freebsd; &tm-attrib.intel; &tm-attrib.sparc; &tm-attrib.general; Introduction and Conventions In order to provide pre-compiled binaries of third-party applications for &os;, the Ports Collection is regularly built on one of the Package Building Clusters. Currently, the main cluster in use is at . Most of the package building magic occurs under the /var/portbuild directory. Unless otherwise specified, all paths will be relative to this location. ${arch} will be used to specify one of the package architectures (amd64, &i386;, ia64, powerpc, and &sparc64;), and ${branch} will be used to specify the build branch (6, 7, 7-exp, 8, 8-exp, 9, 9-exp). Packages are no longer built for Release 4 or 5, nor for the alpha architecture. The scripts that control all of this live in /var/portbuild/scripts/. These are the checked-out copies from /usr/ports/Tools/portbuild/scripts/. Typically, incremental builds are done that use previous - packages as dependendencies; this takes less time, and puts less + packages as dependencies; this takes less time, and puts less load on the mirrors. Full builds are usually only done: right after release time, for the -STABLE branches periodically to test changes to -CURRENT for experimental builds Build Client Management The &i386; clients co-located with pointyhat netboot from it (connected nodes); all other clients (disconnected nodes) are either self-hosted or netboot from some other pxe host. In all cases they set themselves up at boot-time to prepare to build packages. The cluster master rsyncs the interesting data (ports and src trees, bindist tarballs, scripts, etc.) to disconnected nodes during the node-setup phase. Then, the disconnected portbuild directory is nullfs-mounted for chroot builds. The ports-${arch} user can &man.ssh.1; to the client nodes to monitor them. Use sudo and check the portbuild.hostname.conf for the user and access details. The scripts/allgohans script can be used to run a command on all of the ${arch} clients. The scripts/checkmachines script is used to monitor the load on all the nodes of the build cluster, and schedule which nodes build which ports. This script is not very robust, and has a tendency to die. It is best to start up this script on the build master (e.g. pointyhat) after boot time using a &man.while.1; loop. Chroot Build Environment Setup Package builds are performed in a chroot populated by the portbuild script using the ${arch}/${branch}/builds/${buildid}/bindist.tar file. The following command builds a world from the ${arch}/${branch}/src tree and installs it into ${worlddir}. The tree will be updated first unless -nocvs is specified. /var/portbuild&prompt.root; scripts/makeworld ${arch} ${branch} ${buildid} [-nocvs] The bindist.tar tarball is created from the previously installed world by the mkbindist script. It should be run as root with the following command: /var/portbuild&prompt.root; scripts/mkbindist ${arch} ${branch} ${buildid} The per-machine tarballs are located in ${arch}/clients. The bindist.tar file is extracted onto each client at client boot time, and at the start of each pass of the dopackages script. Starting the Build Several separate builds for each architecture - branch combination are supported. All data private to a build (ports tree, src tree, packages, distfiles, log files, bindist, Makefile, etc) are located under ${arch}/${branch}/builds/${buildid}. The last created build can be alternatively referenced under buildid latest, the one before is called previous. New builds are cloned from the latest, which is fast since it uses ZFS. <command>dopackages</command> scripts The scripts/dopackages* scripts are used to perform the builds. Most useful are: dopackages.6 - Perform a 6.X build dopackages.7 - Perform a 7.X build dopackages.7-exp - Perform a 7.X build with experimental patches (7-exp branch) dopackages.8 - Perform a 8.X build dopackages.8-exp - Perform a 8.X build with experimental patches (8-exp branch) dopackages.9 - Perform a 9.X build dopackages.9-exp - Perform a 9.X build with experimental patches (9-exp branch) These are wrappers around dopackages, and are all symlinked to dopackages.wrapper. New branch wrapper scripts can be created by symlinking dopackages.${branch} to dopackages.wrapper. These scripts take a number of arguments. For example: dopackages.6 ${arch} ${buildid} [-options] Most often, you will be using latest for the value of buildid. [-options] may be zero or more of the following: -keep - Do not delete this build in the future, when it would be normally deleted as part of the latest - previous cycle. Don't forget to clean it up manually when you no longer need it. -nofinish - Do not perform post-processing once the build is complete. Useful if you expect that the build will need to be restarted once it finishes. If you use this option, don't forget to cleanup the clients when you don't need the build anymore. -finish - Perform post-processing only. -nocleanup - By default, when the -finish stage of the build is complete, the build data will be deleted from the clients. This option will prevent that. -restart - Restart an interrupted (or non-finished) build from the beginning. Ports that failed on the previous build will be rebuilt. -continue - Restart an interrupted (or non-finished) build. Will not rebuild ports that failed on the previous build. -incremental - Compare the interesting fields of the new INDEX with the previous one, remove packages and log files for the old ports that have changed, and rebuild the rest. This cuts down on build times substantially since unchanged ports do not get rebuilt every time. -cdrom - This package build is intended to end up on a CD-ROM, so NO_CDROM packages and distfiles should be deleted in post-processing. -nobuild - Perform all the preprocessing steps, but do not actually do the package build. -noindex - Do not rebuild INDEX during preprocessing. -noduds - Do not rebuild the duds file (ports that are never built, e.g. those marked IGNORE, NO_PACKAGE, etc.) during preprocessing. -trybroken - Try to build BROKEN ports (off by default because the amd64/&i386; clusters are fast enough now that when doing incremental builds, more time was spent rebuilding things that were going to fail anyway. Conversely, the other clusters are slow enough that it would be a waste of time to try and build BROKEN ports). -nosrc - Do not update the src tree from the ZFS snapshot, keep the tree from previous build instead. -srccvs - Do not update the src tree from the ZFS snapshot, update it with cvs update instead. -noports - Do not update the ports tree from the ZFS snapshot, keep the tree from previous build instead. -portscvs - Do not update the ports tree from the ZFS snapshot, update it with cvs update instead. -norestr - Do not attempt to build RESTRICTED ports. -plistcheck - Make it fatal for ports to leave behind files after deinstallation. -nodistfiles - Do not collect distfiles that pass make checksum for later uploading to ftp-master. -fetch-original - Fetch the distfile from the original MASTER_SITES rather than ftp-master. Unless you specify -restart, -continue, or -finish, the symlinks for the existing builds will be rotated. i.e, the existing symlink for previous will be deleted; the most recent build will have its symlink changed to previous/; and a new build will be created and symlinked into latest/. If the last build finished cleanly you do not need to delete anything. If it was interrupted, or you selected -nocleanup, you need to clean up clients by running build cleanup ${arch} ${branch} ${buildid} -full errors/, logs/, packages/, and so forth, are cleaned by the scripts. If you are short of space, you can also clean out ports/distfiles/. Leave the latest/ directory alone; it is a symlink for the webserver. dosetupnodes is supposed to be run from the dopackages script in the -restart case, but it can be a good idea to run it by hand and then verify that the clients all have the expected job load. Sometimes, dosetupnode cannot clean up a build and you need to do it by hand. (This is a bug.) Make sure the ${arch} build is run as the ports-${arch} user or it will complain loudly. The actual package build itself occurs in two identical phases. The reason for this is that sometimes transient problems (e.g. NFS failures, FTP sites being unreachable, etc.) may halt a build. Doing things in two phases is a workaround for these types of problems. Be careful that ports/Makefile does not specify any empty subdirectories. This is especially important if you are doing an -exp build. If the build process encounters an empty subdirectory, both package build phases will stop short, and an error similar to the following will be written to ${arch}/${branch}/make.[0|1]: don't know how to make dns-all(continuing) To correct this problem, simply comment out or remove the SUBDIR entries that point to empty subdirectories. After doing this, you can restart the build by running the proper dopackages command with the -restart option. This problem also appears if you create a new category Makefile with no SUBDIRs in it. This is probably a bug. Update the i386-6 tree and do a complete build dopackages.6 i386 -nosrc -norestr -nofinish Restart an interrupted amd64-8 build without updating dopackages.8 amd64 -nosrc -noports -norestr -continue -noindex -noduds -nofinish Post-process a completed sparc64-7 tree dopackages.7 sparc64 -finish Hint: it us usually best to run the dopackages command inside of screen(1). <command>build</command> command You may need to manipulate the build data before starting it, especially for experimental builds. This is done with the build command. Here are the useful options for creation: build create arch branch [newid] - Creates newid (or a datestamp if not specified). Only needed when bringing up a new branch or a new architecture. build clone arch branch oldid [newid] - Clones oldid to newid (or a datestamp if not specified). build srcupdate arch branch buildid - Replaces the src tree with a new ZFS snapshot. Don't forget to use -nosrc flag to dopackages later! build portsupdate arch branch buildid - Replaces the ports tree with a new ZFS snapshot. Don't forget to use -noports flag to dopackages later! Building a single package Sometimes there is a need to rebuild a single package from the package set. This can be accomplished with the following invocation: /var/portbuild/evil/qmanager/packagebuild amd64 7-exp 20080904212103 aclock-0.2.3_2.tbz Anatomy of a Build A full build without any -no options performs the following operations in the specified order: An update of the current ports tree from the ZFS snapshot [*] An update of the running branch's src tree from the ZFS snapshot [*] Checks which ports do not have a SUBDIR entry in their respective category's Makefile [*] Creates the duds file, which is a list of ports not to build [*] [+] Generates a fresh INDEX file [*] [+] Sets up the nodes that will be used in the build [*] [+] Builds a list of restricted ports [*] [+] Builds packages (phase 1) [++] Performs another node setup [+] Builds packages (phase 2) [++] [*] Status of these steps can be found in ${arch}/${branch}/build.log as well as on stderr of the tty running the dopackages command. [+] If any of these steps fail, the build will stop cold in its tracks. [++] Status of these steps can be found in ${arch}/${branch}/make.[0|1], where make.0 is the log file used by phase 1 of the package build and make.1 is the log file used by phase 2. Individual ports will write their build logs to ${arch}/${branch}/logs and their error logs to ${arch}/${branch}/errors. Formerly the docs tree was also checked out, however, it has been found to be unnecessary. Build Maintenance There are several cases where you will need to manually clean up a build: You have manually interrupted it. pointyhat has been rebooted while a build was running. qmanager has crashed and has been restarted. Interrupting a Build Manually interrupting a build is a bit messy. First you need to identify the tty in which it's running (either record the output of &man.tty.1; when you start the build, or use ps x to identify it. You need to make sure that nothing else important is running in this tty, e.g. ps -t p1 or whatever. If there is not, you can just kill off the whole term easily with pkill -t pts/1; otherwise issue a kill -HUP in there by, for example, ps -t pts/1 -o pid= | xargs kill -HUP. Replace p1 by whatever the tty is, of course. The package builds dispatched by make to the client machines will clean themselves up after a few minutes (check with ps x until they all go away). If you do not kill &man.make.1;, then it will spawn more jobs. If you do not kill dopackages, then it will restart the entire build. If you do not kill the pdispatch processes, they'll keep going (or respawn) until they've built their package. Cleaning up a Build To free up resources, you will need to clean up client machines by running build cleanup command. For example: &prompt.user; /var/portbuild/scripts/build cleanup i386 8-exp 20080714120411 -full If you forget to do this, then the old build chroots will not be cleaned up for 24 hours, and no new jobs will be dispatched in their place since pointyhat thinks the job slot is still occupied. To check, cat ~/loads/* to display the status of client machines; the first column is the number of jobs it thinks is running, and this should be roughly concordant with the load average. loads is refreshed every 2 minutes. If you do ps x | grep pdispatch and it's less than the number of jobs that loads thinks are in use, you're in trouble. You may have problem with the umount commands hanging. If so, you are going to have to use the allgohans script to run an &man.ssh.1; command across all clients for that buildenv. For example: ssh -l root gohan24 df will get you a df, and allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports" allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/src" are supposed to get rid of the hanging mounts. You will have to keep doing them since there can be multiple mounts. Ignore the following: umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: statfs: No such file or directory umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: unknown file system umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed! /x/tmp/8-exp/chroot/53837/compat/linux/proc: not a file system root directory The former 2 mean that that client did not have those mounted; the latter 2 are a bug. You may also see messages about procfs. After you have done all the above, remove the ${arch}/lock file before trying to restart the build. If you do not, dopackages will simply exit. If you have to do a ports tree update before restarting, you may have to rebuild either duds, INDEX, or both. Maintaining builds with the <command>build</command> command Here are the rest of the options for the build command: build destroy arch branch - Destroy the build id. build list arch branch - Shows the current set of build ids. build upload arch branch - not yet implemented. Monitoring the Build You can use qclient command to monitor the status of build nodes, and to list the currently scheduled jobs: python /var/portbuild/evil/qmanager/qclient jobs python /var/portbuild/evil/qmanager/qclient status The scripts/stats ${branch} command shows the number of packages already built. Running cat /var/portbuild/*/loads/* shows the client loads and number of concurrent builds in progress. The files that have been recently updated are the clients that are online; the others are the offline clients. The pdispatch command does the dispatching of work onto the client, and post-processing. ptimeout.host is a watchdog that kills a build after timeouts. So, having 50 pdispatch processes but only 4 &man.ssh.1; processes means 46 pdispatches are idle, waiting to get an idle node. Running tail -f ${arch}/${branch}/build.log shows the overall build progress. If a port build is failing, and it is not immediately obvious from the log as to why, you can preserve the WRKDIR for further analysis. To do this, touch a file called .keep in the port's directory. The next time the cluster tries to build this port, it will tar, compress, and copy the WRKDIR to ${arch}/${branch}/wrkdirs. If you find that the system is looping trying to build the same package over and over again, you may be able to fix the problem by rebuilding the offending package by hand. If all the builds start failing with complaints that they cannot load the dependent packages, check to see that httpd is still running, and restart it if not. Keep an eye on &man.df.1; output. If the /var/portbuild file system becomes full then Bad Things happen. The status of all current builds is generated twice an hour and posted to . For each buildenv, the following is displayed: cvs date is the contents of cvsdone. This is why we recommend that you update cvsdone for -exp runs (see below). date of latest log number of lines in INDEX the number of current build logs the number of completed packages the number of errors the number of duds (shown as skipped) missing shows the difference between INDEX and the other columns. If you have restarted a run after a cvs update, there will likely be duplicates in the packages and error columns, and this column will be meaningless. (The script is naive). running and completed are guesses based on a &man.grep.1; of build.log. Dealing With Build Errors The easiest way to track build failures is to receive the emailed logs and sort them to a folder, so you can maintain a running list of current failures and detect new ones easily. To do this, add an email address to ${branch}/portbuild.conf. You can easily bounce the new ones to maintainers. After a port appears broken on every build combination multiple times, it is time to mark it BROKEN. Two weeks' notification for the maintainers seems fair. To avoid build errors with ports that need to be manually fetched, put the distfiles into ~ftp/pub/FreeBSD/distfiles. Release Builds When building packages for a release, it may be necessary to manually update the ports and src trees to the release tag and use -nocvs and -noportscvs. To build package sets intended for use on a CD-ROM, use the -cdrom option to dopackages. If the disk space is not available on the cluster, use -nodistfiles to avoid collecting distfiles. After the initial build completes, restart the build with -restart -fetch-original to collect updated distfiles as well. Then, once the build is post-processed, take an inventory of the list of files fetched: &prompt.user; cd ${arch}/${branch} &prompt.user; find distfiles > distfiles-${release} This inventory file typically lives in i386/${branch} on the cluster master. This is useful to aid in periodically cleaning out the distfiles from ftp-master. When space gets tight, distfiles from recent releases can be kept while others can be thrown away. Once the distfiles have been uploaded (see below), the final release package set must be created. Just to be on the safe side, run the ${arch}/${branch}/cdrom.sh script by hand to make sure all the CD-ROM restricted packages and distfiles have been pruned. Then, copy the ${arch}/${branch}/packages directory to ${arch}/${branch}/packages-${release}. Once the packages are safely moved off, contact the &a.re; and inform them of the release package location. Remember to coordinate with the &a.re; about the timing and status of the release builds. Uploading Packages Once a build has completed, packages and/or distfiles can be transferred to ftp-master for propagation to the FTP mirror network. If the build was run with -nofinish, then make sure to follow up with dopackages -finish to post-process the packages (removes RESTRICTED and NO_CDROM packages where appropriate, prunes packages not listed in INDEX, removes from INDEX references to packages not built, and generates a CHECKSUM.MD5 summary); and distfiles (moves them from the temporary distfiles/.pbtmp directory into distfiles/ and removes RESTRICTED and NO_CDROM distfiles). It is usually a good idea to run the restricted.sh and/or cdrom.sh scripts by hand after dopackages finishes just to be safe. Run the restricted.sh script before uploading to ftp-master, then run cdrom.sh before preparing the final package set for a release. The package subdirectories are named by whether they are for release, stable, or current. Examples: packages-6.4-release packages-6-stable packages-7.2-release packages-7-stable packages-8-stable packages-9-current Some of the directories on ftp-master are, in fact, symlinks. Examples: packages-stable packages-current Be sure you move the new packages directory over the real destination directory, and not one of the symlinks that points to it. If you are doing a completely new package set (e.g. for a new release), copy packages to the staging area on ftp-master with something like the following: &prompt.root; cd /var/portbuild/${arch}/${branch} &prompt.root; tar cfv - packages/ | ssh portmgr@ftp-master tar xfC - w/ports/${arch}/tmp/${subdir} Then log into ftp-master, verify that the package set was transferred successfully, remove the package set that the new package set is to replace (in ~/w/ports/${arch}), and move the new set into place. (w/ is merely a shortcut.) For incremental builds, packages should be uploaded using rsync so we do not put too much strain on the mirrors. ALWAYS use -n first with rsync and check the output to make sure it is sane. If it looks good, re-run the rsync without the -n option. Example rsync command for incremental package upload: &prompt.root; rsync -n -r -v -l -t -p --delete packages/ portmgr@ftp-master:w/ports/${arch}/${subdir}/ | tee log Distfiles can be transferred with the cpdistfiles script: &prompt.root; /var/portbuild/scripts/cpdistfiles ${arch} ${branch} Or you can do it by hand using rsync command: &prompt.root; cd /var/portbuild/${arch}/${branch} &prompt.root; rsync -n -r -v -l -p -c distfiles/ portmgr@ftp-master:w/ports/distfiles/ | tee log Again, run the command without the -n option after you have checked it. Experimental Patches Builds Experimental patches builds are run from time to time to new features or bugfixes to the ports infrastructure (i.e. bsd.port.mk), or to test large sweeping upgrades. At any given time there may be several simultaneous experimental patches branches, such as 8-exp on the amd64 architecture. In general, an experimental patches build is run the same way as any other build, except that you should first update the ports tree to the latest version and then apply your patches. To do the former, you can use the following: &prompt.user; cvs -R update -dP > update.out &prompt.user; date > cvsdone This will most closely simulate what the dopackages script does. (While cvsdone is merely informative, it can be a help.) You will need to edit update.out to look for lines beginning with ^M, ^C, or ^? and then deal with them. It is always a good idea to save original copies of all changed files, as well as a list of what you are changing. You can then look back on this list when doing the final commit, to make sure you are committing exactly what you tested. Since the machine is shared, someone else may delete your changes by mistake, so keep a copy of them in e.g. your home directory on freefall. Do not use tmp/; since pointyhat itself runs some version of -CURRENT, you can expect reboots (if nothing else, for updates). In order to have a good control case with which to compare failures, you should first do a package build of the branch on which the experimental patches branch is based for the &i386; architecture (currently this is 7). Then, when preparing for the experimental patches build, checkout a ports tree and a src tree with the same date as was used for the control build. This will ensure an apples-to-apples comparison later. Once the build finishes, compare the control build failures to those of the experimental patches build. Use the following commands to facilitate this (this assumes the 8 branch is the control branch, and the 8-exp branch is the experimental patches branch): &prompt.user; cd /var/portbuild/i386/8-exp/errors &prompt.user; find . -name \*.log\* | sort > /tmp/8-exp-errs &prompt.user; cd /var/portbuild/i386/8/errors &prompt.user; find . -name \*.log\* | sort > /tmp/8-errs If it has been a long time since one of the builds finished, the logs may have been automatically compressed with bzip2. In that case, you must use sort | sed 's,\.bz2,,g' instead. &prompt.user; comm -3 /tmp/8-errs /tmp/8-exp-errs | less This last command will produce a two-column report. The first column is ports that failed on the control build but not in the experimental patches build; the second column is vice versa. Reasons that the port might be in the first column include: Port was fixed since the control build was run, or was upgraded to a newer version that is also broken (thus the newer version should appear in the second column) Port is fixed by the patches in the experimental patches build Port did not build under the experimental patches build due to a dependency failure Reasons for a port appearing in the second column include: Port was broken by the experimental patches [1] Port was upgraded since the control build and has become broken [2] Port was broken due to a transient error (e.g. FTP site down, package client error, etc.) Both columns should be investigated and the reason for the errors understood before committing the experimental patches set. To differentiate between [1] and [2] above, you can do a rebuild of the affected packages under the control branch: &prompt.user; cd /var/portbuild/i386/8/ports Be sure to cvs update this tree to the same date as the experimental patches tree. The following command will set up the control branch for the partial build: &prompt.user; /var/portbuild/scripts/dopackages.8 -noportscvs -nobuild -nocvs -nofinish The builds must be performed from the packages/All directory. This directory should initially be empty except for the Makefile symlink. If this symlink does not exist, it must be created: &prompt.user; cd /var/portbuild/i386/8/packages/All &prompt.user; ln -sf ../../Makefile . &prompt.user; make -k -j<#> <list of packages to build> <#> is the concurrency of the build to attempt. It is usually the sum of the weights listed in /var/portbuild/i386/mlist unless you have a reason to run a heavier or lighter build. The list of packages to build should be a list of package names (including versions) as they appear in INDEX. The PKGSUFFIX (i.e. .tgz or .tbz) is optional. This will build only those packages listed as well as all of their dependencies. You can check the progress of this partial build the same way you would a regular build. Once all the errors have been resolved, you can commit the package set. After committing, it is customary to send a HEADS UP email to ports@FreeBSD.org and copy ports-developers@FreeBSD.org informing people of the changes. A summary of all changes should also be committed to /usr/ports/CHANGES. How to configure a new package building node Before following these steps, please coordinate with portmgr. Node requirements portmgr is still working on characterizing what a node needs to be generally useful. CPU capacity: TBA. However, we have several dual-CPU P-III &i386; 1.0GHz machines available, so anything with less horsepower than that is not as likely to be useful. (However, many of our &sparc64;s are single-CPU, 500MHz machines, so our requirements are lower.) We are able to adjust the number of jobs dispatched to each machine, and we generally tune the number to use 100% of CPU. RAM: TBA. Again, we have been tuning to one job per 512M of RAM. (Anything less than 512M is very unlikely to be useful.) disk: at least 20G is needed for filesystem; 32G is needed for swap. Best performance will be if multiple disks are used, and configured as geom stripes. Performance numbers are also TBA. Package building will test disk drives to destruction. Be aware of what you are signing up for! network bandwidth: TBA. However, an 8-job machine has been shown to saturate a cable modem line. Preparation Pick a unique hostname. It does not have to be a publicly resolvable hostname (it can be a name on your internal network). By default, package building requires the following TCP ports to be accessible: 22 (ssh), 414 (infoseek), and 8649 (ganglia). If these are not accessible, pick others and ensure that an ssh tunnel is set up (see below). (Note: if you have more than one machine at your site, you will need an individual TCP port for each service on each machine, and thus ssh tunnels will be necessary. As such, you will probably need to configure port forwarding on your firewall.) Decide if you will be booting natively or via pxeboot. You will find that it is easier to keep up with changes to -current with the latter, especially if you have multiple machines at your site. Pick a directory to hold ports configuration and chroot subdirectories. It may be best to put it this on its own partition. (Example: /usr2/.) Configuring <literal>src</literal> Create a directory to contain the latest -current source tree and check it out. (Since your machine will likely be asked to build packages for -current, the kernel it runs should be reasonably up-to-date with the bindist that will be exported by our scripts.) If you are using pxeboot: create a directory to contain the install bits. You will probably want to use a subdirectory of /pxeroot, e.g., /pxeroot/${arch}-${branch}. Export that as DESTDIR. If you are cross-building, export TARGET_ARCH=${arch}. The procedure for cross-building ports is not yet defined. Generate a kernel config file. Include GENERIC (or, if you are using more than 3.5G on &i386;, PAE). Required options: options NULLFS options TMPFS Suggested options: options GEOM_CONCAT options GEOM_STRIPE options SHMMAXPGS=65536 options SEMMNI=40 options SEMMNS=240 options SEMUME=40 options SEMMNU=120 options ALT_BREAK_TO_DEBUGGER options PRINTF_BUFR_SIZE=128 For PAE, it is not currently possible to load modules. Therefore, if you are running an architecture that supports Linux emulation, you will need to add: options COMPAT_LINUX options LINPROCFS As root, do the usual build steps, e.g.: make -j4 buildworld make buildkernel KERNCONF=${kernconf} make installkernel KERNCONF=${kernconf} make installworld The install steps use DESTDIR. Customize files in etc/. Whether you do this on the client itself, or another machine, will depend on whether you are using pxeboot. If you are using pxeboot: create a subdirectory of ${DESTDIR} called conf/. Create one subdirectory default/etc/, and (if your site will host multiple nodes), subdirectories ${ip-address}/etc/ to contain override files for individual hosts. (You may find it handy to symlink each of those directories to a hostname.) Copy the entire contents of ${DESTDIR}/etc/ to default/etc/; that is where you will edit your files. The by-ip-address etc/ directories will probably only need customized rc.conf files. In either case, apply the following steps: Create a ports-${arch} user and group. Add it to the wheel group. It can have the '*' password. Create /home/ports-${arch}/.ssh/ and populate authorized_keys. Also add the following users: squid:*:100:100::0:0:User &:/usr/local/squid:/bin/sh ganglia:*:102:102::0:0:User &:/usr/local/ganglia:/bin/sh Add them to etc/group as well. Create the appropriate files in etc/.ssh/. In etc/crontab: add * * * * * root /var/portbuild/scripts/client-metrics Create the appropriate etc/fstab. (If you have multiple, different, machines, you will need to put those in the override directories.) In etc/inetd.conf: add infoseek stream tcp nowait nobody /var/portbuild/scripts/reportload reportload ${arch} We run the cluster on UTC: cp /usr/share/zoneinfo/Etc/UTC etc/localtime Create the appropriate etc/rc.conf. (If you are using pxeboot, and have multiple, different, machines, you will need to put those in the override directories.) Recommended entries: hostname="${hostname} inetd_enable="YES" linux_enable="YES" nfs_client_enable="YES" ntpd_enable="YES" ntpdate_enable="YES" ntpdate_flags="north-america.pool.ntp.org" sendmail_enable="NONE" sshd_enable="YES" sshd_program="/usr/local/sbin/sshd" gmond_enable="YES" squid_enable="YES" squid_chdir="/usr2/squid/logs" squid_pidfile="/usr2/squid/logs/squid.pid" Create etc/resolv.conf, if necessary. Modify etc/sysctl.conf: 9a10,30 > kern.corefile=/usr2/%N.core > kern.sugid_coredump=1 > #debug.witness_ddb=0 > #debug.witness_watch=0 > > # squid needs a lot of fds (leak?) > kern.maxfiles=40000 > kern.maxfilesperproc=30000 > > # Since the NFS root is static we don't need to check frequently for file changes > # This saves >75% of NFS traffic > vfs.nfs.access_cache_timeout=300 > debug.debugger_on_panic=1 > > # For jailing > security.jail.sysvipc_allowed=1 > security.jail.allow_raw_sockets=1 > security.jail.chflags_allowed=1 > security.jail.enforce_statfs=1 > > vfs.lookup_shared=1 If desired, modify etc/syslog.conf to change the logging destinations to @pointyhat.freebsd.org. Configuring <literal>ports</literal> Install the following ports: net/rsync security/openssh-portable (with HPN on) security/sudo sysutils/ganglia-monitor-core (with GMETAD off) www/squid (with SQUID_AUFS on) There is a WIP to create a meta-port, but it is not yet complete. Customize files in usr/local/etc/. Whether you do this on the client itself, or another machine, will depend on whether you are using pxeboot. The trick of using conf override subdirectories is less effective here, because you would need to copy over all subdirectories of usr/. This is an implementation detail of how the pxeboot works. Apply the following steps: Modify usr/local/etc/gmond.conf: 21,22c21,22 < name = "unspecified" < owner = "unspecified" --- > name = "${arch} package build cluster" > owner = "portmgr@FreeBSD.org" 24c24 < url = "unspecified" --- > url = "http://pointyhat.freebsd.org" If there are machines from more than one cluster in the same multicast domain (basically = LAN) then change the multicast groups to different values (.71, .72, etc). Create usr/local/etc/rc.d/portbuild.sh, using the appropriate value for scratchdir: #!/bin/sh # # Configure a package build system post-boot scratchdir=/usr2 ln -sf ${scratchdir}/portbuild /var/ # Identify builds ready for use cd /var/portbuild/${arch} for i in */builds/*; do if [ -f ${i}/.ready ]; then mkdir /tmp/.setup-${i##*/} fi done # Flag that we are ready to accept jobs touch /tmp/.boot_finished Modify usr/local/etc/squid/squid.conf: 288,290c288,290 < #auth_param basic children 5 < #auth_param basic realm Squid proxy-caching web server < #auth_param basic credentialsttl 2 hours --- > auth_param basic children 5 > auth_param basic realm Squid proxy-caching web server > auth_param basic credentialsttl 2 hours 611a612 > acl localnet src 127.0.0.0/255.0.0.0 655a657 > http_access allow localnet 2007a2011 > maximum_object_size 400 MB 2828a2838 > negative_ttl 0 minutes Also, change usr/local to usr2 in cache_dir, access_log, cache_log, cache_store_log, pid_filename, netdb_filename, coredump_dir. Finally, change the cache_dir storage scheme from ufs to aufs (offers better performance). Configure ssh: copy /etc/ssh to /usr/local/etc/ssh and add NoneEnabled yes to sshd_config. Modify usr/local/etc/sudoers: 38a39,42 > > # local changes for package building > %wheel ALL=(ALL) ALL > ports-${arch} ALL=(ALL) NOPASSWD: ALL Configuration on the client itself Change into the port/package directory you picked above, e.g., cd /usr2. As root: mkdir portbuild chown ports-${arch}:ports-${arch} portbuild mkdir pkgbuild chown ports-${arch}:ports-${arch} pkgbuild mkdir squid mkdir squid/cache mkdir squid/logs chown -R squid:squid squid If clients preserve /var/portbuild between boots then they must either preserve their /tmp, or revalidate their available builds at boot time (see the script on the amd64 machines). They must also clean up stale chroots from previous builds before creating /tmp/.boot_finished. Boot the client. As root, initialize the squid directories: squid -z Configuration on <literal>pointyhat</literal> These steps need to be taken by a portmgr acting as root on pointyhat. If any of the default TCP ports is not available (see above), you will need to create an ssh tunnel for it and include it in the appropriate crontab. Add an entry to /home/ports-${arch}/.ssh/config to specify the public IP address, TCP port for ssh, username, and any other necessary information. Add the public IP address to /etc/hosts.allow. (Remember, multiple machines can be on the same IP address.) Create /var/portbuild/${arch}/clients/bindist-${hostname}.tar. Copy one of the existing ones as a template and unpack it in a temporary directory. Customize etc/resolv.conf and etc/make.conf for the local site. tar it up and move it to the right location. Hint: you will need one of these for each machine; however, if you have multiple machines at one site, you may be able to create a site-specific one and symlink to it. Create /var/portbuild/${arch}/portbuild-${hostname} using one of the existing ones as a guide. This file contains overrides to /var/portbuild/${arch}/portbuild.conf. Suggested values: disconnected=1 http_proxy="http://localhost:3128/" squid_dir=/usr2/squid scratchdir=/usr2/pkgbuild client_user=ports-${arch} sudo_cmd="sudo -H" rsync_gzip=-z infoseek_host=localhost infoseek_port=${tunelled-tcp-port} Possible other values: use_md_swap=1 md_size=9g use_zfs=1 scp_cmd="/usr/local/bin/scp" ssh_cmd="/usr/local/bin/ssh" Add an appropriate data_source entry to /usr/local/etc/gmetad.conf: data_source "arch/location Package Build Cluster" 30 hostname You will need to restart gmetad. Enabling the node These steps need to be taken by a portmgr acting as ports-arch on pointyhat. Ensure that ssh is working by executing ssh hostname. Populate /var/portbuild/scripts/ by something like /var/portbuild/dosetupnode arch major latest hostname. Verify that you now have files in that directory. Test the other TCP ports by executing telnet hostname portnumber. 414 (or its tunnel) should give you a few lines of status information including arch and osversion; 8649 should give you an XML response from ganglia. This step needs to be taken by a portmgr acting as root on pointyhat. Tell qmanager about the node. Example: python /var/portbuild/evil/qmanager/qclient add name=uniquename arch=arch osversion=osversion numcpus=number haszfs=0 online=1 domain=domain primarypool=package pools="package all" maxjobs=1 acl="ports-arch,deny_all" How to configure a new &os; branch When a new branch is created, some work needs to be done to specify that the previous branch is no longer equivalent to HEAD. The following instructions apply to the previous branch number: Create a new zfs filesystem for sources: zfs create a/snap/src-branch Checkout a src tree in the new filesystem: cvs -Rq -d /r/ncvs co -r RELENG-branch Edit the master copy of Tools/portbuild/portbuild.conf. For each arch, edit its copy of the above in /var/portbuild/arch/portbuild.conf. Edit /var/portbuild/scripts/buildenv. Add a link from /var/portbuild/scripts/dopackages to /var/portbuild/scripts/dopackages.branch. Modify HEAD_BRANCH and NON_HEAD_BRANCHES in /var/portbuild/scripts/updatesnap. Add the snap directory to /var/portbuild/scripts/zexpire. In the /var/portbuild/errorlogs/ directory, create links for the webserver: ln -s ../arch/branch/builds/latest/bak/errors arch-branch-full ln -s ../arch/branch/builds/latest/bak/logs arch-branch-full-logs ln -s ../arch/branch/builds/latest/errors arch-branch-latest ln -s ../arch/branch/builds/latest/logs arch-branch-latest-logs ln -s ../arch/branch/builds/latest/bak/packages arch-branch-packages-full ln -s ../arch/branch/builds/latest/packages arch-branch-packages-latest Kick-start the build for the branch with build create arch branch Create bindist.tar . How to configure a new architecture Create a new ports-arch user and group. mkdir /var/portbuild/arch; cd /var/portbuild/arch Create a new zfs filesystem: zfs create -o mountpoint=/a/portbuild/arch a/portbuild/arch Create a directory for buildlogs and errorlogs: mkdir /dumpster/pointyhat/arch/archive It is possible that /dumpster/pointyhat will not have enough space. In that case, create the archive directory as /dumpster/pointyhat/arch/archive and symlink to that. (This needs to be sorted out.) Create a link to the above for the webserver: ln -s /dumpster/pointyhat/arch/archive archive In the /var/portbuild/arch directory:mkdir clients Populate clients as usual. mkdir loads mkdir lockfiles Create a local make.conf. In the most trivial case, you can ln ../make.conf ./make.conf Create an empty mlist file. Create pnohang.arch. (The easiest way may be to do the following on a client, and then copy it back): cc pnohang.c -o pnohang-arch Create a fresh portbuild.conf file from one of the ones for another architecture. Create customized portbuild.machinename.conf files as appropriate. cd .ssh && ssh-keygen Edit the .ssh/config file for convenience in using ssh. Make the private configuration directory: mkdir /var/portbuild/conf/arch In that directory: create any dotunnel.* scripts needed. Tell qmanager about the arch: python /var/portbuild/evil/qmanager/qclient add_acl name=ports-arch uidlist=ports-arch gidlist=portmgr sense=1 Edit /var/portbuild/scripts/buildenv. Add the arch directory to /var/portbuild/scripts/zbackup and /var/portbuild/scripts/zexpire. As with the procedure for creating a new branch: in the /var/portbuild/errorlogs/ directory, create links for the webserver: ln -s ../arch/branch/builds/latest/bak/errors arch-branch-full ln -s ../arch/branch/builds/latest/bak/logs arch-branch-full-logs ln -s ../arch/branch/builds/latest/errors arch-branch-latest ln -s ../arch/branch/builds/latest/logs arch-branch-latest-logs ln -s ../arch/branch/builds/latest/bak/packages arch-branch-packages-full ln -s ../arch/branch/builds/latest/packages arch-branch-packages-latest In that directory, create two more links for the webserver: ln -s ../arch/archive/buildlogs arch-buildlogs ln -s ../arch/archive/errorlogs arch-errorlogs For each branch that will be supported, do the following: Kick-start the build for the branch with build create arch branch Create bindist.tar. Only after the first time a dopackages has been run for the arch: add the arch to /var/portbuild/scripts/dopackagestats. Procedures for dealing with disk failures When a machine has a disk failure (e.g. panics due to read errors, etc), then we should do the following steps: Note the time and failure mode (e.g. paste in the relevant console output) in /var/portbuild/${arch}/reboots For i386 gohan clients, scrub the disk by touching /SCRUB in the nfsroot (e.g. /a/nfs/8.dir1/SCRUB) and rebooting. This will dd if=/dev/zero of=/dev/ad0 and force the drive to remap any bad sectors it finds, if it has enough spares left. This is a temporary measure to extend the lifetime of a drive that is on the way out. For the i386 blade systems another signal of a failing disk seems to be that the blade will completely hang and be unresponsive to either console break, or even NMI. For other build systems that don't newfs their disk at boot (e.g. amd64 systems) this step has to be skipped. If the problem recurs, then the disk is probably toast. Take the machine out of mlist and (for ata disks) run smartctl on the drive: smartctl -t long /dev/ad0 It will take about 1/2 hour: gohan51# smartctl -t long /dev/ad0 smartctl version 5.38 [i386-portbld-freebsd8.0] Copyright (C) 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION === Sending command: "Execute SMART Extended self-test routine immediately in off-line mode". Drive command "Execute SMART Extended self-test routine immediately in off-line mode" successful. Testing has begun. Please wait 31 minutes for test to complete. Test will complete after Fri Jul 4 03:59:56 2008 Use smartctl -X to abort test. Then smartctl -a /dev/ad0 shows the status after it finishes: # SMART Self-test log structure revision number 1 # Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed: read failure 80% 15252 319286 It will also display other data including a log of previous drive errors. It is possible for the drive to show previous DMA errors without failing the self-test though (because of sector remapping). When a disk has failed, please inform the cluster administrators so we can try to get it replaced.
diff --git a/en_US.ISO8859-1/articles/problem-reports/article.sgml b/en_US.ISO8859-1/articles/problem-reports/article.sgml index fd4d2ed04b..3b58f4c917 100644 --- a/en_US.ISO8859-1/articles/problem-reports/article.sgml +++ b/en_US.ISO8859-1/articles/problem-reports/article.sgml @@ -1,1284 +1,1284 @@ %articles.ent; ]>
Writing &os; Problem Reports $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.cvsup; &tm-attrib.ibm; &tm-attrib.intel; &tm-attrib.sparc; &tm-attrib.sun; &tm-attrib.general; This article describes how to best formulate and submit a problem report to the &os; Project. Dag-Erling Smørgrav Contributed by Mark Linimon problem reports
Introduction One of the most frustrating experiences one can have as a software user is to submit a problem report only to have it summarily closed with a terse and unhelpful explanation like not a bug or bogus PR. Similarly, one of the most frustrating experiences as a software developer is to be flooded with problem reports that are not really problem reports but requests for support, or that contain little or no information about what the problem is and how to reproduce it. This document attempts to describe how to write good problem reports. What, you ask, is a good problem report? Well, to go straight to the bottom line, a good problem report is one that can be analyzed and dealt with swiftly, to the mutual satisfaction of both user and developer. Although the primary focus of this article is on &os; problem reports, most of it should apply quite well to other software projects. Note that this article is organized thematically, not chronologically, so you should read through the entire document before submitting a problem report, rather than treat it as a step-by-step tutorial.
When to submit a problem report There are many types of problems, and not all of them should engender a problem report. Of course, nobody is perfect, and there will be times when you are convinced you have found a bug in a program when in fact you have misunderstood the syntax for a command or made a typographical error in a configuration file (though that in itself may sometimes be indicative of poor documentation or poor error handling in the application). There are still many cases where submitting a problem report is clearly not the right course of action, and will only serve to frustrate you and the developers. Conversely, there are cases where it might be appropriate to submit a problem report about something else than a bug—an enhancement or a feature request, for instance. So how do you determine what is a bug and what is not? As a simple rule of thumb your problem is not a bug if it can be expressed as a question (usually of the form How do I do X? or Where can I find Y?). It is not always quite so black and white, but the question rule covers a large majority of cases. If you are looking for an answer, consider posing your question to the &a.questions;. Some cases where it may be appropriate to submit a problem report about something that is not a bug are: Requests for feature enhancements. It is generally a good idea to air these on the mailing lists before submitting a problem report. Notification of updates to externally maintained software (mainly ports, but also externally maintained base system components such as BIND or various GNU utilities). For unmaintained ports (MAINTAINER contains ports@FreeBSD.org), such update notifications might get picked up by an interested committer, or you might be asked to provide a patch to update the port; providing it upfront will greatly improve your chances that the port will get updated in a timely manner. If the port is maintained, PRs announcing new upstream releases are usually not very useful since they generate supplementary work for the committers, and the maintainer likely knows already there is a new version, they have probably worked with the developers on it, they are probably testing to see there is no regression, etc. In either case, following the process described in Porter's Handbook will yield the best results. (You might also wish to read Contributing to the FreeBSD Ports Collection.) A bug that can not be reproduced can rarely be fixed. If the bug only occurred once and you can not reproduce it, and it does not seem to happen to anybody else, chances are none of the developers will be able to reproduce it or figure out what is wrong. That does not mean it did not happen, but it does mean that the chances of your problem report ever leading to a bug fix are very slim. To make matters worse, often these kinds of bugs are actually caused by failing hard drives or overheating processors — you should always try to rule out these causes, whenever possible, before submitting a PR. Next, to decide to whom you should file your problem report, you need to understand that the software that makes up &os; is composed of several different elements: Code in the base system that is written and maintained by &os; contributors, such as the kernel, the C library, and the device drivers (categorized as kern); the binary utilities (bin); the manual pages and documentation (docs); and the web pages (www). All bugs in these areas should be reported to the &os; developers. Code in the base system that is written and maintained by others, and imported into &os; and adapted. Examples include bind, &man.gcc.1;, and &man.sendmail.8;. Most bugs in these areas should be reported to the &os; developers; but in some cases they may need to be reported to the original authors instead if the problems are not &os;-specific. Usually these bugs will fall under either the bin or gnu categories. Individual applications that are not in the base system but are instead part of the &os; Ports Collection (category ports). Most of these applications are not written by &os; developers; what &os; provides is merely a framework for installing the application. Therefore, you should only report a problem to the &os; developers when you believe the problem is &os;-specific; otherwise, you should report it to the authors of the software. Then you should ascertain whether or not the problem is timely. There are few things that will annoy a developer more than receiving a problem report about a bug she has already fixed. If the problem is in the base system, you should first read the FAQ section on &os; versions, if you are not already familiar with the topic. It is not possible for &os; to fix problems in anything other than certain recent branches of the base system, so filing a bug report about an older version will probably only result in a developer advising you to upgrade to a supported version to see if the problem still recurs. The Security Officer team maintains the list of supported versions. If the problem is in a port, note that you must first upgrade to the latest version of the Ports Collection and see if the problem still applies. Due to the rapid pace of changes in these applications, it is infeasible for &os; to support anything other than the absolute latest versions, and problems with older version of applications simply cannot be fixed.
Preparations A good rule to follow is to always do a background search before submitting a problem report. Maybe your problem has already been reported; maybe it is being discussed on the mailing lists, or recently was; it may even already be fixed in a newer version than what you are running. You should therefore check all the obvious places before submitting your problem report. For &os;, this means: The &os; Frequently Asked Questions (FAQ) list. The FAQ attempts to provide answers for a wide range of questions, such as those concerning hardware compatibility, user applications, and kernel configuration. The mailing lists—if you are not subscribed, use the searchable archives on the &os; web site. If your problem has not been discussed on the lists, you might try posting a message about it and waiting a few days to see if someone can spot something you have overlooked. Optionally, the entire web—use your favorite search engine to locate any references to your problem. You may even get hits from archived mailing lists or newsgroups you did not know of or had not thought to search through. Next, the searchable &os; PR database (GNATS). Unless your problem is recent or obscure, there is a fair chance it has already been reported. Most importantly, you should attempt to see if existing documentation in the source base addresses your problem. For the base &os; code, you should carefully study the contents of the /usr/src/UPDATING file on your system or its latest version at . (This is vital information if you are upgrading from one version to another—especially if you are upgrading to the &os.current; branch). However, if the problem is in something that was installed as a part of the &os; Ports Collection, you should refer to /usr/ports/UPDATING (for individual ports) or /usr/ports/CHANGES (for changes that affect the entire Ports Collection). and are also available via CVSweb.
Writing the problem report Now that you have decided that your issue merits a problem report, and that it is a &os; problem, it is time to write the actual problem report. Before we get into the mechanics of the program used to generate and submit PRs, here are some tips and tricks to help make sure that your PR will be most effective.
Tips and tricks for writing a good problem report Do not leave the Synopsis line empty. The PRs go both onto a mailing list that goes all over the world (where the Synopsis is used for the Subject: line), but also into a database. Anyone who comes along later and browses the database by synopsis, and finds a PR with a blank subject line, tends just to skip over it. Remember that PRs stay in this database until they are closed by someone; an anonymous one will usually just disappear in the noise. Avoid using a weak Synopsis line. You should not assume that anyone reading your PR has any context for your submission, so the more you provide, the better. For instance, what part of the system does the problem apply to? Do you only see the problem while installing, or while running? To illustrate, instead of Synopsis: portupgrade is broken, see how much more informative this seems: Synopsis: port ports-mgmt/portupgrade coredumps on -current. (In the case of ports, it is especially helpful to have both the category and portname in the Synopsis line.) If you have a patch, say so. A PR with a patch included is much more likely to be looked at than one without. If you are including one, put the string [patch] (including the brackets) at the beginning of the Synopsis. (Although it is not mandatory to use that exact string, by convention, that is the one that is used.) If you are a maintainer, say so. If you are maintaining a part of the source code (for instance, a port), you might consider adding the string [maintainer update] (including the brackets) at the beginning of your synopsis line, and you definitely should set the Class of your PR to maintainer-update. This way any committer that handles your PR will not have to check. Be specific. The more information you supply about what problem you are having, the better your chance of getting a response. Include the version of &os; you are running (there is a place to put that, see below) and on which architecture. You should include whether you are running from a release (e.g. from a CDROM or download), or from a system maintained by &man.cvsup.1; (and, if so, how recently you updated). If you are tracking the &os.current; branch, that is the very first thing someone will ask, because fixes (especially for high-profile problems) tend to get committed very quickly, and &os.current; users are expected to keep up. Include which global options you have specified in your make.conf. Note: specifying -O2 and above to &man.gcc.1; is known to be buggy in many situations. While the &os; developers will accept patches, they are generally unwilling to investigate such issues due to simple lack of time and volunteers, and may instead respond that this just is not supported. If the problem can be reproduced easily, include information that will help a developer to reproduce it themselves. If a problem can be demonstrated with specific input then include an example of that input if possible, and include both the actual and the expected output. If this data is large or cannot be made public, - then do try to create a minimal file that exibits the + then do try to create a minimal file that exhibits the same issue and that can be included within the PR. If this is a kernel problem, then be prepared to supply the following information. (You do not have to include these by default, which only tends to fill up the database, but you should include excerpts that you think might be relevant): your kernel configuration (including which hardware devices you have installed) whether or not you have debugging options enabled (such as WITNESS), and if so, whether the problem persists when you change the sense of that option the full text of any backtrace, panic or other console output, or entries in /var/log/messages, if any were generated the output of pciconf -l and relevant parts of your dmesg output if your problem relates to a specific piece of hardware the fact that you have read src/UPDATING and that your problem is not listed there (someone is guaranteed to ask) whether or not you can run any other kernel as a fallback (this is to rule out hardware-related issues such as failing disks and overheating CPUs, which can masquerade as kernel problems) If this is a ports problem, then be prepared to supply the following information. (You do not have to include these by default, which only tends to fill up the database, but you should include excerpts that you think might be relevant): which ports you have installed any environment variables that override the defaults in bsd.port.mk, such as PORTSDIR the fact that you have read ports/UPDATING and that your problem is not listed there (someone is guaranteed to ask) Avoid vague requests for features. PRs of the form someone should really implement something that does so-and-so are less likely to get results than very specific requests. Remember, the source is available to everyone, so if you want a feature, the best way to ensure it being included is to get to work! Also consider the fact that many things like this would make a better topic for discussion on freebsd-questions than an entry in the PR database, as discussed above. Make sure no one else has already submitted a similar PR. Although this has already been mentioned above, it bears repeating here. It only take a minute or two to use the web-based search engine at . (Of course, everyone is guilty of forgetting to do this now and then.) Report only one issue per Problem Report. Avoid including two or more problems within the same report unless they are related. When submitting patches, avoid adding multiple features or fixing multiple bugs in the same PR unless they are closely related—such PRs often take longer to resolve. Avoid controversial requests. If your PR addresses an area that has been controversial in the past, you should probably be prepared to not only offer patches, but also justification for why the patches are The Right Thing To Do. As noted above, a careful search of the mailing lists using the archives at is always good preparation. Be polite. Almost anyone who would potentially work on your PR is a volunteer. No one likes to be told that they have to do something when they are already doing it for some motivation other than monetary gain. This is a good thing to keep in mind at all times on Open Source projects.
Before you begin If you are using the &man.send-pr.1; program, make sure your VISUAL (or EDITOR if VISUAL is not set) environment variable is set to something sensible. You should also make sure that mail delivery works fine. &man.send-pr.1; uses mail messages for the submission and tracking of problem reports. If you cannot post mail messages from the machine you are running &man.send-pr.1; on, your problem report will not reach the GNATS database. For details on the setup of mail on &os;, see the Electronic Mail chapter of the &os; Handbook at . Make sure that your mailer will not mangle the message on its way to GNATS. In particular, if your mailer automatically breaks lines, changes tabs to spaces, or escapes newline characters, any patch that you submit will be rendered unusable. For the text sections, however, we request that you insert manual linebreaks somewhere around 70 characters, so that the web display of the PR will be readable. Similar considerations apply if you are using the web-based PR submission form instead of &man.send-pr.1;. Note that cut-and-paste operations can have their own side-effects on text formatting. In certain cases it may be necessary to use &man.uuencode.1; to ensure that patches arrive unmodified. Finally, if your submission will be lengthy, you should to prepare your work offline so that nothing will be lost in case there is a problem submitting it. This can especially be a problem with the web form.
Attaching patches or files The following applies to submitting PRs via email: The &man.send-pr.1; program has provisions for attaching files to a problem report. You can attach as many files as you want provided that each has a unique base name (i.e. the name of the file proper, without the path). Just use the command-line option to specify the names of the files you wish to attach: &prompt.user; send-pr -a /var/run/dmesg -a /tmp/errors Do not worry about binary files, they will be automatically encoded so as not to upset your mail agent. If you attach a patch, make sure you use the or option to &man.diff.1; to create a context or unified diff (unified is preferred), and make sure to specify the exact CVS revision numbers of the files you modified so the developers who read your report will be able to apply them easily. For problems with the kernel or the base utilities, a patch against &os.current; (the HEAD CVS branch) is preferred since all new code should be applied and tested there first. After appropriate or substantial testing has been done, the code will be merged/migrated to the &os.stable; branch. If you attach a patch inline, instead of as an attachment, note that the most common problem by far is the tendency of some email programs to render tabs as spaces, which will completely ruin anything intended to be part of a Makefile. Do not send patches as attachments using Content-Transfer-Encoding: quoted-printable. These will perform character escaping and the entire patch will be useless. Also note that while including small patches in a PR is generally all right—particularly when they fix the problem described in the PR—large patches and especially new code which may require substantial review before committing should be placed on a web or ftp server, and the URL should be included in the PR instead of the patch. Patches in email tend to get mangled, especially when GNATS is involved, and the larger the patch, the harder it will be for interested parties to unmangle it. Also, posting a patch on the web allows you to modify it without having to resubmit the entire patch in a followup to the original PR. Finally, large patches simply increase the size of the database, since closed PRs are not actually deleted but instead kept and simply marked as closed. You should also take note that unless you explicitly specify otherwise in your PR or in the patch itself, any patches you submit will be assumed to be licensed under the same terms as the original file you modified.
Filling out the template The next section applies to the email method only: When you run &man.send-pr.1;, you are presented with a template. The template consists of a list of fields, some of which are pre-filled, and some of which have comments explaining their purpose or listing acceptable values. Do not worry about the comments; they will be removed automatically if you do not modify them or remove them yourself. At the top of the template, below the SEND-PR: lines, are the email headers. You do not normally need to modify these, unless you are sending the problem report from a machine or account that can send but not receive mail, in which case you will want to set the From: and Reply-To: to your real email address. You may also want to send yourself (or someone else) a carbon copy of the problem report by adding one or more email addresses to the Cc: header. In the email template you will find the following two single-line fields: Submitter-Id: Do not change this. The default value of current-users is correct, even if you run &os.stable;. Confidential: This is prefilled to no. Changing it makes no sense as there is no such thing as a confidential &os; problem report—the PR database is distributed worldwide by CVSup. The next section describes fields that are common to both the email interface and the web interface: Originator: Please specify your real name, optionally followed by your email address in angle brackets. In the email interface, this is normally prefilled with the gecos field of the currently logged-in user. The email address you use will become public information and may become available to spammers. You should either have spam handling procedures in place, or use a temporary email account. However, please note that if you do not use a valid email account at all, we will not be able to ask you questions about your PR. Organization: Whatever you feel like. This field is not used for anything significant. Synopsis: Fill this out with a short and accurate description of the problem. The synopsis is used as the subject of the problem report email, and is used in problem report listings and summaries; problem reports with obscure synopses tend to get ignored. As noted above, if your problem report includes a patch, please have the synopsis start with [patch] (including the brackets); if this is a ports PR and you are the maintainer, you may consider adding [maintainer update] (including the brackets) and set the Class of your PR to maintainer-update. Severity: One of non-critical, serious or critical. Do not overreact; refrain from labeling your problem critical unless it really is (e.g. data corruption issues, serious regression from previous functionality in -CURRENT) or serious unless it is something that will affect many users (kernel panics or freezes; problems with particular device drivers or system utilities). &os; developers will not necessarily work on your problem faster if you inflate its importance since there are so many other people who have done exactly that — in fact, some developers pay little attention to this field because of this. Major security problems should not be filed in GNATS, because all GNATS information is public knowledge. Please send such problems in private email to &a.security-officer;. Priority: One of low, medium or high. high should be reserved for problems that will affect practically every user of &os; and medium for something that will affect many users. This field has become so widely abused that it is almost completely meaningless. Category: Choose an appropriate category. The first thing you need to do is to decide what part of the system your problem lies in. Remember, &os; is a complete operating system, which installs both a kernel, the standard libraries, many peripheral drivers, and a large number of utilities (the base system). However, there are thousands of additional applications in the Ports Collection. You'll first need to decide if the problem is in the base system or something installed via the Ports Collection. Here is a description of the major categories: If a problem is with the kernel, the libraries (such as standard C library libc), or a peripheral driver in the base system, in general you will use the kern category. (There are a few exceptions; see below). In general these are things that are described in section 2, 3, or 4 of the manual pages. If a problem is with a binary program such as &man.sh.1; or &man.mount.8;, you will first need to determine whether these programs are in the base system or were added via the Ports Collection. If you are unsure, you can do whereis programname. &os;'s convention for the Ports Collection is to install everything underneath /usr/local, although this can be overridden by a system administrator. For these, you will use the ports category (yes, even if the port's category is www; see below). If the location is /bin, /usr/bin, /sbin, or /usr/sbin, it is part of the base system, and you should use the bin category. (A few programs, such as &man.gcc.1;, actually use the gnu category, but do not worry about that for now.) These are all things that are described in section 1 or 8 of the manual pages. If you believe that the error is in the startup (rc) scripts, or in some kind of other non-executable configuration file, then the right category is conf (configuration). These are things that are described in section 5 of the manual pages. If you have found a problem in the documentation set (articles, books, man pages), the correct choice is docs. If you are having a problem with the FreeBSD web pages, the proper choice is www. if you are having a problem with something from a port named www/someportname, this nevertheless goes in the ports category. There are a few more specialized categories. If the problem would otherwise be filed in kern but has to do with the USB subsystem, the correct choice is usb. If the problem would otherwise be filed in kern but has to do with the threading libraries, the correct choice is threads. If the problem would otherwise be in the base system, but has to do with our adherence to standards such as &posix;, the correct choice is standards. If the problem has to do with errors internal to a &java.virtual.machine; (&jvm;), even though &java; was installed from the Ports Collection, you should select the java category. More general problems with &java; ports still go under ports. This leaves everything else. If you are convinced that the problem will only occur under the processor architecture you are using, select one of the architecture-specific categories: commonly i386 for Intel-compatible machines in 32-bit mode; amd64 for AMD machines running in 64-bit mode (this also includes Intel-compatible machines running in EMT64 mode); and less commonly arm, ia64, powerpc, and sparc64. These categories are quite often misused for I do not know problems. Rather than guessing, please just use misc. Correct use of arch-specific category You have a common PC-based machine, and think you have encountered a problem specific to a particular chipset or a particular motherboard: i386 is the right category. Incorrect use of arch-specific category You are having a problem with an add-in peripheral card on a commonly seen bus, or a problem with a particular type of hard disk drive: in this case, it probably applies to more than one architecture, and kern is the right category. If you really do not know where the problem lies (or the explanation does not seem to fit into the ones above), use the misc category. Before you do so, you may wish to ask for help on the &a.questions; first. You may be advised that one of the existing categories really is a better choice. Here is the current list of categories (taken from ): advocacy: problems relating to &os;'s public image. Obsolete. alpha: problems specific to the Alpha platform. amd64: problems specific to the AMD64 platform. arm: problems specific to the ARM platform. bin: problems with userland programs in the base system. conf: problems with configuration files, default values, and so forth. docs: problems with manual pages or on-line documentation. gnu: problems with imported GNU software such as &man.gcc.1; or &man.grep.1;. i386: problems specific to the &i386; platform. ia64: problems specific to the ia64 platform. java: problems related to the &java; Virtual Machine. kern: problems with the kernel, (non-platform-specific) device drivers, or the base libraries. misc: anything that does not fit in any of the other categories. (Note that there is almost nothing that truly belongs in this category, except for problems with the release and build infrastructure. Temporary build failures on HEAD do not belong here. Also note that it is easy for things to get lost in this category). ports: problems relating to the Ports Collection. powerpc: problems specific to the &powerpc; platform. sparc64: problems specific to the &sparc64; platform. standards: Standards conformance issues. threads: problems related to the &os; threads implementation (especially on &os.current;). usb: problems related to the &os; USB implementation. www: Changes or enhancements to the &os; website. Class: Choose one of the following: sw-bug: software bugs. doc-bug: errors in documentation. change-request: requests for additional features or changes in existing features. update: updates to ports or other contributed software. maintainer-update: updates to ports for which you are the maintainer. Release: The version of &os; that you are running. This is filled out automatically if you are using &man.send-pr.1; and need only be changed if you are sending a problem report from a different system than the one that exhibits the problem. Finally, there is a series of multi-line fields: Environment: This should describe, as accurately as possible, the environment in which the problem has been observed. This includes the operating system version, the version of the specific program or file that contains the problem, and any other relevant items such as system configuration, other installed software that influences the problem, etc.—quite simply everything a developer needs to know to reconstruct the environment in which the problem occurs. Description: A complete and accurate description of the problem you are experiencing. Try to avoid speculating about the causes of the problem unless you are certain that you are on the right track, as it may mislead a developer into making incorrect assumptions about the problem. How-To-Repeat: A summary of the actions you need to take to reproduce the problem. Fix: Preferably a patch, or at least a workaround (which not only helps other people with the same problem work around it, but may also help a developer understand the cause for the problem), but if you do not have any firm ideas for either, it is better to leave this field blank than to speculate.
Sending off the problem report If you are using &man.send-pr.1;: Once you are done filling out the template, have saved it, and exit your editor, &man.send-pr.1; will prompt you with s)end, e)dit or a)bort?. You can then hit s to go ahead and submit the problem report, e to restart the editor and make further modifications, or a to abort. If you choose the latter, your problem report will remain on disk (&man.send-pr.1; will tell you the filename before it terminates), so you can edit it at your leisure, or maybe transfer it to a system with better net connectivity, before sending it with the to &man.send-pr.1;: &prompt.user; send-pr -f ~/my-problem-report This will read the specified file, validate the contents, strip comments and send it off. If you are using the web form: Before you hit submit, you will need to fill in a field containing text that is represented in image form on the page. This unfortunate measure has had to be adopted due to misuse by automated systems and a few misguided individuals. It is a necessary evil that no one likes; please do not ask us to remove it. Note that you are strongly advised to save your work somewhere before hitting submit. A common problem for users is to have their web browser displaying a stale image from its cache. If this happens to you, your submission will be rejected and you may lose your work. If you are unable to view images for any reason, and are also unable to use &man.send-pr.1;, please accept our apologies for the inconvenience and email your problem report to the bugbuster team at freebsd-bugbusters@FreeBSD.org.
Follow-up Once your problem report has been filed, you will receive a confirmation by email which will include the tracking number that was assigned to your problem report and a URL you can use to check its status. With a little luck, someone will take an interest in your problem and try to address it, or, as the case may be, explain why it is not a problem. You will be automatically notified of any change of status, and you will receive copies of any comments or patches someone may attach to your problem report's audit trail. If someone requests additional information from you, or you remember or discover something you did not mention in the initial report, please use one of two methods to submit your followup: The easiest way is to use the followup link on the individual PR's web page, which you can reach from the PR search page. Clicking on this link will bring up an an email window with the correct To: and Subject: lines filled in (if your browser is configured to do this). Alternatively, you can just mail it to &a.bugfollowup;, making sure that the tracking number is included in the subject so the bug tracking system will know what problem report to attach it to. If you do not include the tracking number, GNATS will become confused and create an entirely new PR which it then assigns to the GNATS administrator, and then your followup will become lost until someone comes in to clean up the mess, which could be days or weeks afterwards. Wrong way: Subject: that PR I sent Right way: Subject: Re: ports/12345: compilation problem with foo/bar If the problem report remains open after the problem has gone away, just send a follow-up (in the manner prescribed above) saying that the problem report can be closed, and, if possible, explaining how or when the problem was fixed.
If you are having problems Most PRs go through the system and are accepted quickly; however, at times GNATS runs behind and you may not get your email confirmation for 10 minutes or even longer. Please try to be patient. In addition, because GNATS receives all its input via email, it is absolutely vital that &os; runs all its submissions through spam filters. If you do not get a response within an hour or two, you may have fallen afoul of them; if so, please contact the GNATS administrators at bugmeister@FreeBSD.org and ask for help. Among the anti-spam measures is one that weighs against many common abuses seen in HTML-based email (although not necessarily the mere inclusion of HTML in a PR). We strongly recommend against the use of HTML-based email when sending PRs: not only is it more likely to fall afoul of the filters, it also tends to merely clutter up the database. Plain old email is strongly preferred. On rare occasions you will encounter a GNATS bug where a PR is accepted and assigned a tracking number but it does not show up on the list of PRs on any of the web query pages. What may have happened is that the database index has gotten out of synchronization with the database itself. The way that you can test whether this has happened is to pull up the view a single PR page and see whether the PR shows up. If it does, please notify the GNATS administrators at bugmeister@FreeBSD.org. Note that there is a cron job that periodically rebuilds the database, so unless you are in a hurry, no action needs to be taken.
Further Reading This is a list of resources relevant to the proper writing and processing of problem reports. It is by no means complete. How to Report Bugs Effectively—an excellent essay by Simon G. Tatham on composing useful (non-&os;-specific) problem reports. Problem Report Handling Guidelines—valuable insight into how problem reports are handled by the &os; developers.
diff --git a/en_US.ISO8859-1/articles/releng-packages/article.sgml b/en_US.ISO8859-1/articles/releng-packages/article.sgml index bc07a2a2ea..22d9402511 100644 --- a/en_US.ISO8859-1/articles/releng-packages/article.sgml +++ b/en_US.ISO8859-1/articles/releng-packages/article.sgml @@ -1,367 +1,367 @@ %articles.ent; ]>
FreeBSD Release Engineering for Third Party Software Packages Steve Price
steve@FreeBSD.org
 $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.intel; &tm-attrib.xfree86; &tm-attrib.general; This paper describes the approach used by the FreeBSD ports management team to produce a high quality package set suitable for official FreeBSD release media. This document is a work in progress, but eventually it will cover the process used to build a clean package set on the FreeBSD.org Ports Cluster, how to configure any other set of machines as a ports cluster, how to split up the packages for the release media, and how to verify that a package set is consistent. Building packages from the Ports Collection The FreeBSD Ports collection is a collection of over &os.numports; third-party software packages available for FreeBSD. The &a.portmgr; is responsible for maintaining a consistent ports tree that can be used to create the binary packages that accompany a given FreeBSD release. The Ports Cluster In order to provide a consistent set of third-party packages for FreeBSD releases, every port is built in a separate chroot environment, starting with an empty /usr/local and /usr/X11R6. The requisite dependencies are installed as packages before the build proceeds. This enforces consistency in the package build process. By starting the package build in a pristine environment, we can assure that the package metadata (such as required dependencies) is accurate. This way, we will never generate packages that might work on some systems and not on others depending on what software was previously installed. The Ports Cluster for the x86 architecture currently consists of a master node (Dual &pentium; III 733MHz) and 8 slave nodes (&pentium; III 800MHz) to do the actual package builds. With this configuration, a complete package build takes over 24 hours. These machines are co-located with the other FreeBSD Project equipment at Yahoo's corner of Exodus in Santa Clara, CA. The Ports Cluster for the Alpha architecture consists of 7 PWS 500A machines donated by Compaq and also co-located with Yahoo's facilities. The Package Split For FreeBSD 4.4 over 4.1 gigabytes of packages were created. This causes a problem for CDROM distributions because we would like to ship as many packages as possible without making the user insert another disc to satisfy dependencies. The solution is to create clusters of like packages with similar dependencies and group these onto specific discs. This section describes the software and methodology used to create those package sets for the official FreeBSD release discs. The scripts and other files needed to produce a package split can be found in the CVS tree in ports/Tools/scripts/release. Copy this directory to a machine that has enough free disk space to hold 2 to 3 times the size of the package set that you wish to split. The following scripts are present in this directory: config This file contains the free space on each disc and whether packages, distfiles, or both are allowed on any given disc. The first column is the disc name. It must be of the form disc[0-9a-z]. Currently it is set up to allow for 10 discs (4 for the release set and 6 for the toolkit). There is an implied extra disc called scratch where all of the remaining distfiles/packages land if they do not fit elsewhere. The second column can be either a 1 or 0, where 1 says that it is okay to place packages on this disc. The third column works the same way, but it controls whether distfiles are placed on this disc. The last column denotes the number of bytes of free space on a disc. doit.sh This is the workhorse. Once you have all the files in place and things properly configured this script directs the process of splitting packages. Beware it is interactive so you need to keep an eye on it as it runs. More details on what happens in this script will follow. checkdeps.pl Makes sure all packages dependencies are satisfied given an INDEX file and a directory of packages. oneshot.pl This is where all the magic (and I use that term loosely as it is mostly just a brute force approach) happens. Given a list of required packages for each disc and a set of packages/distfiles this is the script that places a package or distfile on a disc along with all of its dependencies. print-cdrom-packages.sh This file is a copy of src/release/scripts/print-cdrom-packages.sh from the release you are working on. scrubindex.pl This script removes lines from an INDEX file for packages that are not present. It also removes the &xfree86; dependencies. NOTE: you will need to tweak the value of the xdep variable to make sure the version number is correct. setup.sh This is a helper script that I use on the ports building cluster to grab a copy of the ports tree and the matching set of the packages/distfiles. Here is a checklist of things you will need to check or configure before going any further. Edit config to denote the number of discs you have, their sizes, and whether you want them want to contain packages, distfiles, both, or neither. Make sure you remove the gen directory if there is an old one laying around. This directory contains working files that will only be valid for the current split. On your first pass through a split it is best to fake the copying of packages and distfiles. This will save - both time and diskspace while you do a couple of trial runs to + both time and disk space while you do a couple of trial runs to make sure things fit, etc. In the oneshot.pl set the fake variable to 1 and instead of actually copying the files it will &man.touch.1; them. Be sure you turn this off or set fake to 0 before you give the resultant discs to the person that will be mastering the discs otherwise they will get a directory full of zero-sized files. Make sure you have a recent copy of the print-cdrom-packages.sh and that it is from the correct release. Check to make sure the &xfree86; dependency in scrubindex.pl has the correct version number. You will also need to make sure this value is correct in doit.sh as well. Next you will need to get a copy of the ports tree, packages, and distfiles from a recent build on the package cluster. See the setup.sh for a working example but essentially here is what needs to be done. Grab a copy of ports.tar.gz and extract it into the ports directory alongside doit.sh and the scripts directory. Remove the packages/distfiles directories or symlinks. Bento has these as symlinks and you will have mixed results if you do not get rid of them before proceeding. Create a new ports/packages directory and copy the package set from the package building cluster. Create a new ports/distfiles directory and copy the distfiles from the package building cluster. NOTE: if you do not want any distfiles simply create the directory and leave it empty. This directory must be present even if it does not contain anything. Now we are finally ready for the fun task of actually splitting the packages. You start the processing by running ./doit.sh. Here is what it does the first time you run it. Create a list of the restricted (can not be on the master FTP site) ports. Asks you if you would like to remove the restricted ports. Most of the time you will want to answer (y)es here. Create a list of the packages/distfiles that can not be put on the discs. Asks you if you would like to remove the non-cdromable packages/distfiles. Most of the time you will want to answer (y)es here. Copies the INDEX from the ports directory to the gen directory. In doing so it removes the lines for ports where the packages do not exist. It also checks to make sure that all of the required dependency packages are present. Create a list of packages that are required on each disc. Asks you if you would like to populate the discs. After populating each disc it will check for missing dependencies, scrub the INDEX file, and create the CHECKSUM.MD5 file. Check to make sure the required packages made it on each disc and gives you a summary of the sizes of each disc. After going through this the first time if you are lucky enough that all of the required packages built and fit on each disc. All you need to do is set fake to 0 in oneshot.pl and re-run ./doit.sh. The second and subsequent times around it will skip steps 1-5 above. If you want to re-run any of those steps refer to doit.sh for which files need to be removed to not short-circuit those steps. If you want to repeat all of these steps then the easiest way is to rm -rf gen. Upon successful completion the packages/distfiles will be in the disc* directories and the leftover will be in the scratch directory. What to do if things go wrong? Here is some common gotchas and workarounds. Missing required packages This is a pretty common occurrence. You will either need to wait for a new set of packages where the missing packages were built or get someone to re-start the package build for you. Do not attempt to build the missing packages on your own machine and add them into the fray. While you might be able to get away with this if you are extremely careful the vast majority of the time you will miss some little detail and the simple process of adding a package could make hundreds of others come up mysteriously broken. Required packages will not fit This happens on occasion too and is relatively easy to fix. Simply edit print-cdrom-packages.sh to move packages around until they fit. Yes this is an iterative process and one of the reasons why you should enable fake in oneshot.pl until you have gotten things the way you want them. Re-run ./doit.sh after you made your adjustments. Required packages not on the right (or any) disc This usually means you did not add them to print-cdrom-packages.sh or you put them on the wrong disc. This script is the gospel by which this whole process determines where a package must be. If you want to force a package to land on a particular disc this is the only way to ensure that it will happen. If you get completely stuck and can not figure out why things are borked or how to fix them then email &a.steve; for assistance.
diff --git a/en_US.ISO8859-1/articles/serial-uart/article.sgml b/en_US.ISO8859-1/articles/serial-uart/article.sgml index 8a896aad2a..0ac993f1fe 100644 --- a/en_US.ISO8859-1/articles/serial-uart/article.sgml +++ b/en_US.ISO8859-1/articles/serial-uart/article.sgml @@ -1,2439 +1,2439 @@ %articles.ent; ]>
Serial and UART Tutorial Frank Durda
uhclem@FreeBSD.org
 $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.microsoft; &tm-attrib.general; This article talks about using serial hardware with FreeBSD. The UART: What it is and how it works Copyright © 1996 &a.uhclem;, All Rights Reserved. 13 January 1996. The Universal Asynchronous Receiver/Transmitter (UART) controller is the key component of the serial communications subsystem of a computer. The UART takes bytes of data and transmits the individual bits in a sequential fashion. At the destination, a second UART re-assembles the bits into complete bytes. Serial transmission is commonly used with modems and for non-networked communication between computers, terminals and other devices. There are two primary forms of serial transmission: Synchronous and Asynchronous. Depending on the modes that are supported by the hardware, the name of the communication sub-system will usually include a A if it supports Asynchronous communications, and a S if it supports Synchronous communications. Both forms are described below. Some common acronyms are:
UART Universal Asynchronous Receiver/Transmitter
USART Universal Synchronous-Asynchronous Receiver/Transmitter
Synchronous Serial Transmission Synchronous serial transmission requires that the sender and receiver share a clock with one another, or that the sender provide a strobe or other timing signal so that the receiver knows when to read the next bit of the data. In most forms of serial Synchronous communication, if there is no data available at a given instant to transmit, a fill character must be sent instead so that data is always being transmitted. Synchronous communication is usually more efficient because only data bits are transmitted between sender and receiver, and synchronous communication can be more costly if extra wiring and circuits are required to share a clock signal between the sender and receiver. A form of Synchronous transmission is used with printers and fixed disk devices in that the data is sent on one set of wires while a clock or strobe is sent on a different wire. Printers and fixed disk devices are not normally serial devices because most fixed disk interface standards send an entire word of data for each clock or strobe signal by using a separate wire for each bit of the word. In the PC industry, these are known as Parallel devices. The standard serial communications hardware in the PC does not support Synchronous operations. This mode is described here for comparison purposes only. Asynchronous Serial Transmission Asynchronous transmission allows data to be transmitted without the sender having to send a clock signal to the receiver. Instead, the sender and receiver must agree on timing parameters in advance and special bits are added to each word which are used to synchronize the sending and receiving units. When a word is given to the UART for Asynchronous transmissions, a bit called the "Start Bit" is added to the beginning of each word that is to be transmitted. The Start Bit is used to alert the receiver that a word of data is about to be sent, and to force the clock in the receiver into synchronization with the clock in the transmitter. These two clocks must be accurate enough to not have the frequency drift by more than 10% during the transmission of the remaining bits in the word. (This requirement was set in the days of mechanical teleprinters and is easily met by modern electronic equipment.) After the Start Bit, the individual bits of the word of data are sent, with the Least Significant Bit (LSB) being sent first. Each bit in the transmission is transmitted for exactly the same amount of time as all of the other bits, and the receiver looks at the wire at approximately halfway through the period assigned to each bit to determine if the bit is a 1 or a 0. For example, if it takes two seconds to send each bit, the receiver will examine the signal to determine if it is a 1 or a 0 after one second has passed, then it will wait two seconds and then examine the value of the next bit, and so on. The sender does not know when the receiver has looked at the value of the bit. The sender only knows when the clock says to begin transmitting the next bit of the word. When the entire data word has been sent, the transmitter may add a Parity Bit that the transmitter generates. The Parity Bit may be used by the receiver to perform simple error checking. Then at least one Stop Bit is sent by the transmitter. When the receiver has received all of the bits in the data word, it may check for the Parity Bits (both sender and receiver must agree on whether a Parity Bit is to be used), and then the receiver looks for a Stop Bit. If the Stop Bit does not appear when it is supposed to, the UART considers the entire word to be garbled and will report a Framing Error to the host processor when the data word is read. The usual cause of a Framing Error is that the sender and receiver clocks were not running at the same speed, or that the signal was interrupted. Regardless of whether the data was received correctly or not, the UART automatically discards the Start, Parity and Stop bits. If the sender and receiver are configured identically, these bits are not passed to the host. If another word is ready for transmission, the Start Bit for the new word can be sent as soon as the Stop Bit for the previous word has been sent. Because asynchronous data is self synchronizing, if there is no data to transmit, the transmission line can be idle. Other UART Functions In addition to the basic job of converting data from parallel to serial for transmission and from serial to parallel on reception, a UART will usually provide additional circuits for signals that can be used to indicate the state of the transmission media, and to regulate the flow of data in the event that the remote device is not prepared to accept more data. For example, when the device connected to the UART is a modem, the modem may report the presence of a carrier on the phone line while the computer may be able to instruct the modem to reset itself or to not take calls by raising or lowering one more of these extra signals. The function of each of these additional signals is defined in the EIA RS232-C standard. The RS232-C and V.24 Standards In most computer systems, the UART is connected to circuitry that generates signals that comply with the EIA RS232-C specification. There is also a CCITT standard named V.24 that mirrors the specifications included in RS232-C. RS232-C Bit Assignments (Marks and Spaces) In RS232-C, a value of 1 is called a Mark and a value of 0 is called a Space. When a communication line is idle, the line is said to be Marking, or transmitting continuous 1 values. The Start bit always has a value of 0 (a Space). The Stop Bit always has a value of 1 (a Mark). This means that there will always be a Mark (1) to Space (0) transition on the line at the start of every word, even when multiple word are transmitted back to back. This guarantees that sender and receiver can resynchronize their clocks regardless of the content of the data bits that are being transmitted. The idle time between Stop and Start bits does not have to be an exact multiple (including zero) of the bit rate of the communication link, but most UARTs are designed this way for simplicity. In RS232-C, the "Marking" signal (a 1) is represented by a voltage between -2 VDC and -12 VDC, and a "Spacing" signal (a 0) is represented by a voltage between 0 and +12 VDC. The transmitter is supposed to send +12 VDC or -12 VDC, and the receiver is supposed to allow for some voltage loss in long cables. Some transmitters in low power devices (like portable computers) sometimes use only +5 VDC and -5 VDC, but these values are still acceptable to a RS232-C receiver, provided that the cable lengths are short. RS232-C Break Signal RS232-C also specifies a signal called a Break, which is caused by sending continuous Spacing values (no Start or Stop bits). When there is no electricity present on the data circuit, the line is considered to be sending Break. The Break signal must be of a duration longer than the time it takes to send a complete byte plus Start, Stop and Parity bits. Most UARTs can distinguish between a Framing Error and a Break, but if the UART cannot do this, the Framing Error detection can be used to identify Breaks. In the days of teleprinters, when numerous printers around the country were wired in series (such as news services), any unit could cause a Break by temporarily opening the entire circuit so that no current flowed. This was used to allow a location with urgent news to interrupt some other location that was currently sending information. In modern systems there are two types of Break signals. If the Break is longer than 1.6 seconds, it is considered a "Modem Break", and some modems can be programmed to terminate the conversation and go on-hook or enter the modems' command mode when the modem detects this signal. If the Break is smaller than 1.6 seconds, it signifies a Data Break and it is up to the remote computer to respond to this signal. Sometimes this form of Break is used as an Attention or Interrupt signal and sometimes is accepted as a substitute for the ASCII CONTROL-C character. Marks and Spaces are also equivalent to Holes and No Holes in paper tape systems. Breaks cannot be generated from paper tape or from any other byte value, since bytes are always sent with Start and Stop bit. The UART is usually capable of generating the continuous Spacing signal in response to a special command from the host processor. RS232-C DTE and DCE Devices The RS232-C specification defines two types of equipment: the Data Terminal Equipment (DTE) and the Data Carrier Equipment (DCE). Usually, the DTE device is the terminal (or computer), and the DCE is a modem. Across the phone line at the other end of a conversation, the receiving modem is also a DCE device and the computer that is connected to that modem is a DTE device. The DCE device receives signals on the pins that the DTE device transmits on, and vice versa. When two devices that are both DTE or both DCE must be connected together without a modem or a similar media - translater between them, a NULL modem must be used. The + translator between them, a NULL modem must be used. The NULL modem electrically re-arranges the cabling so that the transmitter output is connected to the receiver input on the other device, and vice versa. Similar translations are performed on all of the control signals so that each device will see what it thinks are DCE (or DTE) signals from the other device. The number of signals generated by the DTE and DCE devices are not symmetrical. The DTE device generates fewer signals for the DCE device than the DTE device receives from the DCE. RS232-C Pin Assignments The EIA RS232-C specification (and the ITU equivalent, V.24) calls for a twenty-five pin connector (usually a DB25) and defines the purpose of most of the pins in that connector. In the IBM Personal Computer and similar systems, a subset of RS232-C signals are provided via nine pin connectors (DB9). The signals that are not included on the PC connector deal mainly with synchronous operation, and this transmission mode is not supported by the UART that IBM selected for use in the IBM PC. Depending on the computer manufacturer, a DB25, a DB9, or both types of connector may be used for RS232-C communications. (The IBM PC also uses a DB25 connector for the parallel printer interface which causes some confusion.) Below is a table of the RS232-C signal assignments in the DB25 and DB9 connectors. DB25 RS232-C Pin DB9 IBM PC Pin EIA Circuit Symbol CCITT Circuit Symbol Common Name Signal Source Description 1 - AA 101 PG/FG - Frame/Protective Ground 2 3 BA 103 TD DTE Transmit Data 3 2 BB 104 RD DCE Receive Data 4 7 CA 105 RTS DTE Request to Send 5 8 CB 106 CTS DCE Clear to Send 6 6 CC 107 DSR DCE Data Set Ready 7 5 AV 102 SG/GND - Signal Ground 8 1 CF 109 DCD/CD DCE Data Carrier Detect 9 - - - - - Reserved for Test 10 - - - - - Reserved for Test 11 - - - - - Reserved for Test 12 - CI 122 SRLSD DCE Sec. Recv. Line Signal Detector 13 - SCB 121 SCTS DCE Secondary Clear to Send 14 - SBA 118 STD DTE Secondary Transmit Data 15 - DB 114 TSET DCE Trans. Sig. Element Timing 16 - SBB 119 SRD DCE Secondary Received Data 17 - DD 115 RSET DCE Receiver Signal Element Timing 18 - - 141 LOOP DTE Local Loopback 19 - SCA 120 SRS DTE Secondary Request to Send 20 4 CD 108.2 DTR DTE Data Terminal Ready 21 - - - RDL DTE Remote Digital Loopback 22 9 CE 125 RI DCE Ring Indicator 23 - CH 111 DSRS DTE Data Signal Rate Selector 24 - DA 113 TSET DTE Trans. Sig. Element Timing 25 - - 142 - DCE Test Mode Bits, Baud and Symbols Baud is a measurement of transmission speed in asynchronous communication. Because of advances in modem communication technology, this term is frequently misused when describing the data rates in newer devices. Traditionally, a Baud Rate represents the number of bits that are actually being sent over the media, not the amount of data that is actually moved from one DTE device to the other. The Baud count includes the overhead bits Start, Stop and Parity that are generated by the sending UART and removed by the receiving UART. This means that seven-bit words of data actually take 10 bits to be completely transmitted. Therefore, a modem capable of moving 300 bits per second from one place to another can normally only move 30 7-bit words if Parity is used and one Start and Stop bit are present. If 8-bit data words are used and Parity bits are also used, the data rate falls to 27.27 words per second, because it now takes 11 bits to send the eight-bit words, and the modem still only sends 300 bits per second. The formula for converting bytes per second into a baud rate and vice versa was simple until error-correcting modems came along. These modems receive the serial stream of bits from the UART in the host computer (even when internal modems are used the data is still frequently serialized) and converts the bits back into bytes. These bytes are then combined into packets and sent over the phone line using a Synchronous transmission method. This means that the Stop, Start, and Parity bits added by the UART in the DTE (the computer) were removed by the modem before transmission by the sending modem. When these bytes are received by the remote modem, the remote modem adds Start, Stop and Parity bits to the words, converts them to a serial format and then sends them to the receiving UART in the remote computer, who then strips the Start, Stop and Parity bits. The reason all these extra conversions are done is so that the two modems can perform error correction, which means that the receiving modem is able to ask the sending modem to resend a block of data that was not received with the correct checksum. This checking is handled by the modems, and the DTE devices are usually unaware that the process is occurring. By striping the Start, Stop and Parity bits, the additional bits of data that the two modems must share between themselves to perform error-correction are mostly concealed from the effective transmission rate seen by the sending and receiving DTE equipment. For example, if a modem sends ten 7-bit words to another modem without including the Start, Stop and Parity bits, the sending modem will be able to add 30 bits of its own information that the receiving modem can use to do error-correction without impacting the transmission speed of the real data. The use of the term Baud is further confused by modems that perform compression. A single 8-bit word passed over the telephone line might represent a dozen words that were transmitted to the sending modem. The receiving modem will expand the data back to its original content and pass that data to the receiving DTE. Modern modems also include buffers that allow the rate that bits move across the phone line (DCE to DCE) to be a different speed than the speed that the bits move between the DTE and DCE on both ends of the conversation. Normally the speed between the DTE and DCE is higher than the DCE to DCE speed because of the use of compression by the modems. Because the number of bits needed to describe a byte varied during the trip between the two machines plus the differing bits-per-seconds speeds that are used present on the DTE-DCE and DCE-DCE links, the usage of the term Baud to describe the overall communication speed causes problems and can misrepresent the true transmission speed. So Bits Per Second (bps) is the correct term to use to describe the transmission rate seen at the DCE to DCE interface and Baud or Bits Per Second are acceptable terms to use when a connection is made between two systems with a wired connection, or if a modem is in use that is not performing error-correction or compression. Modern high speed modems (2400, 9600, 14,400, and 19,200bps) in reality still operate at or below 2400 baud, or more accurately, 2400 Symbols per second. High speed modem are able to encode more bits of data into each Symbol using a technique called Constellation Stuffing, which is why the effective bits per second rate of the modem is higher, but the modem continues to operate within the limited audio bandwidth that the telephone system provides. Modems operating at 28,800 and higher speeds have variable Symbol rates, but the technique is the same. The IBM Personal Computer UART Starting with the original IBM Personal Computer, IBM selected the National Semiconductor INS8250 UART for use in the IBM PC Parallel/Serial Adapter. Subsequent generations of compatible computers from IBM and other vendors continued to use the INS8250 or improved versions of the National Semiconductor UART family. National Semiconductor UART Family Tree There have been several versions and subsequent generations of the INS8250 UART. Each major version is described below. INS8250 -> INS8250B \ \ \-> INS8250A -> INS82C50A \ \ \-> NS16450 -> NS16C450 \ \ \-> NS16550 -> NS16550A -> PC16550D INS8250 This part was used in the original IBM PC and IBM PC/XT. The original name for this part was the INS8250 ACE (Asynchronous Communications Element) and it is made from NMOS technology. The 8250 uses eight I/O ports and has a one-byte send and a one-byte receive buffer. This original UART has several race conditions and other flaws. The original IBM BIOS includes code to work around these flaws, but this made the BIOS dependent on the flaws being present, so subsequent parts like the 8250A, 16450 or 16550 could not be used in the original IBM PC or IBM PC/XT. INS8250-B This is the slower speed of the INS8250 made from NMOS technology. It contains the same problems as the original INS8250. INS8250A An improved version of the INS8250 using XMOS technology with various functional flaws corrected. The INS8250A was used initially in PC clone computers by vendors who used clean BIOS designs. Because of the corrections in the chip, this part could not be used with a BIOS compatible with the INS8250 or INS8250B. INS82C50A This is a CMOS version (low power consumption) of the INS8250A and has similar functional characteristics. NS16450 Same as NS8250A with improvements so it can be used with faster CPU bus designs. IBM used this part in the IBM AT and updated the IBM BIOS to no longer rely on the bugs in the INS8250. NS16C450 This is a CMOS version (low power consumption) of the NS16450. NS16550 Same as NS16450 with a 16-byte send and receive buffer but the buffer design was flawed and could not be reliably be used. NS16550A Same as NS16550 with the buffer flaws corrected. The 16550A and its successors have become the most popular UART design in the PC industry, mainly due to its ability to reliably handle higher data rates on operating systems with sluggish interrupt response times. NS16C552 This component consists of two NS16C550A CMOS UARTs in a single package. PC16550D Same as NS16550A with subtle flaws corrected. This is revision D of the 16550 family and is the latest design available from National Semiconductor. The NS16550AF and the PC16550D are the same thing National reorganized their part numbering system a few years ago, and the NS16550AFN no longer exists by that name. (If you have a NS16550AFN, look at the date code on the part, which is a four digit number that usually starts with a nine. The first two digits of the number are the year, and the last two digits are the week in that year when the part was packaged. If you have a NS16550AFN, it is probably a few years old.) The new numbers are like PC16550DV, with minor differences in the suffix letters depending on the package material and its shape. (A description of the numbering system can be found below.) It is important to understand that in some stores, you may pay $15(US) for a NS16550AFN made in 1990 and in the next bin are the new PC16550DN parts with minor fixes that National has made since the AFN part was in production, the PC16550DN was probably made in the past six months and it costs half (as low as $5(US) in volume) as much as the NS16550AFN because they are readily available. As the supply of NS16550AFN chips continues to shrink, the price will probably continue to increase until more people discover and accept that the PC16550DN really has the same function as the old part number. National Semiconductor Part Numbering System The older NSnnnnnrqp part numbers are now of the format PCnnnnnrgp. The r is the revision field. The current revision of the 16550 from National Semiconductor is D. The p is the package-type field. The types are: "F" QFP (quad flat pack) L lead type "N" DIP (dual inline package) through hole straight lead type "V" LPCC (lead plastic chip carrier) J lead type The g is the product grade field. If an I precedes the package-type letter, it indicates an industrial grade part, which has higher specs than a standard part but not as high as Military Specification (Milspec) component. This is an optional field. So what we used to call a NS16550AFN (DIP Package) is now called a PC16550DN or PC16550DIN. Other Vendors and Similar UARTs Over the years, the 8250, 8250A, 16450 and 16550 have been licensed or copied by other chip vendors. In the case of the 8250, 8250A and 16450, the exact circuit (the megacell) was licensed to many vendors, including Western Digital and Intel. Other vendors reverse-engineered the part or produced emulations that had similar behavior. In internal modems, the modem designer will frequently emulate the 8250A/16450 with the modem microprocessor, and the emulated UART will frequently have a hidden buffer consisting of several hundred bytes. Because of the size of the buffer, these emulations can be as reliable as a 16550A in their ability to handle high speed data. However, most operating systems will still report that the UART is only a 8250A or 16450, and may not make effective use of the extra buffering present in the emulated UART unless special drivers are used. Some modem makers are driven by market forces to abandon a design that has hundreds of bytes of buffer and instead use a 16550A UART so that the product will compare favorably in market comparisons even though the effective performance may be lowered by this action. A common misconception is that all parts with 16550A written on them are identical in performance. There are differences, and in some cases, outright flaws in most of these 16550A clones. When the NS16550 was developed, the National Semiconductor obtained several patents on the design and they also limited licensing, making it harder for other vendors to provide a chip with similar features. Because of the patents, reverse-engineered designs and emulations had to avoid infringing the claims covered by the patents. Subsequently, these copies almost never perform exactly the same as the NS16550A or PC16550D, which are the parts most computer and modem makers want to buy but are sometimes unwilling to pay the price required to get the genuine part. Some of the differences in the clone 16550A parts are unimportant, while others can prevent the device from being used at all with a given operating system or driver. These differences may show up when using other drivers, or when particular combinations of events occur that were not well tested or considered in the &windows; driver. This is because most modem vendors and 16550-clone makers use the Microsoft drivers from &windows; for Workgroups 3.11 and the µsoft; &ms-dos; utility as the primary tests for compatibility with the NS16550A. This over-simplistic criteria means that if a different operating system is used, problems could appear due to subtle differences between the clones and genuine components. National Semiconductor has made available a program named COMTEST that performs compatibility tests independent of any OS drivers. It should be remembered that the purpose of this type of program is to demonstrate the flaws in the products of the competition, so the program will report major as well as extremely subtle differences in behavior in the part being tested. In a series of tests performed by the author of this document in 1994, components made by National Semiconductor, TI, StarTech, and CMD as well as megacells and emulations embedded in internal modems were tested with COMTEST. A difference count for some of these components is listed below. Because these tests were performed in 1994, they may not reflect the current performance of the given product from a vendor. It should be noted that COMTEST normally aborts when an excessive number or certain types of problems have been detected. As part of this testing, COMTEST was modified so that it would not abort no matter how many differences were encountered. Vendor Part Number Errors (aka "differences" reported) National (PC16550DV) 0 National (NS16550AFN) 0 National (NS16C552V) 0 TI (TL16550AFN) 3 CMD (16C550PE) 19 StarTech (ST16C550J) 23 Rockwell Reference modem with internal 16550 or an emulation (RC144DPi/C3000-25) 117 Sierra Modem with an internal 16550 (SC11951/SC11351) 91 To date, the author of this document has not found any non-National parts that report zero differences using the COMTEST program. It should also be noted that National has had five versions of the 16550 over the years and the newest parts behave a bit differently than the classic NS16550AFN that is considered the benchmark for functionality. COMTEST appears to turn a blind eye to the differences within the National product line and reports no errors on the National parts (except for the original 16550) even when there are official erratas that describe bugs in the A, B and C revisions of the parts, so this bias in COMTEST must be taken into account. It is important to understand that a simple count of differences from COMTEST does not reveal a lot about what differences are important and which are not. For example, about half of the differences reported in the two modems listed above that have internal UARTs were caused by the clone UARTs not supporting five- and six-bit character modes. The real 16550, 16450, and 8250 UARTs all support these modes and COMTEST checks the functionality of these modes so over fifty differences are reported. However, almost no modern modem supports five- or six-bit characters, particularly those with error-correction and compression capabilities. This means that the differences related to five- and six-bit character modes can be discounted. Many of the differences COMTEST reports have to do with timing. In many of the clone designs, when the host reads from one port, the status bits in some other port may not update in the same amount of time (some faster, some slower) as a real NS16550AFN and COMTEST looks for these differences. This means that the number of differences can be misleading in that one device may only have one or two differences but they are extremely serious, and some other device that updates the status registers faster or slower than the reference part (that would probably never affect the operation of a properly written driver) could have dozens of differences reported. COMTEST can be used as a screening tool to alert the administrator to the presence of potentially incompatible components that might cause problems or have to be handled as a special case. If you run COMTEST on a 16550 that is in a modem or a modem is attached to the serial port, you need to first issue a ATE0&W command to the modem so that the modem will not echo any of the test characters. If you forget to do this, COMTEST will report at least this one difference: Error (6)...Timeout interrupt failed: IIR = c1 LSR = 61 8250/16450/16550 Registers The 8250/16450/16550 UART occupies eight contiguous I/O port addresses. In the IBM PC, there are two defined locations for these eight ports and they are known collectively as COM1 and COM2. The makers of PC-clones and add-on cards have created two additional areas known as COM3 and COM4, but these extra COM ports conflict with other hardware on some systems. The most common conflict is with video adapters that provide IBM 8514 emulation. COM1 is located from 0x3f8 to 0x3ff and normally uses IRQ 4. COM2 is located from 0x2f8 to 0x2ff and normally uses IRQ 3. COM3 is located from 0x3e8 to 0x3ef and has no standardized IRQ. COM4 is located from 0x2e8 to 0x2ef and has no standardized IRQ. A description of the I/O ports of the 8250/16450/16550 UART is provided below. I/O Port Access Allowed Description +0x00 write (DLAB==0) Transmit Holding Register (THR).Information written to this port are treated as data words and will be transmitted by the UART. +0x00 read (DLAB==0) Receive Buffer Register (RBR).Any data words received by the UART form the serial link are accessed by the host by reading this port. +0x00 write/read (DLAB==1) Divisor Latch LSB (DLL)This value will be divided from the master input clock (in the IBM PC, the master clock is 1.8432MHz) and the resulting clock will determine the baud rate of the UART. This register holds bits 0 thru 7 of the divisor. +0x01 write/read (DLAB==1) Divisor Latch MSB (DLH)This value will be divided from the master input clock (in the IBM PC, the master clock is 1.8432MHz) and the resulting clock will determine the baud rate of the UART. This register holds bits 8 thru 15 of the divisor. +0x01 write/read (DLAB==0) Interrupt Enable Register (IER)The 8250/16450/16550 UART classifies events into one of four categories. Each category can be configured to generate an interrupt when any of the events occurs. The 8250/16450/16550 UART generates a single external interrupt signal regardless of how many events in the enabled categories have occurred. It is up to the host processor to respond to the interrupt and then poll the enabled interrupt categories (usually all categories have interrupts enabled) to determine the true cause(s) of the interrupt. Bit 7 Reserved, always 0. Bit 6 Reserved, always 0. Bit 5 Reserved, always 0. Bit 4 Reserved, always 0. Bit 3 Enable Modem Status Interrupt (EDSSI). Setting this bit to "1" allows the UART to generate an interrupt when a change occurs on one or more of the status lines. Bit 2 Enable Receiver Line Status Interrupt (ELSI) Setting this bit to "1" causes the UART to generate an interrupt when the an error (or a BREAK signal) has been detected in the incoming data. Bit 1 Enable Transmitter Holding Register Empty Interrupt (ETBEI) Setting this bit to "1" causes the UART to generate an interrupt when the UART has room for one or more additional characters that are to be transmitted. Bit 0 Enable Received Data Available Interrupt (ERBFI) Setting this bit to "1" causes the UART to generate an interrupt when the UART has received enough characters to exceed the trigger level of the FIFO, or the FIFO timer has expired (stale data), or a single character has been received when the FIFO is disabled. +0x02 write FIFO Control Register (FCR) (This port does not exist on the 8250 and 16450 UART.) Bit 7 Receiver Trigger Bit #1 Bit 6 Receiver Trigger Bit #0These two bits control at what point the receiver is to generate an interrupt when the FIFO is active. 7 6 How many words are received before an interrupt is generated 0 0 1 0 1 4 1 0 8 1 1 14 Bit 5 Reserved, always 0. Bit 4 Reserved, always 0. Bit 3 DMA Mode Select. If Bit 0 is set to "1" (FIFOs enabled), setting this bit changes the operation of the -RXRDY and -TXRDY signals from Mode 0 to Mode 1. Bit 2 Transmit FIFO Reset. When a "1" is written to this bit, the contents of the FIFO are discarded. Any word currently being transmitted will be sent intact. This function is useful in aborting transfers. Bit 1 Receiver FIFO Reset. When a "1" is written to this bit, the contents of the FIFO are discarded. Any word currently being assembled in the shift register will be received intact. Bit 0 16550 FIFO Enable. When set, both the transmit and receive FIFOs are enabled. Any contents in the holding register, shift registers or FIFOs are lost when FIFOs are enabled or disabled. +0x02 read Interrupt Identification Register Bit 7 FIFOs enabled. On the 8250/16450 UART, this bit is zero. Bit 6 FIFOs enabled. On the 8250/16450 UART, this bit is zero. Bit 5 Reserved, always 0. Bit 4 Reserved, always 0. Bit 3 Interrupt ID Bit #2. On the 8250/16450 UART, this bit is zero. Bit 2 Interrupt ID Bit #1 Bit 1 Interrupt ID Bit #0.These three bits combine to report the category of event that caused the interrupt that is in progress. These categories have priorities, so if multiple categories of events occur at the same time, the UART will report the more important events first and the host must resolve the events in the order they are reported. All events that caused the current interrupt must be resolved before any new interrupts will be generated. (This is a limitation of the PC architecture.) 2 1 0 Priority Description 0 1 1 First Received Error (OE, PE, BI, or FE) 0 1 0 Second Received Data Available 1 1 0 Second Trigger level identification (Stale data in receive buffer) 0 0 1 Third Transmitter has room for more words (THRE) 0 0 0 Fourth Modem Status Change (-CTS, -DSR, -RI, or -DCD) Bit 0 Interrupt Pending Bit. If this bit is set to "0", then at least one interrupt is pending. +0x03 write/read Line Control Register (LCR) Bit 7 Divisor Latch Access Bit (DLAB). When set, access to the data transmit/receive register (THR/RBR) and the Interrupt Enable Register (IER) is disabled. Any access to these ports is now redirected to the Divisor Latch Registers. Setting this bit, loading the Divisor Registers, and clearing DLAB should be done with interrupts disabled. Bit 6 Set Break. When set to "1", the transmitter begins to transmit continuous Spacing until this bit is set to "0". This overrides any bits of characters that are being transmitted. Bit 5 Stick Parity. When parity is enabled, setting this bit causes parity to always be "1" or "0", based on the value of Bit 4. Bit 4 Even Parity Select (EPS). When parity is enabled and Bit 5 is "0", setting this bit causes even parity to be transmitted and expected. Otherwise, odd parity is used. Bit 3 Parity Enable (PEN). When set to "1", a parity bit is inserted between the last bit of the data and the Stop Bit. The UART will also expect parity to be present in the received data. Bit 2 Number of Stop Bits (STB). If set to "1" and using 5-bit data words, 1.5 Stop Bits are transmitted and expected in each data word. For 6, 7 and 8-bit data words, 2 Stop Bits are transmitted and expected. When this bit is set to "0", one Stop Bit is used on each data word. Bit 1 Word Length Select Bit #1 (WLSB1) Bit 0 Word Length Select Bit #0 (WLSB0) Together these bits specify the number of bits in each data word. 1 0 Word Length 0 0 5 Data Bits 0 1 6 Data Bits 1 0 7 Data Bits 1 1 8 Data Bits +0x04 write/read Modem Control Register (MCR) Bit 7 Reserved, always 0. Bit 6 Reserved, always 0. Bit 5 Reserved, always 0. Bit 4 Loop-Back Enable. When set to "1", the UART transmitter and receiver are internally connected together to allow diagnostic operations. In addition, the UART modem control outputs are connected to the UART modem control inputs. CTS is connected to RTS, DTR is connected to DSR, OUT1 is connected to RI, and OUT 2 is connected to DCD. Bit 3 OUT 2. An auxiliary output that the host processor may set high or low. In the IBM PC serial adapter (and most clones), OUT 2 is used to tri-state (disable) the interrupt signal from the 8250/16450/16550 UART. Bit 2 OUT 1. An auxiliary output that the host processor may set high or low. This output is not used on the IBM PC serial adapter. Bit 1 Request to Send (RTS). When set to "1", the output of the UART -RTS line is Low (Active). Bit 0 Data Terminal Ready (DTR). When set to "1", the output of the UART -DTR line is Low (Active). +0x05 write/read Line Status Register (LSR) Bit 7 Error in Receiver FIFO. On the 8250/16450 UART, this bit is zero. This bit is set to "1" when any of the bytes in the FIFO have one or more of the following error conditions: PE, FE, or BI. Bit 6 Transmitter Empty (TEMT). When set to "1", there are no words remaining in the transmit FIFO or the transmit shift register. The transmitter is completely idle. Bit 5 Transmitter Holding Register Empty (THRE). When set to "1", the FIFO (or holding register) now has room for at least one additional word to transmit. The transmitter may still be transmitting when this bit is set to "1". Bit 4 Break Interrupt (BI). The receiver has detected a Break signal. Bit 3 Framing Error (FE). A Start Bit was detected but the Stop Bit did not appear at the expected time. The received word is probably garbled. Bit 2 Parity Error (PE). The parity bit was incorrect for the word received. Bit 1 Overrun Error (OE). A new word was received and there was no room in the receive buffer. The newly-arrived word in the shift register is discarded. On 8250/16450 UARTs, the word in the holding register is discarded and the newly- arrived word is put in the holding register. Bit 0 Data Ready (DR) One or more words are in the receive FIFO that the host may read. A word must be completely received and moved from the shift register into the FIFO (or holding register for 8250/16450 designs) before this bit is set. +0x06 write/read Modem Status Register (MSR) Bit 7 Data Carrier Detect (DCD). Reflects the state of the DCD line on the UART. Bit 6 Ring Indicator (RI). Reflects the state of the RI line on the UART. Bit 5 Data Set Ready (DSR). Reflects the state of the DSR line on the UART. Bit 4 Clear To Send (CTS). Reflects the state of the CTS line on the UART. Bit 3 Delta Data Carrier Detect (DDCD). Set to "1" if the -DCD line has changed state one more time since the last time the MSR was read by the host. Bit 2 Trailing Edge Ring Indicator (TERI). Set to "1" if the -RI line has had a low to high transition since the last time the MSR was read by the host. Bit 1 Delta Data Set Ready (DDSR). Set to "1" if the -DSR line has changed state one more time since the last time the MSR was read by the host. Bit 0 Delta Clear To Send (DCTS). Set to "1" if the -CTS line has changed state one more time since the last time the MSR was read by the host. +0x07 write/read Scratch Register (SCR). This register performs no function in the UART. Any value can be written by the host to this location and read by the host later on. Beyond the 16550A UART Although National Semiconductor has not offered any components compatible with the 16550 that provide additional features, various other vendors have. Some of these components are described below. It should be understood that to effectively utilize these improvements, drivers may have to be provided by the chip vendor since most of the popular operating systems do not support features beyond those provided by the 16550. ST16650 By default this part is similar to the NS16550A, but an extended 32-byte send and receive buffer can be optionally enabled. Made by StarTech. TIL16660 By default this part behaves similar to the NS16550A, but an extended 64-byte send and receive buffer can be optionally enabled. Made by Texas Instruments. Hayes ESP This proprietary plug-in card contains a 2048-byte send and receive buffer, and supports data rates to 230.4Kbit/sec. Made by Hayes. In addition to these dumb UARTs, many vendors produce intelligent serial communication boards. This type of design usually provides a microprocessor that interfaces with several UARTs, processes and buffers the data, and then alerts the main PC processor when necessary. Because the UARTs are not directly accessed by the PC processor in this type of communication system, it is not necessary for the vendor to use UARTs that are compatible with the 8250, 16450, or the 16550 UART. This leaves the designer free to components that may have better performance characteristics. Configuring the <devicename>sio</devicename> driver The sio driver provides support for NS8250-, NS16450-, NS16550 and NS16550A-based EIA RS-232C (CCITT V.24) communications interfaces. Several multiport cards are supported as well. See the &man.sio.4; manual page for detailed technical documentation. Digi International (DigiBoard) PC/8 Contributed by &a.awebster;. 26 August 1995. Here is a config snippet from a machine with a Digi International PC/8 with 16550. It has 8 modems connected to these 8 lines, and they work just great. Do not forget to add options COM_MULTIPORT or it will not work very well! device sio4 at isa? port 0x100 flags 0xb05 device sio5 at isa? port 0x108 flags 0xb05 device sio6 at isa? port 0x110 flags 0xb05 device sio7 at isa? port 0x118 flags 0xb05 device sio8 at isa? port 0x120 flags 0xb05 device sio9 at isa? port 0x128 flags 0xb05 device sio10 at isa? port 0x130 flags 0xb05 device sio11 at isa? port 0x138 flags 0xb05 irq 9 The trick in setting this up is that the MSB of the flags represent the last SIO port, in this case 11 so flags are 0xb05. Boca 16 Contributed by &a.whiteside;. 26 August 1995. The procedures to make a Boca 16 port board with FreeBSD are pretty straightforward, but you will need a couple things to make it work: You either need the kernel sources installed so you can recompile the necessary options or you will need someone else to compile it for you. The 2.0.5 default kernel does not come with multiport support enabled and you will need to add a device entry for each port anyways. Two, you will need to know the interrupt and IO setting for your Boca Board so you can set these options properly in the kernel. One important note — the actual UART chips for the Boca 16 are in the connector box, not on the internal board itself. So if you have it unplugged, probes of those ports will fail. I have never tested booting with the box unplugged and plugging it back in, and I suggest you do not either. If you do not already have a custom kernel configuration file set up, refer to Kernel Configuration chapter of the FreeBSD Handbook for general procedures. The following are the specifics for the Boca 16 board and assume you are using the kernel name MYKERNEL and editing with vi. Add the line options COM_MULTIPORT to the config file. Where the current device sion lines are, you will need to add 16 more devices. The following example is for a Boca Board with an interrupt of 3, and a base IO address 100h. The IO address for Each port is +8 hexadecimal from the previous port, thus the 100h, 108h, 110h... addresses. device sio1 at isa? port 0x100 flags 0x1005 device sio2 at isa? port 0x108 flags 0x1005 device sio3 at isa? port 0x110 flags 0x1005 device sio4 at isa? port 0x118 flags 0x1005 … device sio15 at isa? port 0x170 flags 0x1005 device sio16 at isa? port 0x178 flags 0x1005 irq 3 The flags entry must be changed from this example unless you are using the exact same sio assignments. Flags are set according to 0xMYY where M indicates the minor number of the master port (the last port on a Boca 16) and YY indicates if FIFO is enabled or disabled(enabled), IRQ sharing is used(yes) and if there is an AST/4 compatible IRQ control register(no). In this example, flags 0x1005 indicates that the master port is sio16. If I added another board and assigned sio17 through sio28, the flags for all 16 ports on that board would be 0x1C05, where 1C indicates the minor number of the master port. Do not change the 05 setting. Save and complete the kernel configuration, recompile, install and reboot. Presuming you have successfully installed the recompiled kernel and have it set to the correct address and IRQ, your boot message should indicate the successful probe of the Boca ports as follows: (obviously the sio numbers, IO and IRQ could be different) sio1 at 0x100-0x107 flags 0x1005 on isa sio1: type 16550A (multiport) sio2 at 0x108-0x10f flags 0x1005 on isa sio2: type 16550A (multiport) sio3 at 0x110-0x117 flags 0x1005 on isa sio3: type 16550A (multiport) sio4 at 0x118-0x11f flags 0x1005 on isa sio4: type 16550A (multiport) sio5 at 0x120-0x127 flags 0x1005 on isa sio5: type 16550A (multiport) sio6 at 0x128-0x12f flags 0x1005 on isa sio6: type 16550A (multiport) sio7 at 0x130-0x137 flags 0x1005 on isa sio7: type 16550A (multiport) sio8 at 0x138-0x13f flags 0x1005 on isa sio8: type 16550A (multiport) sio9 at 0x140-0x147 flags 0x1005 on isa sio9: type 16550A (multiport) sio10 at 0x148-0x14f flags 0x1005 on isa sio10: type 16550A (multiport) sio11 at 0x150-0x157 flags 0x1005 on isa sio11: type 16550A (multiport) sio12 at 0x158-0x15f flags 0x1005 on isa sio12: type 16550A (multiport) sio13 at 0x160-0x167 flags 0x1005 on isa sio13: type 16550A (multiport) sio14 at 0x168-0x16f flags 0x1005 on isa sio14: type 16550A (multiport) sio15 at 0x170-0x177 flags 0x1005 on isa sio15: type 16550A (multiport) sio16 at 0x178-0x17f irq 3 flags 0x1005 on isa sio16: type 16550A (multiport master) If the messages go by too fast to see, &prompt.root; dmesg | more will show you the boot messages. Next, appropriate entries in /dev for the devices must be made using the /dev/MAKEDEV script. This step can be omitted if you are running FreeBSD 5.X with a kernel that has &man.devfs.5; support compiled in. If you do need to create the /dev entries, run the following as root: &prompt.root; cd /dev &prompt.root; ./MAKEDEV tty1 &prompt.root; ./MAKEDEV cua1 (everything in between) &prompt.root; ./MAKEDEV ttyg &prompt.root; ./MAKEDEV cuag If you do not want or need call-out devices for some reason, you can dispense with making the cua* devices. If you want a quick and sloppy way to make sure the devices are working, you can simply plug a modem into each port and (as root) &prompt.root; echo at > ttyd* for each device you have made. You should see the RX lights flash for each working port. Support for Cheap Multi-UART Cards Contributed by Helge Oldach hmo@sep.hamburg.com, September 1999 Ever wondered about FreeBSD support for your 20$ multi-I/O card with two (or more) COM ports, sharing IRQs? Here is how: Usually the only option to support these kind of boards is to use a distinct IRQ for each port. For example, if your CPU board has an on-board COM1 port (aka sio0–I/O address 0x3F8 and IRQ 4) and you have an extension board with two UARTs, you will commonly need to configure them as COM2 (aka sio1–I/O address 0x2F8 and IRQ 3), and the third port (aka sio2) as I/O 0x3E8 and IRQ 5. Obviously this is a waste of IRQ resources, as it should be basically possible to run both extension board ports using a single IRQ with the COM_MULTIPORT configuration described in the previous sections. Such cheap I/O boards commonly have a 4 by 3 jumper matrix for the COM ports, similar to the following: o o o * Port A | o * o * Port B | o * o o IRQ 2 3 4 5 Shown here is port A wired for IRQ 5 and port B wired for IRQ 3. The IRQ columns on your specific board may vary—other boards may supply jumpers for IRQs 3, 4, 5, and 7 instead. One could conclude that wiring both ports for IRQ 3 using a handcrafted wire-made jumper covering all three connection points in the IRQ 3 column would solve the issue, but no. You cannot duplicate IRQ 3 because the output drivers of each UART are wired in a totem pole fashion, so if one of the UARTs drives IRQ 3, the output signal will not be what you would expect. Depending on the implementation of the extension board or your motherboard, the IRQ 3 line will continuously stay up, or always stay low. You need to decouple the IRQ drivers for the two UARTs, so that the IRQ line of the board only goes up if (and only if) one of the UARTs asserts a IRQ, and stays low otherwise. The solution was proposed by Joerg Wunsch j@ida.interface-business.de: To solder up a wired-or consisting of two diodes (Germanium or Schottky-types strongly preferred) and a 1 kOhm resistor. Here is the schematic, starting from the 4 by 3 jumper field above: Diode +---------->|-------+ / | o * o o | 1 kOhm Port A +----|######|-------+ o * o o | | Port B `-------------------+ ==+== o * o o | Ground \ | +--------->|-------+ IRQ 2 3 4 5 Diode The cathodes of the diodes are connected to a common point, together with a 1 kOhm pull-down resistor. It is essential to connect the resistor to ground to avoid floating of the IRQ line on the bus. Now we are ready to configure a kernel. Staying with this example, we would configure: # standard on-board COM1 port device sio0 at isa? port "IO_COM1" flags 0x10 # patched-up multi-I/O extension board options COM_MULTIPORT device sio1 at isa? port "IO_COM2" flags 0x205 device sio2 at isa? port "IO_COM3" flags 0x205 irq 3 Note that the flags setting for sio1 and sio2 is truly essential; refer to &man.sio.4; for details. (Generally, the 2 in the "flags" attribute refers to sio2 which holds the IRQ, and you surely want a 5 low nibble.) With kernel verbose mode turned on this should yield something similar to this: sio0: irq maps: 0x1 0x11 0x1 0x1 sio0 at 0x3f8-0x3ff irq 4 flags 0x10 on isa sio0: type 16550A sio1: irq maps: 0x1 0x9 0x1 0x1 sio1 at 0x2f8-0x2ff flags 0x205 on isa sio1: type 16550A (multiport) sio2: irq maps: 0x1 0x9 0x1 0x1 sio2 at 0x3e8-0x3ef irq 3 flags 0x205 on isa sio2: type 16550A (multiport master) Though /sys/i386/isa/sio.c is somewhat cryptic with its use of the irq maps array above, the basic idea is that you observe 0x1 in the first, third, and fourth place. This means that the corresponding IRQ was set upon output and cleared after, which is just what we would expect. If your kernel does not display this behavior, most likely there is something wrong with your wiring. Configuring the <devicename>cy</devicename> driver Contributed by Alex Nash. 6 June 1996. The Cyclades multiport cards are based on the cy driver instead of the usual sio driver used by other multiport cards. Configuration is a simple matter of: Add the cy device to your kernel configuration (note that your irq and iomem settings may differ). device cy0 at isa? irq 10 iomem 0xd4000 iosiz 0x2000 Rebuild and install the new kernel. Make the device nodes by typing (the following example assumes an 8-port board) You can omit this part if you are running FreeBSD 5.X with &man.devfs.5;. : &prompt.root; cd /dev &prompt.root; for i in 0 1 2 3 4 5 6 7;do ./MAKEDEV cuac$i ttyc$i;done If appropriate, add dialup entries to /etc/ttys by duplicating serial device (ttyd) entries and using ttyc in place of ttyd. For example: ttyc0 "/usr/libexec/getty std.38400" unknown on insecure ttyc1 "/usr/libexec/getty std.38400" unknown on insecure ttyc2 "/usr/libexec/getty std.38400" unknown on insecure … ttyc7 "/usr/libexec/getty std.38400" unknown on insecure Reboot with the new kernel. Configuring the <devicename>si</devicename> driver Contributed by &a.nsayer;. 25 March 1998. The Specialix SI/XIO and SX multiport cards use the si driver. A single machine can have up to 4 host cards. The following host cards are supported: ISA SI/XIO host card (2 versions) EISA SI/XIO host card PCI SI/XIO host card ISA SX host card PCI SX host card Although the SX and SI/XIO host cards look markedly different, their functionality are basically the same. The host cards do not use I/O locations, but instead require a 32K chunk of memory. The factory configuration for ISA cards places this at 0xd0000-0xd7fff. They also require an IRQ. PCI cards will, of course, auto-configure themselves. You can attach up to 4 external modules to each host card. The external modules contain either 4 or 8 serial ports. They come in the following varieties: SI 4 or 8 port modules. Up to 57600 bps on each port supported. XIO 8 port modules. Up to 115200 bps on each port supported. One type of XIO module has 7 serial and 1 parallel port. SXDC 8 port modules. Up to 921600 bps on each port supported. Like XIO, a module is available with one parallel port as well. To configure an ISA host card, add the following line to your kernel configuration file, changing the numbers as appropriate: device si0 at isa? iomem 0xd0000 irq 11 Valid IRQ numbers are 9, 10, 11, 12 and 15 for SX ISA host cards and 11, 12 and 15 for SI/XIO ISA host cards. To configure an EISA or PCI host card, use this line: device si0 After adding the configuration entry, rebuild and install your new kernel. The following step, is not necessary if you are using &man.devfs.5; in FreeBSD 5.X. After rebooting with the new kernel, you need to make the device nodes in /dev. The MAKEDEV script will take care of this for you. Count how many total ports you have and type: &prompt.root; cd /dev &prompt.root; ./MAKEDEV ttyAnn cuaAnn (where nn is the number of ports) If you want login prompts to appear on these ports, you will need to add lines like this to /etc/ttys: ttyA01 "/usr/libexec/getty std.9600" vt100 on insecure Change the terminal type as appropriate. For modems, dialup or unknown is fine.
diff --git a/en_US.ISO8859-1/articles/storage-devices/article.sgml b/en_US.ISO8859-1/articles/storage-devices/article.sgml index 97e8b25043..30485174e6 100644 --- a/en_US.ISO8859-1/articles/storage-devices/article.sgml +++ b/en_US.ISO8859-1/articles/storage-devices/article.sgml @@ -1,2643 +1,2643 @@ %articles.ent; ]>
Storage Devices Wilko Bulte
wilko@FreeBSD.org
 $FreeBSD$ &tm-attrib.freebsd; &tm-attrib.general; This article talks about storage devices with FreeBSD. Using ESDI hard disks Copyright © 1995, &a.wilko;. 24 September 1995. ESDI is an acronym that means Enhanced Small Device Interface. It is loosely based on the good old ST506/412 interface originally devised by Seagate Technology, the makers - of the first affordable 5.25" winchester disk. + of the first affordable 5.25" Winchester disk. The acronym says Enhanced, and rightly so. In the first place the speed of the interface is higher, 10 or 15 Mbits/second instead of the 5 Mbits/second of ST412 interfaced drives. Secondly some higher level commands are added, making the ESDI interface somewhat smarter to the operating system driver writers. It is by no means as smart as SCSI by the way. ESDI is standardized by ANSI. Capacities of the drives are boosted by putting more sectors on each track. Typical is 35 sectors per track, high capacity drives I have seen were up to 54 sectors/track. Although ESDI has been largely obsoleted by IDE and SCSI interfaces, the availability of free or cheap surplus drives makes them ideal for low (or now) budget systems. Concepts of ESDI Physical connections The ESDI interface uses two cables connected to each drive. One cable is a 34 pin flat cable edge connector that carries the command and status signals from the controller to the drive and vice-versa. The command cable is daisy chained between all the drives. So, it forms a bus onto which all drives are connected. The second cable is a 20 pin flat cable edge connector that carries the data to and from the drive. This cable is radially connected, so each drive has its own direct connection to the controller. To the best of my knowledge PC ESDI controllers are limited to using a maximum of 2 drives per controller. This is compatibility feature(?) left over from the WD1003 standard that reserves only a single bit for device addressing. Device addressing On each command cable a maximum of 7 devices and 1 controller can be present. To enable the controller to uniquely identify which drive it addresses, each ESDI device is equipped with jumpers or switches to select the devices address. On PC type controllers the first drive is set to address 0, the second disk to address 1. Always make sure you set each disk to an unique address! So, on a PC with its two drives/controller maximum the first drive is drive 0, the second is drive 1. Termination The daisy chained command cable (the 34 pin cable remember?) needs to be terminated at the last drive on the chain. For this purpose ESDI drives come with a termination resistor network that can be removed or disabled by a jumper when it is not used. So, one and only one drive, the one at the farthest end of the command cable has its terminator installed/enabled. The controller automatically terminates the other end of the cable. Please note that this implies that the controller must be at one end of the cable and not in the middle. Using ESDI disks with FreeBSD Why is ESDI such a pain to get working in the first place? People who tried ESDI disks with FreeBSD are known to have developed a profound sense of frustration. A combination of factors works against you to produce effects that are hard to understand when you have never seen them before. This has also led to the popular legend ESDI and FreeBSD is a plain NO-GO. The following sections try to list all the pitfalls and solutions. ESDI speed variants As briefly mentioned before, ESDI comes in two speed flavors. The older drives and controllers use a 10 Mbits/second data transfer rate. Newer stuff uses 15 Mbits/second. It is not hard to imagine that 15 Mbits/second drive cause problems on controllers laid out for 10 Mbits/second. As always, consult your controller and drive documentation to see if things match. Stay on track Mainstream ESDI drives use 34 to 36 sectors per track. Most (older) controllers cannot handle more than this number of sectors. Newer, higher capacity, drives use higher numbers of sectors per track. For instance, I own a 670 MB drive that has 54 sectors per track. In my case, the controller could not handle this number of sectors. It proved to work well except that it only used 35 sectors on each track. This meant losing a lot of disk space. Once again, check the documentation of your hardware for more info. Going out-of-spec like in the example might or might not work. Give it a try or get another more capable controller. Hard or soft sectoring Most ESDI drives allow hard or soft sectoring to be selected using a jumper. Hard sectoring means that the drive will produce a sector pulse on the start of each new sector. The controller uses this pulse to tell when it should start to write or read. Hard sectoring allows a selection of sector size (normally 256, 512 or 1024 bytes per formatted sector). FreeBSD uses 512 byte sectors. The number of sectors per track also varies while still using the same number of bytes per formatted sector. The number of unformatted bytes per sector varies, dependent on your controller it needs more or less overhead bytes to work correctly. Pushing more sectors on a track of course gives you more usable space, but might give problems if your controller needs more bytes than the drive offers. In case of soft sectoring, the controller itself determines where to start/stop reading or writing. For ESDI hard sectoring is the default (at least on everything I came across). I never felt the urge to try soft sectoring. In general, experiment with sector settings before you install FreeBSD because you need to re-run the low-level format after each change. Low level formatting ESDI drives need to be low level formatted before they are usable. A reformat is needed whenever you figgle with the number of sectors/track jumpers or the physical orientation of the drive (horizontal, vertical). So, first think, then format. The format time must not be underestimated, for big disks it can take hours. After a low level format, a surface scan is done to find and flag bad sectors. Most disks have a manufacturer bad block list listed on a piece of paper or adhesive sticker. In addition, on most disks the list is also written onto the disk. Please use the manufacturer's list. It is much easier to remap a defect now than after FreeBSD is installed. Stay away from low-level formatters that mark all sectors of a track as bad as soon as they find one bad sector. Not only does this waste space, it also and more importantly causes you grief with bad144 (see the section on bad144). Translations Translations, although not exclusively a ESDI-only problem, might give you real trouble. Translations come in multiple flavors. Most of them have in common that they attempt to work around the limitations posed upon disk geometries by the original IBM PC/AT design (thanks IBM!). First of all there is the (in)famous 1024 cylinder limit. For a system to be able to boot, the stuff (whatever operating system) must be in the first 1024 cylinders of a disk. Only 10 bits are available to encode the cylinder number. For the number of sectors the limit is 64 (0-63). When you combine the 1024 cylinder limit with the 16 head limit (also a design feature) you max out at fairly limited disk sizes. To work around this problem, the manufacturers of ESDI PC controllers added a BIOS prom extension on their boards. This BIOS extension handles disk I/O for booting (and for some operating systems all disk I/O) by using translation. For instance, a big drive might be presented to the system as having 32 heads and 64 sectors/track. The result is that the number of cylinders is reduced to something below 1024 and is therefore usable by the system without problems. It is noteworthy to know that FreeBSD does not use the BIOS after its kernel has started. More on this later. A second reason for translations is the fact that most older system BIOSes could only handle drives with 17 sectors per track (the old ST412 standard). Newer system BIOSes usually have a user-defined drive type (in most cases this is drive type 47). Whatever you do to translations after reading this document, keep in mind that if you have multiple operating systems on the same disk, all must use the same translation While on the subject of translations, I have seen one controller type (but there are probably more like this) offer the option to logically split a drive in multiple partitions as a BIOS option. I had select 1 drive == 1 partition because this controller wrote this info onto the disk. On power-up it read the info and presented itself to the system based on the info from the disk. Spare sectoring Most ESDI controllers offer the possibility to remap bad sectors. During/after the low-level format of the disk bad sectors are marked as such, and a replacement sector is put in place (logically of course) of the bad one. In most cases the remapping is done by using N-1 sectors on each track for actual data storage, and sector N itself is the spare sector. N is the total number of sectors physically available on the track. The idea behind this is that the operating system sees a perfect disk without bad sectors. In the case of FreeBSD this concept is not usable. The problem is that the translation from bad to good is performed by the BIOS of the ESDI controller. FreeBSD, being a true 32 bit operating system, does not use the BIOS after it has been booted. Instead, it has device drivers that talk directly to the hardware. So: do not use spare sectoring, bad block remapping or whatever it may be called by the controller manufacturer when you want to use the disk for FreeBSD. Bad block handling The preceding section leaves us with a problem. The controller's bad block handling is not usable and still FreeBSD's filesystems assume perfect media without any flaws. To solve this problem, FreeBSD use the bad144 tool. Bad144 (named after a Digital Equipment standard for bad block handling) scans a FreeBSD slice for bad blocks. Having found these bad blocks, it writes a table with the offending block numbers to the end of the FreeBSD slice. When the disk is in operation, the disk accesses are checked against the table read from the disk. Whenever a block number is requested that is in the bad144 list, a replacement block (also from the end of the FreeBSD slice) is used. In this way, the bad144 replacement scheme presents perfect media to the FreeBSD filesystems. There are a number of potential pitfalls associated with the use of bad144. First of all, the slice cannot have more than 126 bad sectors. If your drive has a high number of bad sectors, you might need to divide it into multiple FreeBSD slices each containing less than 126 bad sectors. Stay away from low-level format programs that mark every sector of a track as bad when they find a flaw on the track. As you can imagine, the 126 limit is quickly reached when the low-level format is done this way. Second, if the slice contains the root filesystem, the slice should be within the 1024 cylinder BIOS limit. During the boot process the bad144 list is read using the BIOS and this only succeeds when the list is within the 1024 cylinder limit. The restriction is not that only the root filesystem must be within the 1024 cylinder limit, but rather the entire slice that contains the root filesystem. Kernel configuration ESDI disks are handled by the same wddriver as IDE and ST412 MFM disks. The wd driver should work for all WD1003 compatible interfaces. Most hardware is jumperable for one of two different I/O address ranges and IRQ lines. This allows you to have two wd type controllers in one system. When your hardware allows non-standard strappings, you can use these with FreeBSD as long as you enter the correct info into the kernel config file. An example from the kernel config file (they live in /sys/i386/conf BTW). # First WD compatible controller controller wdc0 at isa? port "IO_WD1" bio irq 14 vector wdintr disk wd0 at wdc0 drive 0 disk wd1 at wdc0 drive 1 # Second WD compatible controller controller wdc1 at isa? port "IO_WD2" bio irq 15 vector wdintr disk wd2 at wdc1 drive 0 disk wd3 at wdc1 drive 1 Particulars on ESDI hardware Adaptec 2320 controllers I successfully installed FreeBSD onto a ESDI disk controlled by a ACB-2320. No other operating system was present on the disk. To do so I low level formatted the disk using NEFMT.EXE (ftpable from www.adaptec.com) and answered NO to the question whether the disk should be formatted with a spare sector on each track. The BIOS on the ACD-2320 was disabled. I used the free configurable option in the system BIOS to allow the BIOS to boot it. Before using NEFMT.EXE I tried to format the disk using the ACB-2320 BIOS built-in formatter. This proved to be a show stopper, because it did not give me an option to disable spare sectoring. With spare sectoring enabled the FreeBSD installation process broke down on the bad144 run. Please check carefully which ACB-232xy variant you have. The x is either 0 or 2, indicating a controller without or with a floppy controller on board. The y is more interesting. It can either be a blank, a A-8 or a D. A blank indicates a plain 10 Mbits/second controller. An A-8 indicates a 15 Mbits/second controller capable of handling 52 sectors/track. A D means a 15 Mbits/second controller that can also handle drives with > 36 sectors/track (also 52?). All variations should be capable of using 1:1 interleaving. Use 1:1, FreeBSD is fast enough to handle it. Western Digital WD1007 controllers I successfully installed FreeBSD onto a ESDI disk controlled by a WD1007 controller. To be precise, it was a WD1007-WA2. Other variations of the WD1007 do exist. To get it to work, I had to disable the sector translation and the WD1007's onboard BIOS. This implied I could not use the low-level formatter built into this BIOS. Instead, I grabbed WDFMT.EXE from www.wdc.com Running this formatted my drive just fine. Ultrastor U14F controllers According to multiple reports from the net, Ultrastor ESDI boards work OK with FreeBSD. I lack any further info on particular settings. Further reading If you intend to do some serious ESDI hacking, you might want to have the official standard at hand: The latest ANSI X3T10 committee document is: Enhanced Small Device Interface (ESDI) [X3.170-1990/X3.170a-1991] [X3T10/792D Rev 11] On Usenet the newsgroup comp.periphs is a noteworthy place to look for more info. The World Wide Web (WWW) also proves to be a very handy info source: For info on Adaptec ESDI controllers see . For info on Western Digital controllers see . Thanks to... Andrew Gordon for sending me an Adaptec 2320 controller and ESDI disk for testing. What is SCSI? Copyright © 1995, &a.wilko;. July 6, 1996. SCSI is an acronym for Small Computer Systems Interface. It is an ANSI standard that has become one of the leading I/O buses in the computer industry. The foundation of the SCSI standard was laid by Shugart Associates (the same guys that gave the world the first mini floppy disks) when they introduced the SASI bus (Shugart Associates Standard Interface). After some time an industry effort was started to come to a more strict standard allowing devices from different vendors to work together. This effort was recognized in the ANSI SCSI-1 standard. The SCSI-1 standard (approximately 1985) is rapidly becoming obsolete. The current standard is SCSI-2 (see Further reading), with SCSI-3 on the drawing boards. In addition to a physical interconnection standard, SCSI defines a logical (command set) standard to which disk devices must adhere. This standard is called the Common Command Set (CCS) and was developed more or less in parallel with ANSI SCSI-1. SCSI-2 includes the (revised) CCS as part of the standard itself. The commands are dependent on the type of device at hand. It does not make much sense of course to define a Write command for a scanner. The SCSI bus is a parallel bus, which comes in a number of variants. The oldest and most used is an 8 bit wide bus, with single-ended signals, carried on 50 wires. (If you do not know what single-ended means, do not worry, that is what this document is all about.) Modern designs also use 16 bit wide buses, with differential signals. This allows transfer speeds of 20Mbytes/second, on cables lengths of up to 25 meters. SCSI-2 allows a maximum bus width of 32 bits, using an additional cable. Quickly emerging are Ultra SCSI (also called Fast-20) and Ultra2 (also called Fast-40). Fast-20 is 20 million transfers per second (20 Mbytes/sec on a 8 bit bus), Fast-40 is 40 million transfers per second (40 Mbytes/sec on a 8 bit bus). Most hard drives sold today are single-ended Ultra SCSI (8 or 16 bits). Of course the SCSI bus not only has data lines, but also a number of control signals. A very elaborate protocol is part of the standard to allow multiple devices to share the bus in an efficient manner. In SCSI-2, the data is always checked using a separate parity line. In pre-SCSI-2 designs parity was optional. In SCSI-3 even faster bus types are introduced, along with a serial SCSI busses that reduces the cabling overhead and allows a higher maximum bus length. You might see names like SSA and fibre channel in this context. None of the serial buses are currently in widespread use (especially not in the typical FreeBSD environment). For this reason the serial bus types are not discussed any further. As you could have guessed from the description above, SCSI devices are intelligent. They have to be to adhere to the SCSI standard (which is over 2 inches thick BTW). So, for a hard disk drive for instance you do not specify a head/cylinder/sector to address a particular block, but simply the number of the block you want. Elaborate caching schemes, automatic bad block replacement etc are all made possible by this intelligent device approach. On a SCSI bus, each possible pair of devices can communicate. Whether their function allows this is another matter, but the standard does not restrict it. To avoid signal contention, the 2 devices have to arbitrate for the bus before using it. The philosophy of SCSI is to have a standard that allows older-standard devices to work with newer-standard ones. So, an old SCSI-1 device should normally work on a SCSI-2 bus. I say Normally, because it is not absolutely sure that the implementation of an old device follows the (old) standard closely enough to be acceptable on a new bus. Modern devices are usually more well-behaved, because the standardization has become more strict and is better adhered to by the device manufacturers. Generally speaking, the chances of getting a working set of devices on a single bus is better when all the devices are SCSI-2 or newer. This implies that you do not have to dump all your old stuff when you get that shiny 80GB disk: I own a system on which a pre-SCSI-1 disk, a SCSI-2 QIC tape unit, a SCSI-1 helical scan tape unit and 2 SCSI-1 disks work together quite happily. From a performance standpoint you might want to separate your older and newer (=faster) devices however. This is especially advantageous if you have an Ultra160 host adapter where you should separate your U160 devices from the Fast and Wide SCSI-2 devices. Components of SCSI As said before, SCSI devices are smart. The idea is to put the knowledge about intimate hardware details onto the SCSI device itself. In this way, the host system does not have to worry about things like how many heads a hard disks has, or how many tracks there are on a specific tape device. If you are curious, the standard specifies commands with which you can query your devices on their hardware particulars. FreeBSD uses this capability during boot to check out what devices are connected and whether they need any special treatment. The advantage of intelligent devices is obvious: the device drivers on the host can be made in a much more generic fashion, there is no longer a need to change (and qualify!) drivers for every odd new device that is introduced. For cabling and connectors there is a golden rule: get good stuff. With bus speeds going up all the time you will save yourself a lot of grief by using good material. So, gold plated connectors, shielded cabling, sturdy connector hoods with strain reliefs etc are the way to go. Second golden rule: do no use cables longer than necessary. I once spent 3 days hunting down a problem with a flaky machine only to discover that shortening the SCSI bus by 1 meter solved the problem. And the original bus length was well within the SCSI specification. SCSI bus types From an electrical point of view, there are two incompatible bus types: single-ended and differential. This means that there are two different main groups of SCSI devices and controllers, which cannot be mixed on the same bus. It is possible however to use special converter hardware to transform a single-ended bus into a differential one (and vice versa). The differences between the bus types are explained in the next sections. In lots of SCSI related documentation there is a sort of jargon in use to abbreviate the different bus types. A small list: FWD: Fast Wide Differential FND: Fast Narrow Differential SE: Single Ended FN: Fast Narrow etc. With a minor amount of imagination one can usually imagine what is meant. Wide is a bit ambiguous, it can indicate 16 or 32 bit buses. As far as I know, the 32 bit variant is not (yet) in use, so wide normally means 16 bit. Fast means that the timing on the bus is somewhat different, so that on a narrow (8 bit) bus 10 Mbytes/sec are possible instead of 5 Mbytes/sec for slow SCSI. As discussed before, bus speeds of 20 and 40 million transfers/second are also emerging (Fast-20 == Ultra SCSI and Fast-40 == Ultra2 SCSI). The data lines > 8 are only used for data transfers and device addressing. The transfers of commands and status messages etc are only performed on the lowest 8 data lines. The standard allows narrow devices to operate on a wide bus. The usable bus width is negotiated between the devices. You have to watch your device addressing closely when mixing wide and narrow. Single ended buses A single-ended SCSI bus uses signals that are either 5 Volts or 0 Volts (indeed, TTL levels) and are relative to a COMMON ground reference. A singled ended 8 bit SCSI bus has approximately 25 ground lines, who are all tied to a single rail on all devices. A standard single ended bus has a maximum length of 6 meters. If the same bus is used with fast-SCSI devices, the maximum length allowed drops to 3 meters. Fast-SCSI means that instead of 5Mbytes/sec the bus allows 10Mbytes/sec transfers. Fast-20 (Ultra SCSI) and Fast-40 allow for 20 and 40 million transfers/second respectively. So, F20 is 20 Mbytes/second on a 8 bit bus, 40 Mbytes/second on a 16 bit bus etc. For F20 the max bus length is 1.5 meters, for F40 it becomes 0.75 meters. Be aware that F20 is pushing the limits quite a bit, so you will quickly find out if your SCSI bus is electrically sound. If some devices on your bus use fast to communicate your bus must adhere to the length restrictions for fast buses! It is obvious that with the newer fast-SCSI devices the bus length can become a real bottleneck. This is why the differential SCSI bus was introduced in the SCSI-2 standard. For connector pinning and connector types please refer to the SCSI-2 standard (see Further reading) itself, connectors etc are listed there in painstaking detail. Beware of devices using non-standard cabling. For instance Apple uses a 25pin D-type connecter (like the one on serial ports and parallel printers). Considering that the official SCSI bus needs 50 pins you can imagine the use of this connector needs some creative cabling. The reduction of the number of ground wires they used is a bad idea, you better stick to 50 pins cabling in accordance with the SCSI standard. For Fast-20 and 40 do not even think about buses like this. Differential buses A differential SCSI bus has a maximum length of 25 meters. Quite a difference from the 3 meters for a single-ended fast-SCSI bus. The idea behind differential signals is that each bus signal has its own return wire. So, each signal is carried on a (preferably twisted) pair of wires. The voltage difference between these two wires determines whether the signal is asserted or de-asserted. To a certain extent the voltage difference between ground and the signal wire pair is not relevant (do not try 10 kVolts though). It is beyond the scope of this document to explain why this differential idea is so much better. Just accept that electrically seen the use of differential signals gives a much better noise margin. You will normally find differential buses in use for inter-cabinet connections. Because of the lower cost single ended is mostly used for shorter buses like inside cabinets. There is nothing that stops you from using differential stuff with FreeBSD, as long as you use a controller that has device driver support in FreeBSD. As an example, Adaptec marketed the AHA1740 as a single ended board, whereas the AHA1744 was differential. The software interface to the host is identical for both. Terminators Terminators in SCSI terminology are resistor networks that are used to get a correct impedance matching. Impedance matching is important to get clean signals on the bus, without reflections or ringing. If you once made a long distance telephone call on a bad line you probably know what reflections are. With 20Mbytes/sec traveling over your SCSI bus, you do not want signals echoing back. Terminators come in various incarnations, with more or less sophisticated designs. Of course, there are internal and external variants. Many SCSI devices come with a number of sockets in which a number of resistor networks can (must be!) installed. If you remove terminators from a device, carefully store them. You will need them when you ever decide to reconfigure your SCSI bus. There is enough variation in even these simple tiny things to make finding the exact replacement a frustrating business. There are also SCSI devices that have a single jumper to enable or disable a built-in terminator. There are special terminators you can stick onto a flat cable bus. Others look like external connectors, or a connector hood without a cable. So, lots of choice as you can see. There is much debate going on if and when you should switch from simple resistor (passive) terminators to active terminators. Active terminators contain slightly more elaborate circuit to give cleaner bus signals. The general consensus seems to be that the usefulness of active termination increases when you have long buses and/or fast devices. If you ever have problems with your SCSI buses you might consider trying an active terminator. Try to borrow one first, they reputedly are quite expensive. Please keep in mind that terminators for differential and single-ended buses are not identical. You should not mix the two variants. OK, and now where should you install your terminators? This is by far the most misunderstood part of SCSI. And it is by far the simplest. The rule is: every single line on the SCSI bus has 2 (two) terminators, one at each end of the bus. So, two and not one or three or whatever. Do yourself a favor and stick to this rule. It will save you endless grief, because wrong termination has the potential to introduce highly mysterious bugs. (Note the potential here; the nastiest part is that it may or may not work.) A common pitfall is to have an internal (flat) cable in a machine and also an external cable attached to the controller. It seems almost everybody forgets to remove the terminators from the controller. The terminator must now be on the last external device, and not on the controller! In general, every reconfiguration of a SCSI bus must pay attention to this. Termination is to be done on a per-line basis. This means if you have both narrow and wide buses connected to the same host adapter, you need to enable termination on the higher 8 bits of the bus on the adapter (as well as the last devices on each bus, of course). What I did myself is remove all terminators from my SCSI devices and controllers. I own a couple of external terminators, for both the Centronics-type external cabling and for the internal flat cable connectors. This makes reconfiguration much easier. On modern devices, sometimes integrated terminators are used. These things are special purpose integrated circuits that can be enabled or disabled with a control pin. It is not necessary to physically remove them from a device. You may find them on newer host adapters, sometimes they are software configurable, using some sort of setup tool. Some will even auto-detect the cables attached to the connectors and automatically set up the termination as necessary. At any rate, consult your documentation! Terminator power The terminators discussed in the previous chapter need power to operate properly. On the SCSI bus, a line is dedicated to this purpose. So, simple huh? Not so. Each device can provide its own terminator power to the terminator sockets it has on-device. But if you have external terminators, or when the device supplying the terminator power to the SCSI bus line is switched off you are in trouble. The idea is that initiators (these are devices that initiate actions on the bus, a discussion follows) must supply terminator power. All SCSI devices are allowed (but not required) to supply terminator power. To allow for un-powered devices on a bus, the terminator power must be supplied to the bus via a diode. This prevents the backflow of current to un-powered devices. To prevent all kinds of nastiness, the terminator power is usually fused. As you can imagine, fuses might blow. This can, but does not have to, lead to a non functional bus. If multiple devices supply terminator power, a single blown fuse will not put you out of business. A single supplier with a blown fuse certainly will. Clever external terminators sometimes have a LED indication that shows whether terminator power is present. In newer designs auto-restoring fuses that reset themselves after some time are sometimes used. Device addressing Because the SCSI bus is, ehh, a bus there must be a way to distinguish or address the different devices connected to it. This is done by means of the SCSI or target ID. Each device has a unique target ID. You can select the ID to which a device must respond using a set of jumpers, or a dip switch, or something similar. Some SCSI host adapters let you change the target ID from the boot menu. (Yet some others will not let you change the ID from 7.) Consult the documentation of your device for more information. Beware of multiple devices configured to use the same ID. Chaos normally reigns in this case. A pitfall is that one of the devices sharing the same ID sometimes even manages to answer to I/O requests! For an 8 bit bus, a maximum of 8 targets is possible. The maximum is 8 because the selection is done bitwise using the 8 data lines on the bus. For wide buses this increases to the number of data lines (usually 16). A narrow SCSI device can not communicate with a SCSI device with a target ID larger than 7. This means it is generally not a good idea to move your SCSI host adapter's target ID to something higher than 7 (or your CDROM will stop working). The higher the SCSI target ID, the higher the priority the devices has. When it comes to arbitration between devices that want to use the bus at the same time, the device that has the highest SCSI ID will win. This also means that the SCSI host adapter usually uses target ID 7. Note however that the lower 8 IDs have higher priorities than the higher 8 IDs on a wide-SCSI bus. Thus, the order of target IDs is: [7 6 .. 1 0 15 14 .. 9 8] on a wide-SCSI system. (If you are wondering why the lower 8 have higher priority, read the previous paragraph for a hint.) For a further subdivision, the standard allows for Logical Units or LUNs for short. A single target ID may have multiple LUNs. For example, a tape device including a tape changer may have LUN 0 for the tape device itself, and LUN 1 for the tape changer. In this way, the host system can address each of the functional units of the tape changer as desired. Bus layout SCSI buses are linear. So, not shaped like Y-junctions, star topologies, rings, cobwebs or whatever else people might want to invent. One of the most common mistakes is for people with wide-SCSI host adapters to connect devices on all three connecters (external connector, internal wide connector, internal narrow connector). Do not do that. It may appear to work if you are really lucky, but I can almost guarantee that your system will stop functioning at the most unfortunate moment (this is also known as Murphy's law). You might notice that the terminator issue discussed earlier becomes rather hairy if your bus is not linear. Also, if you have more connectors than devices on your internal SCSI cable, make sure you attach devices on connectors on both ends instead of using the connectors in the middle and let one or both ends dangle. This will screw up the termination of the bus. The electrical characteristics, its noise margins and ultimately the reliability of it all are tightly related to linear bus rule. Stick to the linear bus rule! Using SCSI with FreeBSD About translations, BIOSes and magic... As stated before, you should first make sure that you have a electrically sound bus. When you want to use a SCSI disk on your PC as boot disk, you must aware of some quirks related to PC BIOSes. The PC BIOS in its first incarnation used a low level physical interface to the hard disk. So, you had to tell the BIOS (using a setup tool or a BIOS built-in setup) how your disk physically looked like. This involved stating number of heads, number of cylinders, number of sectors per track, obscure things like precompensation and reduced write current cylinder etc. One might be inclined to think that since SCSI disks are smart you can forget about this. Alas, the arcane setup issue is still present today. The system BIOS needs to know how to access your SCSI disk with the head/cyl/sector method in order to load the FreeBSD kernel during boot. The SCSI host adapter or SCSI controller you have put in your AT/EISA/PCI/whatever bus to connect your disk therefore has its own on-board BIOS. During system startup, the SCSI BIOS takes over the hard disk interface routines from the system BIOS. To fool the system BIOS, the system setup is normally set to No hard disk present. Obvious, is it not? The SCSI BIOS itself presents to the system a so called translated drive. This means that a fake drive table is constructed that allows the PC to boot the drive. This translation is often (but not always) done using a pseudo drive with 64 heads and 32 sectors per track. By varying the number of cylinders, the SCSI BIOS adapts to the actual drive size. It is useful to note that 32 * 64 / 2 = the size of your drive in megabytes. The division by 2 is to get from disk blocks that are normally 512 bytes in size to Kbytes. Right. All is well now?! No, it is not. The system BIOS has another quirk you might run into. The number of cylinders of a bootable hard disk cannot be greater than 1024. Using the translation above, this is a show-stopper for disks greater than 1 GB. With disk capacities going up all the time this is causing problems. Fortunately, the solution is simple: just use another translation, e.g. with 128 heads instead of 32. In most cases new SCSI BIOS versions are available to upgrade older SCSI host adapters. Some newer adapters have an option, in the form of a jumper or software setup selection, to switch the translation the SCSI BIOS uses. It is very important that all operating systems on the disk use the same translation to get the right idea about where to find the relevant partitions. So, when installing FreeBSD you must answer any questions about heads/cylinders etc using the translated values your host adapter uses. Failing to observe the translation issue might lead to un-bootable systems or operating systems overwriting each others partitions. Using fdisk you should be able to see all partitions. You might have heard some talk of lying devices? Older FreeBSD kernels used to report the geometry of SCSI disks when booting. An example from one of my systems: aha0 targ 0 lun 0: <MICROP 1588-15MB1057404HSP4> da0: 636MB (1303250 total sec), 1632 cyl, 15 head, 53 sec, bytes/sec 512 Newer kernels usually do not report this information. e.g. (bt0:0:0): "SEAGATE ST41651 7574" type 0 fixed SCSI 2 da0(bt0:0:0): Direct-Access 1350MB (2766300 512 byte sectors) Why has this changed? This info is retrieved from the SCSI disk itself. Newer disks often use a technique called zone bit recording. The idea is that on the outer cylinders of the drive there is more space so more sectors per track can be put on them. This results in disks that have more tracks on outer cylinders than on the inner cylinders and, last but not least, have more capacity. You can imagine that the value reported by the drive when inquiring about the geometry now becomes suspect at best, and nearly always misleading. When asked for a geometry, it is nearly always better to supply the geometry used by the BIOS, or if the BIOS is never going to know about this disk, (e.g. it is not a booting disk) to supply a fictitious geometry that is convenient. SCSI subsystem design FreeBSD uses a layered SCSI subsystem. For each different controller card a device driver is written. This driver knows all the intimate details about the hardware it controls. The driver has a interface to the upper layers of the SCSI subsystem through which it receives its commands and reports back any status. On top of the card drivers there are a number of more generic drivers for a class of devices. More specific: a driver for tape devices (abbreviation: sa, for serial access), magnetic disks (da, for direct access), CDROMs (cd) etc. In case you are wondering where you can find this stuff, it all lives in /sys/cam/scsi. See the man pages in section 4 for more details. The multi level design allows a decoupling of low-level bit banging and more high level stuff. Adding support for another piece of hardware is a much more manageable problem. Kernel configuration Dependent on your hardware, the kernel configuration file must contain one or more lines describing your host adapter(s). This includes I/O addresses, interrupts etc. Consult the manual page for your adapter driver to get more info. Apart from that, check out /sys/i386/conf/LINT for an overview of a kernel config file. LINT contains every possible option you can dream of. It does not imply LINT will actually get you to a working kernel at all. Although it is probably stating the obvious: the kernel config file should reflect your actual hardware setup. So, interrupts, I/O addresses etc must match the kernel config file. During system boot messages will be displayed to indicate whether the configured hardware was actually found. Note that most of the EISA/PCI drivers (namely ahb, ahc, ncr and amd will automatically obtain the correct parameters from the host adapters themselves at boot time; thus, you just need to write, for instance, controller ahc0. An example loosely based on the FreeBSD 2.2.5-Release kernel config file LINT with some added comments (between []): # SCSI host adapters: `aha', `ahb', `aic', `bt', `nca' # # aha: Adaptec 154x # ahb: Adaptec 174x # ahc: Adaptec 274x/284x/294x # aic: Adaptec 152x and sound cards using the Adaptec AIC-6360 (slow!) # amd: AMD 53c974 based SCSI cards (e.g., Tekram DC-390 and 390T) # bt: Most Buslogic controllers # nca: ProAudioSpectrum cards using the NCR 5380 or Trantor T130 # ncr: NCR/Symbios 53c810/815/825/875 etc based SCSI cards # uha: UltraStore 14F and 34F # sea: Seagate ST01/02 8 bit controller (slow!) # wds: Western Digital WD7000 controller (no scatter/gather!). # [For an Adaptec AHA274x/284x/294x/394x etc controller] controller ahc0 [For an NCR/Symbios 53c875 based controller] controller ncr0 [For an Ultrastor adapter] controller uha0 at isa? port "IO_UHA0" bio irq ? drq 5 vector uhaintr # Map SCSI buses to specific SCSI adapters controller scbus0 at ahc0 controller scbus2 at ncr0 controller scbus1 at uha0 # The actual SCSI devices disk da0 at scbus0 target 0 unit 0 [SCSI disk 0 is at scbus 0, LUN 0] disk da1 at scbus0 target 1 [implicit LUN 0 if omitted] disk da2 at scbus1 target 3 [SCSI disk on the uha0] disk da3 at scbus2 target 4 [SCSI disk on the ncr0] tape sa1 at scbus0 target 6 [SCSI tape at target 6] device cd0 at scbus? [the first ever CDROM found, no wiring] The example above tells the kernel to look for a ahc (Adaptec 274x) controller, then for an NCR/Symbios board, and so on. The lines following the controller specifications tell the kernel to configure specific devices but only attach them when they match the target ID and LUN specified on the corresponding bus. Wired down devices get first shot at the unit numbers so the first non wired down device, is allocated the unit number one greater than the highest wired down unit number for that kind of device. So, if you had a SCSI tape at target ID 2 it would be configured as sa2, as the tape at target ID 6 is wired down to unit number 1. Wired down devices need not be found to get their unit number. The unit number for a wired down device is reserved for that device, even if it is turned off at boot time. This allows the device to be turned on and brought on-line at a later time, without rebooting. Notice that a device's unit number has no relationship with its target ID on the SCSI bus. Below is another example of a kernel config file as used by FreeBSD version < 2.0.5. The difference with the first example is that devices are not wired down. Wired down means that you specify which SCSI target belongs to which device. A kernel built to the config file below will attach the first SCSI disk it finds to da0, the second disk to da1 etc. If you ever removed or added a disk, all other devices of the same type (disk in this case) would move around. This implies you have to change /etc/fstab each time. Although the old style still works, you are strongly recommended to use this new feature. It will save you a lot of grief whenever you shift your hardware around on the SCSI buses. So, when you re-use your old trusty config file after upgrading from a pre-FreeBSD2.0.5.R system check this out. [driver for Adaptec 174x] controller ahb0 at isa? bio irq 11 vector ahbintr [for Adaptec 154x] controller aha0 at isa? port "IO_AHA0" bio irq 11 drq 5 vector ahaintr [for Seagate ST01/02] controller sea0 at isa? bio irq 5 iomem 0xc8000 iosiz 0x2000 vector seaintr controller scbus0 device da0 [support for 4 SCSI harddisks, da0 up da3] device sa0 [support for 2 SCSI tapes] [for the CDROM] device cd0 #Only need one of these, the code dynamically grows Both examples support SCSI disks. If during boot more devices of a specific type (e.g. da disks) are found than are configured in the booting kernel, the system will simply allocate more devices, incrementing the unit number starting at the last number wired down. If there are no wired down devices then counting starts at unit 0. Use man 4 scsi to check for the latest info on the SCSI subsystem. For more detailed info on host adapter drivers use e.g., man 4 ahc for info on the Adaptec 294x driver. Tuning your SCSI kernel setup Experience has shown that some devices are slow to respond to INQUIRY commands after a SCSI bus reset (which happens at boot time). An INQUIRY command is sent by the kernel on boot to see what kind of device (disk, tape, CDROM etc.) is connected to a specific target ID. This process is called device probing by the way. To work around the slow response problem, FreeBSD allows a tunable delay time before the SCSI devices are probed following a SCSI bus reset. You can set this delay time in your kernel configuration file using a line like: options SCSI_DELAY=15 #Be pessimistic about Joe SCSI device This line sets the delay time to 15 seconds. On my own system I had to use 3 seconds minimum to get my trusty old CDROM drive to be recognized. Start with a high value (say 30 seconds or so) when you have problems with device recognition. If this helps, tune it back until it just stays working. Rogue SCSI devices Although the SCSI standard tries to be complete and concise, it is a complex standard and implementing things correctly is no easy task. Some vendors do a better job then others. This is exactly where the rogue devices come into view. Rogues are devices that are recognized by the FreeBSD kernel as behaving slightly (...) non-standard. Rogue devices are reported by the kernel when booting. An example for two of my cartridge tape units: Feb 25 21:03:34 yedi /kernel: ahb0 targ 5 lun 0: <TANDBERG TDC 3600 -06:> Feb 25 21:03:34 yedi /kernel: sa0: Tandberg tdc3600 is a known rogue Mar 29 21:16:37 yedi /kernel: aha0 targ 5 lun 0: <ARCHIVE VIPER 150 21247-005> Mar 29 21:16:37 yedi /kernel: sa1: Archive Viper 150 is a known rogue For instance, there are devices that respond to all LUNs on a certain target ID, even if they are actually only one device. It is easy to see that the kernel might be fooled into believing that there are 8 LUNs at that particular target ID. The confusion this causes is left as an exercise to the reader. The SCSI subsystem of FreeBSD recognizes devices with bad habits by looking at the INQUIRY response they send when probed. Because the INQUIRY response also includes the version number of the device firmware, it is even possible that for different firmware versions different workarounds are used. See e.g. /sys/cam/scsi/scsi_sa.c and /sys/cam/scsi/scsi_all.c for more info on how this is done. This scheme works fine, but keep in mind that it of course only works for devices that are known to be weird. If you are the first to connect your bogus Mumbletech SCSI CDROM you might be the one that has to define which workaround is needed. After you got your Mumbletech working, please send the required workaround to the FreeBSD development team for inclusion in the next release of FreeBSD. Other Mumbletech owners will be grateful to you. Multiple LUN devices In some cases you come across devices that use multiple logical units (LUNs) on a single SCSI ID. In most cases FreeBSD only probes devices for LUN 0. An example are so called bridge boards that connect 2 non-SCSI hard disks to a SCSI bus (e.g. an Emulex MD21 found in old Sun systems). This means that any devices with LUNs != 0 are not normally found during device probe on system boot. To work around this problem you must add an appropriate entry in /sys/cam/scsi and rebuild your kernel. Look for a struct that is initialized like below: (FIXME: which file? Do these entries still exist in this form now that we use CAM?) { T_DIRECT, T_FIXED, "MAXTOR", "XT-4170S", "B5A", "mx1", SC_ONE_LU } For your Mumbletech BRIDGE2000 that has more than one LUN, acts as a SCSI disk and has firmware revision 123 you would add something like: { T_DIRECT, T_FIXED, "MUMBLETECH", "BRIDGE2000", "123", "da", SC_MORE_LUS } The kernel on boot scans the inquiry data it receives against the table and acts accordingly. See the source for more info. Tagged command queuing Modern SCSI devices, particularly magnetic disks, support what is called tagged command queuing (TCQ). In a nutshell, TCQ allows the device to have multiple I/O requests outstanding at the same time. Because the device is intelligent, it can optimize its operations (like head positioning) based on its own request queue. On SCSI devices like RAID (Redundant Array of Independent Disks) arrays the TCQ function is indispensable to take advantage of the device's inherent parallelism. Each I/O request is uniquely identified by a tag (hence the name tagged command queuing) and this tag is used by FreeBSD to see which I/O in the device drivers queue is reported as complete by the device. It should be noted however that TCQ requires device driver support and that some devices implemented it not quite right in their firmware. This problem bit me once, and it leads to highly mysterious problems. In such cases, try to disable TCQ. Bus-master host adapters Most, but not all, SCSI host adapters are bus mastering controllers. This means that they can do I/O on their own without putting load onto the host CPU for data movement. This is of course an advantage for a multitasking operating system like FreeBSD. It must be noted however that there might be some rough edges. For instance an Adaptec 1542 controller can be set to use different transfer speeds on the host bus (ISA or AT in this case). The controller is settable to different rates because not all motherboards can handle the higher speeds. Problems like hang-ups, bad data etc might be the result of using a higher data transfer rate then your motherboard can stomach. The solution is of course obvious: switch to a lower data transfer rate and try if that works better. In the case of a Adaptec 1542, there is an option that can be put into the kernel config file to allow dynamic determination of the right, read: fastest feasible, transfer rate. This option is disabled by default: options "TUNE_1542" #dynamic tune of bus DMA speed Check the manual pages for the host adapter that you use. Or better still, use the ultimate documentation (read: driver source). Tracking down problems The following list is an attempt to give a guideline for the most common SCSI problems and their solutions. It is by no means complete. Check for loose connectors and cables. Check and double check the location and number of your terminators. Check if your bus has at least one supplier of terminator power (especially with external terminators. Check if no double target IDs are used. Check if all devices to be used are powered up. Make a minimal bus config with as little devices as possible. If possible, configure your host adapter to use slow bus speeds. Disable tagged command queuing to make things as simple as possible (for a NCR host adapter based system see man ncrcontrol) If you can compile a kernel, make one with the SCSIDEBUG option, and try accessing the device with debugging turned on for that device. If your device does not even probe at startup, you may have to define the address of the device that is failing, and the desired debug level in /sys/cam/cam_debug.h. If it probes but just does not work, you can use the &man.camcontrol.8; command to dynamically set a debug level to it in a running kernel (if CAMDEBUG is defined). This will give you copious debugging output with which to confuse the gurus. See man camcontrol for more exact information. Also look at man 4 pass. Further reading If you intend to do some serious SCSI hacking, you might want to have the official standard at hand: Approved American National Standards can be purchased from ANSI at
13th Floor 11 West 42nd Street New York NY 10036 Sales Dept: (212) 642-4900
You can also buy many ANSI standards and most committee draft documents from Global Engineering Documents,
15 Inverness Way East Englewood CO, 80112-5704 Phone: (800) 854-7179 Outside USA and Canada: (303) 792-2181 Fax: (303) 792- 2192
Many X3T10 draft documents are available electronically on the SCSI BBS (719-574-0424) and on the ncrinfo.ncr.com anonymous FTP site. Latest X3T10 committee documents are: AT Attachment (ATA or IDE) [X3.221-1994] (Approved) ATA Extensions (ATA-2) [X3T10/948D Rev 2i] Enhanced Small Device Interface (ESDI) [X3.170-1990/X3.170a-1991] (Approved) Small Computer System Interface — 2 (SCSI-2) [X3.131-1994] (Approved) SCSI-2 Common Access Method Transport and SCSI Interface Module (CAM) [X3T10/792D Rev 11] Other publications that might provide you with additional information are: SCSI: Understanding the Small Computer System Interface, written by NCR Corporation. Available from: Prentice Hall, Englewood Cliffs, NJ, 07632 Phone: (201) 767-5937 ISBN 0-13-796855-8 Basics of SCSI, a SCSI tutorial written by Ancot Corporation Contact Ancot for availability information at: Phone: (415) 322-5322 Fax: (415) 322-0455 SCSI Interconnection Guide Book, an AMP publication (dated 4/93, Catalog 65237) that lists the various SCSI connectors and suggests cabling schemes. Available from AMP at (800) 522-6752 or (717) 564-0100 Fast Track to SCSI, A Product Guide written by Fujitsu. Available from: Prentice Hall, Englewood Cliffs, NJ, 07632 Phone: (201) 767-5937 ISBN 0-13-307000-X The SCSI Bench Reference, The SCSI Encyclopedia, and the SCSI Tutor, ENDL Publications, 14426 Black Walnut Court, Saratoga CA, 95070 Phone: (408) 867-6642 Zadian SCSI Navigator (quick ref. book) and Discover the Power of SCSI (First book along with a one-hour video and tutorial book), Zadian Software, Suite 214, 1210 S. Bascom Ave., San Jose, CA 92128, (408) 293-0800 On Usenet the newsgroups comp.periphs.scsi and comp.periphs are noteworthy places to look for more info. You can also find the SCSI-FAQ there, which is posted periodically. Most major SCSI device and host adapter suppliers operate FTP sites and/or BBS systems. They may be valuable sources of information about the devices you own. * Disk/tape controllers * SCSI * IDE * Floppy Hard drives SCSI hard drives Contributed by &a.asami;. 17 February 1998. As mentioned in the SCSI section, virtually all SCSI hard drives sold today are SCSI-2 compliant and thus will work fine as long as you connect them to a supported SCSI host adapter. Most problems people encounter are either due to badly designed cabling (cable too long, star topology, etc.), insufficient termination, or defective parts. Please refer to the SCSI section first if your SCSI hard drive is not working. However, there are a couple of things you may want to take into account before you purchase SCSI hard drives for your system. Rotational speed Rotational speeds of SCSI drives sold today range from around 4,500RPM to 15,000RPM. Most of them are either 7,200RPM or 10,000RPM, with 15,000RPM becoming affordable (June 2002). Even though the 10,000RPM drives can generally transfer data faster, they run considerably hotter than their 7,200RPM counterparts. A large fraction of today's disk drive malfunctions are heat-related. If you do not have very good cooling in your PC case, you may want to stick with 7,200RPM or slower drives. Note that newer drives, with higher areal recording densities, can deliver much more bits per rotation than older ones. Today's top-of-line 7,200RPM drives can sustain a throughput comparable to 10,000RPM drives of one or two model generations ago. The number to find on the spec sheet for bandwidth is internal data (or transfer) rate. It is usually in megabits/sec so divide it by 8 and you will get the rough approximation of how much megabytes/sec you can get out of the drive. (If you are a speed maniac and want a 15,000RPM drive for your cute little PC, be my guest; however, those drives become extremely hot. Do not even think about it if you do not have a fan blowing air directly at the drive or a properly ventilated disk enclosure.) Obviously, the latest 15,000RPM drives and 10,000RPM drives can deliver more data than the latest 7,200RPM drives, so if absolute bandwidth is the necessity for your applications, you have little choice but to get the faster drives. Also, if you need low latency, faster drives are better; not only do they usually have lower average seek times, but also the rotational delay is one place where slow-spinning drives can never beat a faster one. (The average rotational latency is half the time it takes to rotate the drive once; thus, it is 2 milliseconds for 15,000RPM, 3ms for 10,000RPM drives, 4.2ms for 7,200RPM drives and 5.6ms for 5,400RPM drives.) Latency is seek time plus rotational delay. Make sure you understand whether you need low latency or more accesses per second, though; in the latter case (e.g., news servers), it may not be optimal to purchase one big fast drive. You can achieve similar or even better results by using the ccd (concatenated disk) driver to create a striped disk array out of multiple slower drives for comparable overall cost. Make sure you have adequate air flow around the drive, especially if you are going to use a fast-spinning drive. You generally need at least 1/2” (1.25cm) of spacing above and below a drive. Understand how the air flows through your PC case. Most cases have the power supply suck the air out of the back. See where the air flows in, and put the drive where it will have the largest volume of cool air flowing around it. You may need to seal some unwanted holes or add a new fan for effective cooling. Another consideration is noise. Many 10,000 or faster drives generate a high-pitched whine which is quite unpleasant to most people. That, plus the extra fans often required for cooling, may make 10,000 or faster drives unsuitable for some office and home environments. Form factor Most SCSI drives sold today are of 3.5” form factor. They come in two different heights; 1.6” (half-height) or 1” (low-profile). The half-height drive is the same height as a CDROM drive. However, do not forget the spacing rule mentioned in the previous section. If you have three standard 3.5” drive bays, you will not be able to put three half-height drives in there (without frying them, that is). Interface The majority of SCSI hard drives sold today are Ultra, Ultra-wide, or Ultra160 SCSI. As of this writing (June 2002), the first Ultra320 host adapters and devices become available. The maximum bandwidth of Ultra SCSI is 20MB/sec, and Ultra-wide SCSI is 40MB/sec. Ultra160 can transfer 160MB/sec and Ultra320 can transfer 320MB/sec. There is no difference in max cable length between Ultra and Ultra-wide; however, the more devices you have on the same bus, the sooner you will start having bus integrity problems. Unless you have a well-designed disk enclosure, it is not easy to make more than 5 or 6 Ultra SCSI drives work on a single bus. On the other hand, if you need to connect many drives, going for Fast-wide SCSI may not be a bad idea. That will have the same max bandwidth as Ultra (narrow) SCSI, while electronically it is much easier to get it right. My advice would be: if you want to connect many disks, get wide or Ultra160 SCSI drives; they usually cost a little more but it may save you down the road. (Besides, if you can not afford the cost difference, you should not be building a disk array.) There are two variant of wide SCSI drives; 68-pin and 80-pin SCA (Single Connector Attach). The SCA drives do not have a separate 4-pin power connector, and also read the SCSI ID settings through the 80-pin connector. If you are really serious about building a large storage system, get SCA drives and a good SCA enclosure (dual power supply with at least one extra fan). They are more electronically sound than 68-pin counterparts because there is no stub of the SCSI bus inside the disk canister as in arrays built from 68-pin drives. They are easier to install too (you just need to screw the drive in the canister, instead of trying to squeeze in your fingers in a tight place to hook up all the little cables (like the SCSI ID and disk activity LED lines). * IDE hard drives Tape drives Contributed by &a.jmb;. 2 July 1996. General tape access commands &man.mt.1; provides generic access to the tape drives. Some of the more common commands are rewind, erase, and status. See the &man.mt.1; manual page for a detailed description. Controller Interfaces There are several different interfaces that support tape drives. The interfaces are SCSI, IDE, Floppy and Parallel Port. A wide variety of tape drives are available for these interfaces. Controllers are discussed in Disk/tape controllers. SCSI drives The &man.st.4; driver provides support for 8mm (Exabyte), 4mm (DAT: Digital Audio Tape), QIC (Quarter-Inch Cartridge), DLT (Digital Linear Tape), QIC Mini cartridge and 9-track (remember the big reels that you see spinning in Hollywood computer rooms) tape drives. See the &man.st.4; manual page for a detailed description. The drives listed below are currently being used by members of the FreeBSD community. They are not the only drives that will work with FreeBSD. They just happen to be the ones that we use. 4mm (DAT: Digital Audio Tape) Archive Python 28454 Archive Python 04687 HP C1533A HP C1534A HP 35450A HP 35470A HP 35480A SDT-5000 Wangtek 6200 8mm (Exabyte) EXB-8200 EXB-8500 EXB-8505 QIC (Quarter-Inch Cartridge) Archive Anaconda 2750 Archive Viper 60 Archive Viper 150 Archive Viper 2525 Tandberg TDC 3600 Tandberg TDC 3620 Tandberg TDC 3800 Tandberg TDC 4222 Wangtek 5525ES DLT (Digital Linear Tape) Digital TZ87 Mini-Cartridge Conner CTMS 3200 Exabyte 2501 Autoloaders/Changers Hewlett-Packard HP C1553A Autoloading DDS2 * IDE drives Floppy drives Conner 420R * Parallel port drives Detailed Information Archive Anaconda 2750 The boot message identifier for this drive is ARCHIVE ANCDA 2750 28077 -003 type 1 removable SCSI 2 This is a QIC tape drive. Native capacity is 1.35GB when using QIC-1350 tapes. This drive will read and write QIC-150 (DC6150), QIC-250 (DC6250), and QIC-525 (DC6525) tapes as well. Data transfer rate is 350kB/s using &man.dump.8;. Rates of 530kB/s have been reported when using Amanda Production of this drive has been discontinued. The SCSI bus connector on this tape drive is reversed from that on most other SCSI devices. Make sure that you have enough SCSI cable to twist the cable one-half turn before and after the Archive Anaconda tape drive, or turn your other SCSI devices upside-down. Two kernel code changes are required to use this drive. This drive will not work as delivered. If you have a SCSI-2 controller, short jumper 6. Otherwise, the drive behaves are a SCSI-1 device. When operating as a SCSI-1 device, this drive, locks the SCSI bus during some tape operations, including: fsf, rewind, and rewoffl. If you are using the NCR SCSI controllers, patch the file /usr/src/sys/pci/ncr.c (as shown below). Build and install a new kernel. *** 4831,4835 **** }; ! if (np->latetime>4) { /* ** Although we tried to wake it up, --- 4831,4836 ---- }; ! if (np->latetime>1200) { /* ** Although we tried to wake it up, Reported by: &a.jmb; Archive Python 28454 The boot message identifier for this drive is ARCHIVE Python 28454-XXX4ASB type 1 removable SCSI 2 density code 0x8c, 512-byte blocks This is a DDS-1 tape drive. Native capacity is 2.5GB on 90m tapes. Data transfer rate is XXX. This drive was repackaged by Sun Microsystems as model 595-3067. Reported by: Bob Bishop rb@gid.co.uk Throughput is in the 1.5 MByte/sec range, however this will drop if the disks and tape drive are on the same SCSI controller. Reported by: Robert E. Seastrom rs@seastrom.com Archive Python 04687 The boot message identifier for this drive is ARCHIVE Python 04687-XXX 6580 Removable Sequential Access SCSI-2 device This is a DAT-DDS-2 drive. Native capacity is 4GB when using 120m tapes. This drive supports hardware data compression. Switch 4 controls MRS (Media Recognition System). MRS tapes have stripes on the transparent leader. Switch 4 off enables MRS, on disables MRS. Parity is controlled by switch 5. Switch 5 on to enable parity control. Compression is enabled with Switch 6 off. It is possible to override compression with the SCSI MODE SELECT command (see &man.mt.1;). Data transfer rate is 800kB/s. Archive Viper 60 The boot message identifier for this drive is ARCHIVE VIPER 60 21116 -007 type 1 removable SCSI 1 This is a QIC tape drive. Native capacity is 60MB. Data transfer rate is XXX. Production of this drive has been discontinued. Reported by: Philippe Regnauld regnauld@hsc.fr Archive Viper 150 The boot message identifier for this drive is ARCHIVE VIPER 150 21531 -004 Archive Viper 150 is a known rogue type 1 removable SCSI 1. A multitude of firmware revisions exist for this drive. Your drive may report different numbers (e.g 21247 -005. This is a QIC tape drive. Native capacity is 150/250MB. Both 150MB (DC6150) and 250MB (DC6250) tapes have the recording format. The 250MB tapes are approximately 67% longer than the 150MB tapes. This drive can read 120MB tapes as well. It can not write 120MB tapes. Data transfer rate is 100kB/s This drive reads and writes DC6150 (150MB) and DC6250 (250MB) tapes. This drives quirks are known and pre-compiled into the SCSI tape device driver (&man.st.4;). Under FreeBSD 2.2-CURRENT, use mt blocksize 512 to set the blocksize. (The particular drive had firmware revision 21247 -005. Other firmware revisions may behave differently) Previous versions of FreeBSD did not have this problem. Production of this drive has been discontinued. Reported by: Pedro A M Vazquez vazquez@IQM.Unicamp.BR &a.msmith; Archive Viper 2525 The boot message identifier for this drive is ARCHIVE VIPER 2525 25462 -011 type 1 removable SCSI 1 This is a QIC tape drive. Native capacity is 525MB. Data transfer rate is 180kB/s at 90 inches/sec. The drive reads QIC-525, QIC-150, QIC-120 and QIC-24 tapes. Writes QIC-525, QIC-150, and QIC-120. Firmware revisions prior to 25462 -011 are bug ridden and will not function properly. Production of this drive has been discontinued. Conner 420R The boot message identifier for this drive is Conner tape. This is a floppy controller, mini cartridge tape drive. Native capacity is XXXX Data transfer rate is XXX The drive uses QIC-80 tape cartridges. Reported by: Mark Hannon mark@seeware.DIALix.oz.au Conner CTMS 3200 The boot message identifier for this drive is CONNER CTMS 3200 7.00 type 1 removable SCSI 2. This is a mini cartridge tape drive. Native capacity is XXXX Data transfer rate is XXX The drive uses QIC-3080 tape cartridges. Reported by: Thomas S. Traylor tst@titan.cs.mci.com <ulink url="http://www.digital.com/info/Customer-Update/931206004.txt.html">DEC TZ87</ulink> The boot message identifier for this drive is DEC TZ87 (C) DEC 9206 type 1 removable SCSI 2 density code 0x19 This is a DLT tape drive. Native capacity is 10GB. This drive supports hardware data compression. Data transfer rate is 1.2MB/s. This drive is identical to the Quantum DLT2000. The drive firmware can be set to emulate several well-known drives, including an Exabyte 8mm drive. Reported by: &a.wilko; <ulink url="http://www.Exabyte.COM:80/Products/Minicartridge/2501/Rfeatures.html">Exabyte EXB-2501</ulink> The boot message identifier for this drive is EXABYTE EXB-2501 This is a mini-cartridge tape drive. Native capacity is 1GB when using MC3000XL mini cartridges. Data transfer rate is XXX This drive can read and write DC2300 (550MB), DC2750 (750MB), MC3000 (750MB), and MC3000XL (1GB) mini cartridges. WARNING: This drive does not meet the SCSI-2 specifications. The drive locks up completely in response to a SCSI MODE_SELECT command unless there is a formatted tape in the drive. Before using this drive, set the tape blocksize with &prompt.root; mt -f /dev/st0ctl.0 blocksize 1024 Before using a mini cartridge for the first time, the - mini cartridge must be formated. FreeBSD 2.1.0-RELEASE and + mini cartridge must be formatted. FreeBSD 2.1.0-RELEASE and earlier: &prompt.root; /sbin/scsi -f /dev/rst0.ctl -s 600 -c "4 0 0 0 0 0" (Alternatively, fetch a copy of the scsiformat shell script from FreeBSD 2.1.5/2.2.) FreeBSD 2.1.5 and later: &prompt.root; /sbin/scsiformat -q -w /dev/rst0.ctl Right now, this drive cannot really be recommended for FreeBSD. Reported by: Bob Beaulieu ez@eztravel.com Exabyte EXB-8200 The boot message identifier for this drive is EXABYTE EXB-8200 252X type 1 removable SCSI 1 This is an 8mm tape drive. Native capacity is 2.3GB. Data transfer rate is 270kB/s. This drive is fairly slow in responding to the SCSI bus during boot. A custom kernel may be required (set SCSI_DELAY to 10 seconds). There are a large number of firmware configurations for this drive, some have been customized to a particular vendor's hardware. The firmware can be changed via EPROM replacement. Production of this drive has been discontinued. Reported by: &a.msmith; Exabyte EXB-8500 The boot message identifier for this drive is EXABYTE EXB-8500-85Qanx0 0415 type 1 removable SCSI 2 This is an 8mm tape drive. Native capacity is 5GB. Data transfer rate is 300kB/s. Reported by: Greg Lehey grog@lemis.de <ulink url="http://www.Exabyte.COM:80/Products/8mm/8505XL/Rfeatures.html">Exabyte EXB-8505</ulink> The boot message identifier for this drive is EXABYTE EXB-85058SQANXR1 05B0 type 1 removable SCSI 2 This is an 8mm tape drive which supports compression, and is upward compatible with the EXB-5200 and EXB-8500. Native capacity is 5GB. The drive supports hardware data compression. Data transfer rate is 300kB/s. Reported by: Glen Foster gfoster@gfoster.com Hewlett-Packard HP C1533A The boot message identifier for this drive is HP C1533A 9503 type 1 removable SCSI 2. This is a DDS-2 tape drive. DDS-2 means hardware data compression and narrower tracks for increased data capacity. Native capacity is 4GB when using 120m tapes. This drive supports hardware data compression. Data transfer rate is 510kB/s. This drive is used in Hewlett-Packard's SureStore 6000eU and 6000i tape drives and C1533A DDS-2 DAT drive. The drive has a block of 8 dip switches. The proper settings for FreeBSD are: 1 ON; 2 ON; 3 OFF; 4 ON; 5 ON; 6 ON; 7 ON; 8 ON. switch 1 switch 2 Result On On Compression enabled at power-on, with host control On Off Compression enabled at power-on, no host control Off On Compression disabled at power-on, with host control Off Off Compression disabled at power-on, no host control Switch 3 controls MRS (Media Recognition System). MRS tapes have stripes on the transparent leader. These identify the tape as DDS (Digital Data Storage) grade media. Tapes that do not have the stripes will be treated as write-protected. Switch 3 OFF enables MRS. Switch 3 ON disables MRS. See HP SureStore Tape Products and Hewlett-Packard Disk and Tape Technical Information for more information on configuring this drive. Warning: Quality control on these drives varies greatly. One FreeBSD core-team member has returned 2 of these drives. Neither lasted more than 5 months. Reported by: &a.se; Hewlett-Packard HP 1534A The boot message identifier for this drive is HP HP35470A T503 type 1 removable SCSI 2 Sequential-Access density code 0x13, variable blocks. This is a DDS-1 tape drive. DDS-1 is the original DAT tape format. Native capacity is 2GB when using 90m tapes. Data transfer rate is 183kB/s. The same mechanism is used in Hewlett-Packard's SureStore 2000i tape drive, C35470A DDS format DAT drive, C1534A DDS format DAT drive and HP C1536A DDS format DAT drive. The HP C1534A DDS format DAT drive has two indicator lights, one green and one amber. The green one indicates tape action: slow flash during load, steady when loaded, fast flash during read/write operations. The amber one indicates warnings: slow flash when cleaning is required or tape is nearing the end of its useful life, steady indicates an hard fault. (factory service required?) Reported by Gary Crutcher gcrutchr@nightflight.com Hewlett-Packard HP C1553A Autoloading DDS2 The boot message identifier for this drive is "". This is a DDS-2 tape drive with a tape changer. DDS-2 means hardware data compression and narrower tracks for increased data capacity. Native capacity is 24GB when using 120m tapes. This drive supports hardware data compression. Data transfer rate is 510kB/s (native). This drive is used in Hewlett-Packard's SureStore 12000e tape drive. The drive has two selectors on the rear panel. The selector closer to the fan is SCSI id. The other selector should be set to 7. There are four internal switches. These should be set: 1 ON; 2 ON; 3 ON; 4 OFF. At present the kernel drivers do not automatically change tapes at the end of a volume. This shell script can be used to change tapes: #!/bin/sh PATH="/sbin:/usr/sbin:/bin:/usr/bin"; export PATH usage() { echo "Usage: dds_changer [123456ne] raw-device-name echo "1..6 = Select cartridge" echo "next cartridge" echo "eject magazine" exit 2 } if [ $# -ne 2 ] ; then usage fi cdb3=0 cdb4=0 cdb5=0 case $1 in [123456]) cdb3=$1 cdb4=1 ;; n) ;; e) cdb5=0x80 ;; ?) usage ;; esac scsi -f $2 -s 100 -c "1b 0 0 $cdb3 $cdb4 $cdb5" Hewlett-Packard HP 35450A The boot message identifier for this drive is HP HP35450A -A C620 type 1 removable SCSI 2 Sequential-Access density code 0x13 This is a DDS-1 tape drive. DDS-1 is the original DAT tape format. Native capacity is 1.2GB. Data transfer rate is 160kB/s. Reported by: Mark Thompson mark.a.thompson@pobox.com Hewlett-Packard HP 35470A The boot message identifier for this drive is HP HP35470A 9 09 type 1 removable SCSI 2 This is a DDS-1 tape drive. DDS-1 is the original DAT tape format. Native capacity is 2GB when using 90m tapes. Data transfer rate is 183kB/s. The same mechanism is used in Hewlett-Packard's SureStore 2000i tape drive, C35470A DDS format DAT drive, C1534A DDS format DAT drive, and HP C1536A DDS format DAT drive. Warning: Quality control on these drives varies greatly. One FreeBSD core-team member has returned 5 of these drives. None lasted more than 9 months. Reported by: David Dawes dawes@rf900.physics.usyd.edu.au (9 09) Hewlett-Packard HP 35480A The boot message identifier for this drive is HP HP35480A 1009 type 1 removable SCSI 2 Sequential-Access density code 0x13. This is a DDS-DC tape drive. DDS-DC is DDS-1 with hardware data compression. DDS-1 is the original DAT tape format. Native capacity is 2GB when using 90m tapes. It cannot handle 120m tapes. This drive supports hardware data compression. Please refer to the section on HP C1533A for the proper switch settings. Data transfer rate is 183kB/s. This drive is used in Hewlett-Packard's SureStore 5000eU and 5000i tape drives and C35480A DDS format DAT drive.. This drive will occasionally hang during a tape eject operation (mt offline). Pressing the front panel button will eject the tape and bring the tape drive back to life. WARNING: HP 35480-03110 only. On at least two occasions this tape drive when used with FreeBSD 2.1.0, an IBM Server 320 and an 2940W SCSI controller resulted in all SCSI disk partitions being lost. The problem has not be analyzed or resolved at this time. <ulink url="http://www.sel.sony.com/SEL/ccpg/storage/tape/t5000.html">Sony SDT-5000</ulink> There are at least two significantly different models: one is a DDS-1 and the other DDS-2. The DDS-1 version is SDT-5000 3.02. The DDS-2 version is SONY SDT-5000 327M. The DDS-2 version has a 1MB cache. This cache is able to keep the tape streaming in almost any circumstances. The boot message identifier for this drive is SONY SDT-5000 3.02 type 1 removable SCSI 2 Sequential-Access density code 0x13 Native capacity is 4GB when using 120m tapes. This drive supports hardware data compression. Data transfer rate is depends upon the model or the drive. The rate is 630kB/s for the SONY SDT-5000 327M while compressing the data. For the SONY SDT-5000 3.02, the data transfer rate is 225kB/s. In order to get this drive to stream, set the blocksize to 512 bytes (mt blocksize 512) reported by Kenneth Merry ken@ulc199.residence.gatech.edu. SONY SDT-5000 327M information reported by Charles Henrich henrich@msu.edu. Reported by: &a.jmz; Tandberg TDC 3600 The boot message identifier for this drive is TANDBERG TDC 3600 =08: type 1 removable SCSI 2 This is a QIC tape drive. Native capacity is 150/250MB. This drive has quirks which are known and work around code is present in the SCSI tape device driver (&man.st.4;). Upgrading the firmware to XXX version will fix the quirks and provide SCSI 2 capabilities. Data transfer rate is 80kB/s. IBM and Emerald units will not work. Replacing the firmware EPROM of these units will solve the problem. Reported by: &a.msmith; Tandberg TDC 3620 This is very similar to the Tandberg TDC 3600 drive. Reported by: &a.joerg; Tandberg TDC 3800 The boot message identifier for this drive is TANDBERG TDC 3800 =04Y Removable Sequential Access SCSI-2 device This is a QIC tape drive. Native capacity is 525MB. Reported by: &a.jhs; Tandberg TDC 4222 The boot message identifier for this drive is TANDBERG TDC 4222 =07 type 1 removable SCSI 2 This is a QIC tape drive. Native capacity is 2.5GB. The drive will read all cartridges from the 60 MB (DC600A) upwards, and write 150 MB (DC6150) upwards. Hardware compression is optionally supported for the 2.5 GB cartridges. This drives quirks are known and pre-compiled into the SCSI tape device driver (&man.st.4;) beginning with FreeBSD 2.2-CURRENT. For previous versions of FreeBSD, use mt to read one block from the tape, rewind the tape, and then execute the backup program (mt fsr 1; mt rewind; dump ...) Data transfer rate is 600kB/s (vendor claim with compression), 350 KB/s can even be reached in start/stop mode. The rate decreases for smaller cartridges. Reported by: &a.joerg; Wangtek 5525ES The boot message identifier for this drive is WANGTEK 5525ES SCSI REV7 3R1 type 1 removable SCSI 1 density code 0x11, 1024-byte blocks This is a QIC tape drive. Native capacity is 525MB. Data transfer rate is 180kB/s. The drive reads 60, 120, 150, and 525MB tapes. The drive will not write 60MB (DC600 cartridge) tapes. In order to overwrite 120 and 150 tapes reliably, first erase (mt erase) the tape. 120 and 150 tapes used a wider track (fewer tracks per tape) than 525MB tapes. The extra width of the previous tracks is not overwritten, as a result the new data lies in a band surrounded on both sides by the previous data unless the tape have been erased. This drives quirks are known and pre-compiled into the SCSI tape device driver (&man.st.4;). Other firmware revisions that are known to work are: M75D Reported by: Marc van Kempen marc@bowtie.nl REV73R1 Andrew Gordon Andrew.Gordon@net-tel.co.uk M75D Wangtek 6200 The boot message identifier for this drive is WANGTEK 6200-HS 4B18 type 1 removable SCSI 2 Sequential-Access density code 0x13 This is a DDS-1 tape drive. Native capacity is 2GB using 90m tapes. Data transfer rate is 150kB/s. Reported by: Tony Kimball alk@Think.COM * Problem drives CDROM drives Contributed by &a.obrien;. 23 November 1997. Generally speaking those in The FreeBSD Project prefer SCSI CDROM drives over IDE CDROM drives. However not all SCSI CDROM drives are equal. Some feel the quality of some SCSI CDROM drives have been deteriorating to that of IDE CDROM drives. Toshiba used to be the favored stand-by, but many on the SCSI mailing list have found displeasure with the 12x speed XM-5701TA as its volume (when playing audio CDROMs) is not controllable by the various audio player software. Another area where SCSI CDROM manufacturers are cutting corners is adherence to the SCSI specification. Many SCSI CDROMs will respond to multiple LUNs for its target address. Known violators include the 6x Teac CD-56S 1.0D.