diff --git a/en_US.ISO8859-1/books/arch-handbook/book.sgml b/en_US.ISO8859-1/books/arch-handbook/book.sgml index 1006c9f5d2..2cf7d725aa 100644 --- a/en_US.ISO8859-1/books/arch-handbook/book.sgml +++ b/en_US.ISO8859-1/books/arch-handbook/book.sgml @@ -1,520 +1,523 @@ %bookinfo; %man; %chapters; + %authors; ]> FreeBSD Developers' Handbook The FreeBSD Documentation Project
doc@FreeBSD.org
August 2000 2000 + 2001 The FreeBSD Documentation Project &bookinfo.legalnotice; Welcome to the Developers' Handbook. Introduction Developing on FreeBSD This will need to discuss FreeBSD as a development platform, the vision of BSD, architectural overview, layout of /usr/src, history, etc. Thank you for considering FreeBSD as your development platform! We hope it will not let you down. The BSD Vision Architectural Overview The Layout of /usr/src The complete source code to FreeBSD is available from our public CVS repository. The source code is normally installed in /usr/src which contains the following subdirectories. Directory Description bin/ Source for files in /bin contrib/ Source for files from contribued software. crypto/ DES source etc/ Source for files in /etc games/ Source for files in /usr/games gnu/ Utilities covered by the GNU Public License include/ Source for files in /usr/include kerberosIV/ Source for Kerbereros version IV kerberos5/ Source for Kerbereros version 5 lib/ Source for files in /usr/lib libexec/ Source for files in /usr/libexec release/ Files required to produce a FreeBSD release sbin/ Source for files in /sbin secure/ FreeSec sources share/ Source for files in /sbin sys/ Kernel source files tools/ Tools used for maintenance and testing of FreeBSD usr.bin/ Source for files in /usr/bin usr.sbin/ Source for files in /usr/sbin Basics &chap.tools; &chap.secure; Kernel History of the Unix Kernel Some history of the Unix/BSD kernel, system calls, how do processes work, blocking, scheduling, threads (kernel), context switching, signals, interrupts, modules, etc. &chap.locking; Memory and Virtual Memory Virtual Memory VM, paging, swapping, allocating memory, testing for memory leaks, mmap, vnodes, etc. I/O System UFS UFS, FFS, Ext2FS, JFS, inodes, buffer cache, labeling, locking, metadata, soft-updates, LFS, portalfs, procfs, vnodes, memory sharing, memory objects, TLBs, caching Interprocess Communication Signals Signals, pipes, semaphores, message queues, shared memory, ports, sockets, doors Networking Sockets Sockets, bpf, IP, TCP, UDP, ICMP, OSI, bridging, firewalling, NAT, switching, etc Network Filesystems AFS AFS, NFS, SANs etc] Terminal Handling Syscons Syscons, tty, PCVT, serial console, screen savers, etc Sound OSS OSS, waveforms, etc Device Drivers &chap.driverbasics; &chap.pci; + &chap.scsi; USB Devices This chapter will talk about the FreeBSD mechanisms for writing a device driver for a device on a USB bus. NewBus This chapter will talk about the FreeBSD NewBus architecture. Architectures IA-32 Talk about the architectural specifics of FreeBSD/x86. Alpha Talk about the architectural specifics of FreeBSD/alpha. Explanation of allignment errors, how to fix, how to ignore. Example assembly language code for FreeBSD/alpha. IA-64 Talk about the architectural specifics of FreeBSD/ia64. Debugging Truss various descriptions on how to debug certain aspects of the system using truss, ktrace, gdb, kgdb, etc Compatibility Layers Linux Linux, SVR4, etc Appendices Dave A Patterson John L Hennessy 1998Morgan Kaufmann Publishers, Inc. 1-55860-428-6 Morgan Kaufmann Publishers, Inc. Computer Organization and Design The Hardware / Software Interface 1-2 W. Richard Stevens 1993Addison Wesley Longman, Inc. 0-201-56317-7 Addison Wesley Longman, Inc. Advanced Programming in the Unix Environment 1-2 Marshall Kirk McKusick Keith Bostic Michael J Karels John S Quarterman 1996Addison-Wesley Publishing Company, Inc. 0-201-54979-4 Addison-Wesley Publishing Company, Inc. The Design and Implementation of the 4.4 BSD Operating System 1-2 Aleph One Phrack 49; "Smashing the Stack for Fun and Profit" Chrispin Cowan Calton Pu Dave Maier StackGuard; Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks Todd Miller Theo de Raadt strlcpy and strlcat -- consistent, safe string copy and concatenation. diff --git a/en_US.ISO8859-1/books/arch-handbook/chapters.ent b/en_US.ISO8859-1/books/arch-handbook/chapters.ent index c2149880e2..a367f40239 100644 --- a/en_US.ISO8859-1/books/arch-handbook/chapters.ent +++ b/en_US.ISO8859-1/books/arch-handbook/chapters.ent @@ -1,58 +1,59 @@ + diff --git a/en_US.ISO8859-1/books/arch-handbook/scsi/chapter.sgml b/en_US.ISO8859-1/books/arch-handbook/scsi/chapter.sgml new file mode 100644 index 0000000000..837a8fd60d --- /dev/null +++ b/en_US.ISO8859-1/books/arch-handbook/scsi/chapter.sgml @@ -0,0 +1,2079 @@ + + + + Common Access Method SCSI Controllers + + This chapter was written by &a.babkin; + Modifications for the handbook made by + &a.murray;. + + + Synopsis + + This document assumes that the reader has a general + understanding of device drivers in FreeBSD and of the SCSI + protocol. Much of the information in this document was + extracted from the drivers : + + + + ncr (/sys/pci/ncr.c) by + Wolfgang Stanglmeier and Stefan Esser + + sym (/sys/pci/sym.c) by + Gerard Roudier + + aic7xxx + (/sys/dev/aic7xxx/aic7xxx.c) by Justin + T. Gibbs + + + + and from the CAM code itself (by Justing T. Gibbs, see + /sys/cam/*). When some solution looked the + most logical and was essentially verbatim extracted from the code + by Justin Gibbs, I marked it as "recommended". + + The document is illustrated with examples in + pseudo-code. Although sometimes the examples have many details + and look like real code, it's still pseudo-code. It was written + to demonstrate the concepts in an understandable way. For a real + driver other approaches may be more modular and efficient. It + also abstracts from the hardware details, as well as issues that + would cloud the demonstrated concepts or that are supposed to be + described in the other chapters of the developers handbook. Such + details are commonly shown as calls to functions with descriptive + names, comments or pseudo-statements. Fortunately real life + full-size examples with all the details can be found in the real + drivers. + + + + + General architecture + + CAM stands for Common Access Method. It's a generic way to + address the I/O buses in a SCSI-like way. This allows a + separation of the generic device drivers from the drivers + controlling the I/O bus: for example the disk driver becomes able + to control disks on both SCSI, IDE, and/or any other bus so the + disk driver portion does not have to be rewritten (or copied and + modified) for every new I/O bus. Thus the two most important + active entities are: + + + Peripheral Modules - a + driver for peripheral devices (disk, tape, CDROM, + etc.) + SCSI Interface Modules (SIM) + - a Host Bus Adapter drivers for connecting to an I/O bus such + as SCSI or IDE. + + + A peripheral driver receives requests from the OS, converts + them to a sequence of SCSI commands and passes these SCSI + commands to a SCSI Interface Module. The SCSI Interface Module + is responsible for passing these commands to the actual hardware + (or if the actual hardware is not SCSI but, for example, IDE + then also converting the SCSI commands to the native commands of + the hardware). + + Because we are interested in writing a SCSI adapter driver + here, from this point on we will consider everything from the + SIM standpoint. + + A typical SIM driver needs to include the following + CAM-related header files: + + +#include <cam/cam.h> +#include <cam/cam_ccb.h> +#include <cam/cam_sim.h> +#include <cam/cam_xpt_sim.h> +#include <cam/cam_debug.h> +#include <cam/scsi/scsi_all.h> + + + The first thing each SIM driver must do is register itself + with the CAM subsystem. This is done during the driver's + xxx_attach() function (here and further + xxx_ is used to denote the unique driver name prefix). The + xxx_attach() function itself is called by + the system bus auto-configuration code which we don't describe + here. + + This is achieved in multiple steps: first it's necessary to + allocate the queue of requests associated with this SIM: + + + struct cam_devq *devq; + + if(( devq = cam_simq_alloc(SIZE) )==NULL) { + error; /* some code to handle the error */ + } + + + Here SIZE is the size of the queue to be allocated, maximal + number of requests it could contain. It's the number of requests + that the SIM driver can handle in parallel on one SCSI + card. Commonly it can be calculated as: + + +SIZE = NUMBER_OF_SUPPORTED_TARGETS * MAX_SIMULTANEOUS_COMMANDS_PER_TARGET + + + Next we create a descriptor of our SIM: + + + struct cam_sim *sim; + + if(( sim = cam_sim_alloc(action_func, poll_func, driver_name, + softc, unit, max_dev_transactions, + max_tagged_dev_transactions, devq) )==NULL) { + cam_simq_free(devq); + error; /* some code to handle the error */ + } + + + Note that if we are not able to create a SIM descriptor we + free the devq also because we can do + nothing else with it and we want to conserve memory. + + If a SCSI card has multiple SCSI buses on it then each bus + requires its own cam_sim + structure. + + An interesting question is what to do if a SCSI card has + more than one SCSI bus, do we need one + devq structure per card or per SCSI + bus? The answer given in the comments to the CAM code is: + either way, as the driver's author prefers. + + The arguments are : + + + action_func - pointer to + the driver's xxx_action function. + + static void + xxx_action + + + struct cam_sim *sim, + union ccb *ccb + + + + + poll_func - pointer to + the driver's xxx_poll() + + static void + xxx_poll + + + struct cam_sim *sim + + + + + driver_name - the name of the actual driver, + such as "ncr" or "wds" + + softc - pointer to the + driver's internal descriptor for this SCSI card. This + pointer will be used by the driver in future to get private + data. + + unit - the controller unit number, for example + for controller "wds0" this number will be + 0 + + max_dev_transactions - maximal number of + simultaneous transactions per SCSI target in the non-tagged + mode. This value will be almost universally equal to 1, with + possible exceptions only for the non-SCSI cards. Also the + drivers that hope to take advantage by preparing one + transaction while another one is executed may set it to 2 + but this does not seem to be worth the + complexity. + + max_tagged_dev_transactions - the same thing, + but in the tagged mode. Tags are the SCSI way to initiate + multiple transactions on a device: each transaction is + assigned a unique tag and the transaction is sent to the + device. When the device completes some transaction it sends + back the result together with the tag so that the SCSI + adapter (and the driver) can tell which transaction was + completed. This argument is also known as the maximal tag + depth. It depends on the abilities of the SCSI + adapter. + + + + Finally we register the SCSI buses associated with our SCSI + adapter: + + + if(xpt_bus_register(sim, bus_number) != CAM_SUCCESS) { + cam_sim_free(sim, /*free_devq*/ TRUE); + error; /* some code to handle the error */ + } + + + If there is one devq structure per + SCSI bus (i.e. we consider a card with multiple buses as + multiple cards with one bus each) then the bus number will + always be 0, otherwise each bus on the SCSI card should be get a + distinct number. Each bus needs its own separate structure + cam_sim. + + After that our controller is completely hooked to the CAM + system. The value of devq can be + discarded now: sim will be passed as an argument in all further + calls from CAM and devq can be derived from it. + + CAM provides the framework for such asynchronous + events. Some events originate from the lower levels (the SIM + drivers), some events originate from the peripheral drivers, + some events originate from the CAM subsystem itself. Any driver + can register callbacks for some types of the asynchronous + events, so that it would be notified if these events + occur. + + A typical example of such an event is a device reset. Each + transaction and event identifies the devices to which it applies + by the means of "path". The target-specific events normally + occur during a transaction with this device. So the path from + that transaction may be re-used to report this event (this is + safe because the event path is copied in the event reporting + routine but not deallocated nor passed anywhere further). Also + it's safe to allocate paths dynamically at any time including + the interrupt routines, although that incurs certain overhead, + and a possible problem with this approach is that there may be + no free memory at that time. For a bus reset event we need to + define a wildcard path including all devices on the bus. So we + can create the path for the future bus reset events in advance + and avoid problems with the future memory shortage: + + + struct cam_path *path; + + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), CAM_TARGET_WILDCARD, + CAM_LUN_WILDCARD) != CAM_REQ_CMP) { + xpt_bus_deregister(cam_sim_path(sim)); + cam_sim_free(sim, /*free_devq*/TRUE); + error; /* some code to handle the error */ + } + + softc->wpath = path; + softc->sim = sim; + + + As you can see the path includes: + + + ID of the peripheral driver (NULL here because we have + none) + + ID of the SIM driver + (cam_sim_path(sim)) + + SCSI target number of the device (CAM_TARGET_WILDCARD + means "all devices") + + SCSI LUN number of the subdevice (CAM_LUN_WILDCARD means + "all LUNs") + + + If the driver can't allocate this path it won't be able to + work normally, so in that case we dismantle that SCSI + bus. + + And we save the path pointer in the + softc structure for future use. After + that we save the value of sim (or we can also discard it on the + exit from xxx_probe() if we wish). + + That's all for a minimalistic initialization. To do things + right there is one more issue left. + + For a SIM driver there is one particularly interesting + event: when a target device is considered lost. In this case + resetting the SCSI negotiations with this device may be a good + idea. So we register a callback for this event with CAM. The + request is passed to CAM by requesting CAM action on a CAM + control block for this type of request: + + + struct ccb_setasync csa; + + xpt_setup_ccb(&csa.ccb_h, path, /*priority*/5); + csa.ccb_h.func_code = XPT_SASYNC_CB; + csa.event_enable = AC_LOST_DEVICE; + csa.callback = xxx_async; + csa.callback_arg = sim; + xpt_action((union ccb *)&csa); + + + Now we take a look at the xxx_action() + and xxx_poll() driver entry points. + + + + static void + xxx_action + + + struct cam_sim *sim, + union ccb *ccb + + + + + Do some action on request of the CAM subsystem. Sim + describes the SIM for the request, CCB is the request + itself. CCB stands for "CAM Control Block". It is a union of + many specific instances, each describing arguments for some type + of transactions. All of these instances share the CCB header + where the common part of arguments is stored. + + CAM supports the SCSI controllers working in both initiator + ("normal") mode and target (simulating a SCSI device) mode. Here + we only consider the part relevant to the initiator mode. + + There are a few function and macros (in other words, + methods) defined to access the public data in the struct sim: + + + cam_sim_path(sim) - the + path ID (see above) + + cam_sim_name(sim) - the + name of the sim + + cam_sim_softc(sim) - the + pointer to the softc (driver private data) + structure + + cam_sim_unit(sim) - the + unit number + + cam_sim_bus(sim) - the bus + ID + + + To identify the device, xxx_action() can + get the unit number and pointer to its structure softc using + these functions. + + The type of request is stored in + ccb->ccb_h.func_code. So generally + xxx_action() consists of a big + switch: + + + struct xxx_softc *softc = (struct xxx_softc *) cam_sim_softc(sim); + struct ccb_hdr *ccb_h = &ccb->ccb_h; + int unit = cam_sim_unit(sim); + int bus = cam_sim_bus(sim); + + switch(ccb_h->func_code) { + case ...: + ... + default: + ccb_h->status = CAM_REQ_INVALID; + xpt_done(ccb); + break; + } + + + As can be seen from the default case (if an unknown command + was received) the return code of the command is set into + ccb->ccb_h.status and the completed + CCB is returned back to CAM by calling + xpt_done(ccb). + + xpt_done() does not have to be called + from xxx_action(): For example an I/O + request may be enqueued inside the SIM driver and/or its SCSI + controller. Then when the device would post an interrupt + signaling that the processing of this request is complete + xpt_done() may be called from the interrupt + handling routine. + + Actually, the CCB status is not only assigned as a return + code but a CCB has some status all the time. Before CCB is + passed to the xxx_action() routine it gets + the status CCB_REQ_INPROG meaning that it's in progress. There + are a surprising number of status values defined in + /sys/cam/cam.h which should be able to + represent the status of a request in great detail. More + interesting yet, the status is in fact a "bitwise or" of an + enumerated status value (the lower 6 bits) and possible + additional flag-like bits (the upper bits). The enumerated + values will be discussed later in more detail. The summary of + them can be found in the Errors Summary section. The possible + status flags are: + + + + CAM_DEV_QFRZN - if the + SIM driver gets a serious error (for example, the device does + not respond to the selection or breaks the SCSI protocol) when + processing a CCB it should freeze the request queue by calling + xpt_freeze_simq(), return the other + enqueued but not processed yet CCBs for this device back to + the CAM queue, then set this flag for the troublesome CCB and + call xpt_done(). This flag causes the CAM + subsystem to unfreeze the queue after it handles the + error. + + CAM_AUTOSNS_VALID - if + the device returned an error condition and the flag + CAM_DIS_AUTOSENSE is not set in CCB the SIM driver must + execute the REQUEST SENSE command automatically to extract the + sense (extended error information) data from the device. If + this attempt was successful the sense data should be saved in + the CCB and this flag set. + + CAM_RELEASE_SIMQ - like + CAM_DEV_QFRZN but used in case there is some problem (or + resource shortage) with the SCSI controller itself. Then all + the future requests to the controller should be stopped by + xpt_freeze_simq(). The controller queue + will be restarted after the SIM driver overcomes the shortage + and informs CAM by returning some CCB with this flag + set. + + CAM_SIM_QUEUED - when SIM + puts a CCB into its request queue this flag should be set (and + removed when this CCB gets dequeued before being returned back + to CAM). This flag is not used anywhere in the CAM code now, + so its purpose is purely diagnostic. + + + + The function xxx_action() is not + allowed to sleep, so all the synchronization for resource access + must be done using SIM or device queue freezing. Besides the + aforementioned flags the CAM subsystem provides functions + xpt_selease_simq() and + xpt_release_devq() to unfreeze the queues + directly, without passing a CCB to CAM. + + The CCB header contains the following fields: + + + + path - path ID for the + request + + target_id - target device + ID for the request + + target_lun - LUN ID of + the target device + + timeout - timeout + interval for this command, in milliseconds + + timeout_ch - a + convenience place for the SIM driver to store the timeout handle + (the CAM subsystem itself does not make any assumptions about + it) + + flags - various bits of + information about the request spriv_ptr0, spriv_ptr1 - fields + reserved for private use by the SIM driver (such as linking to + the SIM queues or SIM private control blocks); actually, they + exist as unions: spriv_ptr0 and spriv_ptr1 have the type (void + *), spriv_field0 and spriv_field1 have the type unsigned long, + sim_priv.entries[0].bytes and sim_priv.entries[1].bytes are byte + arrays of the size consistent with the other incarnations of the + union and sim_priv.bytes is one array, twice + bigger. + + + + The recommended way of using the SIM private fields of CCB + is to define some meaningful names for them and use these + meaningful names in the driver, like: + + +#define ccb_some_meaningful_name sim_priv.entries[0].bytes +#define ccb_hcb spriv_ptr1 /* for hardware control block */ + + + The most common initiator mode requests are: + + XPT_SCSI_IO - execute an + I/O transaction + + The instance "struct ccb_scsiio csio" of the union ccb is + used to transfer the arguments. They are: + + + cdb_io - pointer to + the SCSI command buffer or the buffer + itself + + cdb_len - SCSI + command length + + data_ptr - pointer to + the data buffer (gets a bit complicated if scatter/gather is + used) + + dxfer_len - length of + the data to transfer + + sglist_cnt - counter + of the scatter/gather segments + + scsi_status - place + to return the SCSI status + + sense_data - buffer + for the SCSI sense information if the command returns an + error (the SIM driver is supposed to run the REQUEST SENSE + command automatically in this case if the CCB flag + CAM_DIS_AUTOSENSE is not set) + + sense_len - the + length of that buffer (if it happens to be higher than size + of sense_data the SIM driver must silently assume the + smaller value) resid, sense_resid - if the transfer of data + or SCSI sense returned an error these are the returned + counters of the residual (not transferred) data. They do not + seem to be especially meaningful, so in a case when they are + difficult to compute (say, counting bytes in the SCSI + controller's FIFO buffer) an approximate value will do as + well. For a successfully completed transfer they must be set + to zero. + + tag_action - the kind + of tag to use: + + + CAM_TAG_ACTION_NONE - don't use tags for this + transaction + MSG_SIMPLE_Q_TAG, MSG_HEAD_OF_Q_TAG, + MSG_ORDERED_Q_TAG - value equal to the appropriate tag + message (see /sys/cam/scsi/scsi_message.h); this gives only + the tag type, the SIM driver must assign the tag value + itself + + + + + + + The general logic of handling this request is the + following: + + The first thing to do is to check for possible races, to + make sure that the command did not get aborted when it was + sitting in the queue: + + + struct ccb_scsiio *csio = &ccb->csio; + + if ((ccb_h->status & CAM_STATUS_MASK) != CAM_REQ_INPROG) { + xpt_done(ccb); + return; + } + + + Also we check that the device is supported at all by our + controller: + + + if(ccb_h->target_id > OUR_MAX_SUPPORTED_TARGET_ID + || cch_h->target_id == OUR_SCSI_CONTROLLERS_OWN_ID) { + ccb_h->status = CAM_TID_INVALID; + xpt_done(ccb); + return; + } + if(ccb_h->target_lun > OUR_MAX_SUPPORTED_LUN) { + ccb_h->status = CAM_LUN_INVALID; + xpt_done(ccb); + return; + } + + + Then allocate whatever data structures (such as + card-dependent hardware control block) we need to process this + request. If we can't then freeze the SIM queue and remember + that we have a pending operation, return the CCB back and ask + CAM to re-queue it. Later when the resources become available + the SIM queue must be unfrozen by returning a ccb with the + CAM_SIMQ_RELEASE bit set in its status. Otherwise, if all went + well, link the CCB with the hardware control block (HCB) and + mark it as queued. + + + struct xxx_hcb *hcb = allocate_hcb(softc, unit, bus); + + if(hcb == NULL) { + softc->flags |= RESOURCE_SHORTAGE; + xpt_freeze_simq(sim, /*count*/1); + ccb_h->status = CAM_REQUEUE_REQ; + xpt_done(ccb); + return; + } + + hcb->ccb = ccb; ccb_h->ccb_hcb = (void *)hcb; + ccb_h->status |= CAM_SIM_QUEUED; + + + Extract the target data from CCB into the hardware control + block. Check if we are asked to assign a tag and if yes then + generate an unique tag and build the SCSI tag messages. The + SIM driver is also responsible for negotiations with the + devices to set the maximal mutually supported bus width, + synchronous rate and offset. + + + hcb->target = ccb_h->target_id; hcb->lun = ccb_h->target_lun; + generate_identify_message(hcb); + if( ccb_h->tag_action != CAM_TAG_ACTION_NONE ) + generate_unique_tag_message(hcb, ccb_h->tag_action); + if( !target_negotiated(hcb) ) + generate_negotiation_messages(hcb); + + + Then set up the SCSI command. The command storage may be + specified in the CCB in many interesting ways, specified by + the CCB flags. The command buffer can be contained in CCB or + pointed to, in the latter case the pointer may be physical or + virtual. Since the hardware commonly needs physical address we + always convert the address to the physical one. + + A NOT-QUITE RELATED NOTE: Normally this is done by a call + to vtophys(), but for the PCI device (which account for most + of the SCSI controllers now) drivers' portability to the Alpha + architecture the conversion must be done by vtobus() instead + due to special Alpha quirks. [IMHO it would be much better to + have two separate functions, vtop() and ptobus() then vtobus() + would be a simple superposition of them.] In case if a + physical address is requested it's OK to return the CCB with + the status CAM_REQ_INVALID, the current drivers do that. But + it's also possible to compile the Alpha-specific piece of + code, as in this example (there should be a more direct way to + do that, without conditional compilation in the drivers). If + necessary a physical address can be also converted or mapped + back to a virtual address but with big pain, so we don't do + that. + + + if(ccb_h->flags & CAM_CDB_POINTER) { + /* CDB is a pointer */ + if(!(ccb_h->flags & CAM_CDB_PHYS)) { + /* CDB pointer is virtual */ + hcb->cmd = vtobus(csio->cdb_io.cdb_ptr); + } else { + /* CDB pointer is physical */ +#if defined(__alpha__) + hcb->cmd = csio->cdb_io.cdb_ptr | alpha_XXX_dmamap_or ; +#else + hcb->cmd = csio->cdb_io.cdb_ptr ; +#endif + } + } else { + /* CDB is in the ccb (buffer) */ + hcb->cmd = vtobus(csio->cdb_io.cdb_bytes); + } + hcb->cmdlen = csio->cdb_len; + + + Now it's time to set up the data. Again, the data storage + may be specified in the CCB in many interesting ways, + specified by the CCB flags. First we get the direction of the + data transfer. The simplest case is if there is no data to + transfer: + + + int dir = (ccb_h->flags & CAM_DIR_MASK); + + if (dir == CAM_DIR_NONE) + goto end_data; + + + Then we check if the data is in one chunk or in a + scatter-gather list, and the addresses are physical or + virtual. The SCSI controller may be able to handle only a + limited number of chunks of limited length. If the request + hits this limitation we return an error. We use a special + function to return the CCB to handle in one place the HCB + resource shortages. The functions to add chunks are + driver-dependent, and here we leave them without detailed + implementation. See description of the SCSI command (CDB) + handling for the details on the address-translation issues. + If some variation is too difficult or impossible to implement + with a particular card it's OK to return the status + CAM_REQ_INVALID. Actually, it seems like the scatter-gather + ability is not used anywhere in the CAM code now. But at least + the case for a single non-scattered virtual buffer must be + implemented, it's actively used by CAM. + + + int rv; + + initialize_hcb_for_data(hcb); + + if((!(ccb_h->flags & CAM_SCATTER_VALID)) { + /* single buffer */ + if(!(ccb_h->flags & CAM_DATA_PHYS)) { + rv = add_virtual_chunk(hcb, csio->data_ptr, csio->dxfer_len, dir); + } + } else { + rv = add_physical_chunk(hcb, csio->data_ptr, csio->dxfer_len, dir); + } + } else { + int i; + struct bus_dma_segment *segs; + segs = (struct bus_dma_segment *)csio->data_ptr; + + if ((ccb_h->flags & CAM_SG_LIST_PHYS) != 0) { + /* The SG list pointer is physical */ + rv = setup_hcb_for_physical_sg_list(hcb, segs, csio->sglist_cnt); + } else if (!(ccb_h->flags & CAM_DATA_PHYS)) { + /* SG buffer pointers are virtual */ + for (i = 0; i < csio->sglist_cnt; i++) { + rv = add_virtual_chunk(hcb, segs[i].ds_addr, + segs[i].ds_len, dir); + if (rv != CAM_REQ_CMP) + break; + } + } else { + /* SG buffer pointers are physical */ + for (i = 0; i < csio->sglist_cnt; i++) { + rv = add_physical_chunk(hcb, segs[i].ds_addr, + segs[i].ds_len, dir); + if (rv != CAM_REQ_CMP) + break; + } + } + } + if(rv != CAM_REQ_CMP) { + /* we expect that add_*_chunk() functions return CAM_REQ_CMP + * if they added a chunk successfully, CAM_REQ_TOO_BIG if + * the request is too big (too many bytes or too many chunks), + * CAM_REQ_INVALID in case of other troubles + */ + free_hcb_and_ccb_done(hcb, ccb, rv); + return; + } + end_data: + + + If disconnection is disabled for this CCB we pass this + information to the hcb: + + + if(ccb_h->flags & CAM_DIS_DISCONNECT) + hcb_disable_disconnect(hcb); + + + If the controller is able to run REQUEST SENSE command all + by itself then the value of the flag CAM_DIS_AUTOSENSE should + also be passed to it, to prevent automatic REQUEST SENSE if the + CAM subsystem does not want it. + + The only thing left is to set up the timeout, pass our hcb + to the hardware and return, the rest will be done by the + interrupt handler (or timeout handler). + + + ccb_h->timeout_ch = timeout(xxx_timeout, (caddr_t) hcb, + (ccb_h->timeout * hz) / 1000); /* convert milliseconds to ticks */ + put_hcb_into_hardware_queue(hcb); + return; + + + And here is a possible implementation of the function + returning CCB: + + + static void + free_hcb_and_ccb_done(struct xxx_hcb *hcb, union ccb *ccb, u_int32_t status) + { + struct xxx_softc *softc = hcb->softc; + + ccb->ccb_h.ccb_hcb = 0; + if(hcb != NULL) { + untimeout(xxx_timeout, (caddr_t) hcb, ccb->ccb_h.timeout_ch); + /* we're about to free a hcb, so the shortage has ended */ + if(softc->flags & RESOURCE_SHORTAGE) { + softc->flags &= ~RESOURCE_SHORTAGE; + status |= CAM_RELEASE_SIMQ; + } + free_hcb(hcb); /* also removes hcb from any internal lists */ + } + ccb->ccb_h.status = status | + (ccb->ccb_h.status & ~(CAM_STATUS_MASK|CAM_SIM_QUEUED)); + xpt_done(ccb); + } + + + + XPT_RESET_DEV - send the SCSI "BUS + DEVICE RESET" message to a device + + There is no data transferred in CCB except the header and + the most interesting argument of it is target_id. Depending on + the controller hardware a hardware control block just like for + the XPT_SCSI_IO request may be constructed (see XPT_SCSI_IO + request description) and sent to the controller or the SCSI + controller may be immediately programmed to send this RESET + message to the device or this request may be just not supported + (and return the status CAM_REQ_INVALID). Also on completion of + the request all the disconnected transactions for this target + must be aborted (probably in the interrupt routine). + + Also all the current negotiations for the target are lost on + reset, so they might be cleaned too. Or they clearing may be + deferred, because anyway the target would request re-negotiation + on the next transaction. + + XPT_RESET_BUS - send the RESET signal + to the SCSI bus + + No arguments are passed in the CCB, the only interesting + argument is the SCSI bus indicated by the struct sim + pointer. + + A minimalistic implementation would forget the SCSI + negotiations for all the devices on the bus and return the + status CAM_REQ_CMP. + + The proper implementation would in addition actually reset + the SCSI bus (possible also reset the SCSI controller) and mark + all the CCBs being processed, both those in the hardware queue + and those being disconnected, as done with the status + CAM_SCSI_BUS_RESET. Like: + + + int targ, lun; + struct xxx_hcb *h, *hh; + struct ccb_trans_settings neg; + struct cam_path *path; + + /* The SCSI bus reset may take a long time, in this case its completion + * should be checked by interrupt or timeout. But for simplicity + * we assume here that it's really fast. + */ + reset_scsi_bus(softc); + + /* drop all enqueued CCBs */ + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* the clean values of negotiations to report */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + + /* drop all disconnected CCBs and clean negotiations */ + for(targ=0; targ <= OUR_MAX_SUPPORTED_TARGET; targ++) { + clean_negotiations(softc, targ); + + /* report the event if possible */ + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), targ, + CAM_LUN_WILDCARD) == CAM_REQ_CMP) { + xpt_async(AC_TRANSFER_NEG, path, &neg); + xpt_free_path(path); + } + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[targ][lun]; h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + } + + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + + /* report the event */ + xpt_async(AC_BUS_RESET, softc->wpath, NULL); + return; + + + Implementing the SCSI bus reset as a function may be a good + idea because it would be re-used by the timeout function as a + last resort if the things go wrong. + + XPT_ABORT - abort the specified + CCB + + The arguments are transferred in the instance "struct + ccb_abort cab" of the union ccb. The only argument field in it + is: + + abort_ccb - pointer to the CCB to be + aborted + + If the abort is not supported just return the status + CAM_UA_ABORT. This is also the easy way to minimally implement + this call, return CAM_UA_ABORT in any case. + + The hard way is to implement this request honestly. First + check that abort applies to a SCSI transaction: + + + struct ccb *abort_ccb; + abort_ccb = ccb->cab.abort_ccb; + + if(abort_ccb->ccb_h.func_code != XPT_SCSI_IO) { + ccb->ccb_h.status = CAM_UA_ABORT; + xpt_done(ccb); + return; + } + + + Then it's necessary to find this CCB in our queue. This can + be done by walking the list of all our hardware control blocks + in search for one associated with this CCB: + + + struct xxx_hcb *hcb, *h; + + hcb = NULL; + + /* We assume that softc->first_hcb is the head of the list of all + * HCBs associated with this bus, including those enqueued for + * processing, being processed by hardware and disconnected ones. + */ + for(h = softc->first_hcb; h != NULL; h = h->next) { + if(h->ccb == abort_ccb) { + hcb = h; + break; + } + } + + if(hcb == NULL) { + /* no such CCB in our queue */ + ccb->ccb_h.status = CAM_PATH_INVALID; + xpt_done(ccb); + return; + } + + hcb=found_hcb; + + + Now we look at the current processing status of the HCB. It + may be either sitting in the queue waiting to be sent to the + SCSI bus, being transferred right now, or disconnected and + waiting for the result of the command, or actually completed by + hardware but not yet marked as done by software. To make sure + that we don't get in any races with hardware we mark the HCB as + being aborted, so that if this HCB is about to be sent to the + SCSI bus the SCSI controller will see this flag and skip + it. + + + int hstatus; + + /* shown as a function, in case special action is needed to make + * this flag visible to hardware + */ + set_hcb_flags(hcb, HCB_BEING_ABORTED); + + abort_again: + + hstatus = get_hcb_status(hcb); + switch(hstatus) { + case HCB_SITTING_IN_QUEUE: + remove_hcb_from_hardware_queue(hcb); + /* FALLTHROUGH */ + case HCB_COMPLETED: + /* this is an easy case */ + free_hcb_and_ccb_done(hcb, abort_ccb, CAM_REQ_ABORTED); + break; + + + If the CCB is being transferred right now we would like to + signal to the SCSI controller in some hardware-dependent way + that we want to abort the current transfer. The SCSI controller + would set the SCSI ATTENTION signal and when the target responds + to it send an ABORT message. We also reset the timeout to make + sure that the target is not sleeping forever. If the command + would not get aborted in some reasonable time like 10 seconds + the timeout routine would go ahead and reset the whole SCSI bus. + Because the command will be aborted in some reasonable time we + can just return the abort request now as successfully completed, + and mark the aborted CCB as aborted (but not mark it as done + yet). + + + case HCB_BEING_TRANSFERRED: + untimeout(xxx_timeout, (caddr_t) hcb, abort_ccb->ccb_h.timeout_ch); + abort_ccb->ccb_h.timeout_ch = + timeout(xxx_timeout, (caddr_t) hcb, 10 * hz); + abort_ccb->ccb_h.status = CAM_REQ_ABORTED; + /* ask the controller to abort that HCB, then generate + * an interrupt and stop + */ + if(signal_hardware_to_abort_hcb_and_stop(hcb) < 0) { + /* oops, we missed the race with hardware, this transaction + * got off the bus before we aborted it, try again */ + goto abort_again; + } + + break; + + + If the CCB is in the list of disconnected then set it up as + an abort request and re-queue it at the front of hardware + queue. Reset the timeout and report the abort request to be + completed. + + + case HCB_DISCONNECTED: + untimeout(xxx_timeout, (caddr_t) hcb, abort_ccb->ccb_h.timeout_ch); + abort_ccb->ccb_h.timeout_ch = + timeout(xxx_timeout, (caddr_t) hcb, 10 * hz); + put_abort_message_into_hcb(hcb); + put_hcb_at_the_front_of_hardware_queue(hcb); + break; + } + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + That's all for the ABORT request, although there is one more + issue. Because the ABORT message cleans all the ongoing + transactions on a LUN we have to mark all the other active + transactions on this LUN as aborted. That should be done in the + interrupt routine, after the transaction gets aborted. + + Implementing the CCB abort as a function may be quite a good + idea, this function can be re-used if an I/O transaction times + out. The only difference would be that the timed out transaction + would return the status CAM_CMD_TIMEOUT for the timed out + request. Then the case XPT_ABORT would be small, like + that: + + + case XPT_ABORT: + struct ccb *abort_ccb; + abort_ccb = ccb->cab.abort_ccb; + + if(abort_ccb->ccb_h.func_code != XPT_SCSI_IO) { + ccb->ccb_h.status = CAM_UA_ABORT; + xpt_done(ccb); + return; + } + if(xxx_abort_ccb(abort_ccb, CAM_REQ_ABORTED) < 0) + /* no such CCB in our queue */ + ccb->ccb_h.status = CAM_PATH_INVALID; + else + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + + XPT_SET_TRAN_SETTINGS - explicitly + set values of SCSI transfer settings + + The arguments are transferred in the instance "struct ccb_trans_setting cts" +of the union ccb: + + + valid - a bitmask showing + which settings should be updated: + + CCB_TRANS_SYNC_RATE_VALID + - synchronous transfer rate + + CCB_TRANS_SYNC_OFFSET_VALID + - synchronous offset + + CCB_TRANS_BUS_WIDTH_VALID + - bus width + + CCB_TRANS_DISC_VALID - + set enable/disable disconnection + + CCB_TRANS_TQ_VALID - set + enable/disable tagged queuing + + flags - consists of two + parts, binary arguments and identification of + sub-operations. The binary arguments are : + + CCB_TRANS_DISC_ENB - enable disconnection + CCB_TRANS_TAG_ENB - + enable tagged queuing + + + + the sub-operations are: + + CCB_TRANS_CURRENT_SETTINGS + - change the current negotiations + + CCB_TRANS_USER_SETTINGS + - remember the desired user values sync_period, sync_offset - + self-explanatory, if sync_offset==0 then the asynchronous mode + is requested bus_width - bus width, in bits (not + bytes) + + + + + + Two sets of negotiated parameters are supported, the user + settings and the current settings. The user settings are not + really used much in the SIM drivers, this is mostly just a piece + of memory where the upper levels can store (and later recall) + its ideas about the parameters. Setting the user parameters + does not cause re-negotiation of the transfer rates. But when + the SCSI controller does a negotiation it must never set the + values higher than the user parameters, so it's essentially the + top boundary. + + The current settings are, as the name says, + current. Changing them means that the parameters must be + re-negotiated on the next transfer. Again, these "new current + settings" are not supposed to be forced on the device, just they + are used as the initial step of negotiations. Also they must be + limited by actual capabilities of the SCSI controller: for + example, if the SCSI controller has 8-bit bus and the request + asks to set 16-bit wide transfers this parameter must be + silently truncated to 8-bit transfers before sending it to the + device. + + One caveat is that the bus width and synchronous parameters + are per target while the disconnection and tag enabling + parameters are per lun. + + The recommended implementation is to keep 3 sets of + negotiated (bus width and synchronous transfer) + parameters: + + + user - the user set, as + above + + current - those actually + in effect + + goal - those requested by + setting of the "current" parameters + + + The code looks like: + + + struct ccb_trans_settings *cts; + int targ, lun; + int flags; + + cts = &ccb->cts; + targ = ccb_h->target_id; + lun = ccb_h->target_lun; + flags = cts->flags; + if(flags & CCB_TRANS_USER_SETTINGS) { + if(flags & CCB_TRANS_SYNC_RATE_VALID) + softc->user_sync_period[targ] = cts->sync_period; + if(flags & CCB_TRANS_SYNC_OFFSET_VALID) + softc->user_sync_offset[targ] = cts->sync_offset; + if(flags & CCB_TRANS_BUS_WIDTH_VALID) + softc->user_bus_width[targ] = cts->bus_width; + + if(flags & CCB_TRANS_DISC_VALID) { + softc->user_tflags[targ][lun] &= ~CCB_TRANS_DISC_ENB; + softc->user_tflags[targ][lun] |= flags & CCB_TRANS_DISC_ENB; + } + if(flags & CCB_TRANS_TQ_VALID) { + softc->user_tflags[targ][lun] &= ~CCB_TRANS_TQ_ENB; + softc->user_tflags[targ][lun] |= flags & CCB_TRANS_TQ_ENB; + } + } + if(flags & CCB_TRANS_CURRENT_SETTINGS) { + if(flags & CCB_TRANS_SYNC_RATE_VALID) + softc->goal_sync_period[targ] = + max(cts->sync_period, OUR_MIN_SUPPORTED_PERIOD); + if(flags & CCB_TRANS_SYNC_OFFSET_VALID) + softc->goal_sync_offset[targ] = + min(cts->sync_offset, OUR_MAX_SUPPORTED_OFFSET); + if(flags & CCB_TRANS_BUS_WIDTH_VALID) + softc->goal_bus_width[targ] = min(cts->bus_width, OUR_BUS_WIDTH); + + if(flags & CCB_TRANS_DISC_VALID) { + softc->current_tflags[targ][lun] &= ~CCB_TRANS_DISC_ENB; + softc->current_tflags[targ][lun] |= flags & CCB_TRANS_DISC_ENB; + } + if(flags & CCB_TRANS_TQ_VALID) { + softc->current_tflags[targ][lun] &= ~CCB_TRANS_TQ_ENB; + softc->current_tflags[targ][lun] |= flags & CCB_TRANS_TQ_ENB; + } + } + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + Then when the next I/O request will be processed it will + check if it has to re-negotiate, for example by calling the + function target_negotiated(hcb). It can be implemented like + this: + + + int + target_negotiated(struct xxx_hcb *hcb) + { + struct softc *softc = hcb->softc; + int targ = hcb->targ; + + if( softc->current_sync_period[targ] != softc->goal_sync_period[targ] + || softc->current_sync_offset[targ] != softc->goal_sync_offset[targ] + || softc->current_bus_width[targ] != softc->goal_bus_width[targ] ) + return 0; /* FALSE */ + else + return 1; /* TRUE */ + } + + + After the values are re-negotiated the resulting values must + be assigned to both current and goal parameters, so for future + I/O transactions the current and goal parameters would be the + same and target_negotiated() would return + TRUE. When the card is initialized (in + xxx_attach()) the current negotiation + values must be initialized to narrow asynchronous mode, the goal + and current values must be initialized to the maximal values + supported by controller. + + XPT_GET_TRAN_SETTINGS - get values of + SCSI transfer settings + + This operations is the reverse of + XPT_SET_TRAN_SETTINGS. Fill up the CCB instance "struct + ccb_trans_setting cts" with data as requested by the flags + CCB_TRANS_CURRENT_SETTINGS or CCB_TRANS_USER_SETTINGS (if both + are set then the existing drivers return the current + settings). Set all the bits in the valid field. + + XPT_CALC_GEOMETRY - calculate logical + (BIOS) geometry of the disk + + The arguments are transferred in the instance "struct + ccb_calc_geometry ccg" of the union ccb: + + + + block_size - input, block + (A.K.A sector) size in bytes + + volume_size - input, + volume size in bytes + + cylinders - output, + logical cylinders + + heads - output, logical + heads + + secs_per_track - output, + logical sectors per track + + + + If the returned geometry differs much enough from what the + SCSI controller BIOS thinks and a disk on this SCSI controller + is used as bootable the system may not be able to boot. The + typical calculation example taken from the aic7xxx driver + is: + + + struct ccb_calc_geometry *ccg; + u_int32_t size_mb; + u_int32_t secs_per_cylinder; + int extended; + + ccg = &ccb->ccg; + size_mb = ccg->volume_size + / ((1024L * 1024L) / ccg->block_size); + extended = check_cards_EEPROM_for_extended_geometry(softc); + + if (size_mb > 1024 && extended) { + ccg->heads = 255; + ccg->secs_per_track = 63; + } else { + ccg->heads = 64; + ccg->secs_per_track = 32; + } + secs_per_cylinder = ccg->heads * ccg->secs_per_track; + ccg->cylinders = ccg->volume_size / secs_per_cylinder; + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + This gives the general idea, the exact calculation depends + on the quirks of the particular BIOS. If BIOS provides no way + set the "extended translation" flag in EEPROM this flag should + normally be assumed equal to 1. Other popular geometries + are: + + + 128 heads, 63 sectors - Symbios controllers + 16 heads, 63 sectors - old controllers + + + Some system BIOSes and SCSI BIOSes fight with each other + with variable success, for example a combination of Symbios + 875/895 SCSI and Phoenix BIOS can give geometry 128/63 after + power up and 255/63 after a hard reset or soft reboot. + + + XPT_PATH_INQ - path inquiry, in other + words get the SIM driver and SCSI controller (also known as HBA + - Host Bus Adapter) properties + + The properties are returned in the instance "struct +ccb_pathinq cpi" of the union ccb: + + + + version_num - the SIM driver version number, now + all drivers use 1 + + hba_inquiry - bitmask of features supported by + the controller: + + PI_MDP_ABLE - supports MDP message (something + from SCSI3?) + + PI_WIDE_32 - supports 32 bit wide + SCSI + + PI_WIDE_16 - supports 16 bit wide + SCSI + + PI_SDTR_ABLE - can negotiate synchronous + transfer rate + + PI_LINKED_CDB - supports linked + commands + + PI_TAG_ABLE - supports tagged + commands + + PI_SOFT_RST - supports soft reset alternative + (hard reset and soft reset are mutually exclusive within a + SCSI bus) + + target_sprt - flags for target mode support, 0 + if unsupported + + hba_misc - miscellaneous controller + features: + + PIM_SCANHILO - bus scans from high ID to low + ID + + PIM_NOREMOVE - removable devices not included in + scan + + PIM_NOINITIATOR - initiator role not + supported + + PIM_NOBUSRESET - user has disabled initial BUS + RESET + + hba_eng_cnt - mysterious HBA engine count, + something related to compression, now is always set to + 0 + + vuhba_flags - vendor-unique flags, unused + now + + max_target - maximal supported target ID (7 for + 8-bit bus, 15 for 16-bit bus, 127 for Fibre + Channel) + + max_lun - maximal supported LUN ID (7 for older + SCSI controllers, 63 for newer ones) + + async_flags - bitmask of installed Async + handler, unused now + + hpath_id - highest Path ID in the subsystem, + unused now + + unit_number - the controller unit number, + cam_sim_unit(sim) + + bus_id - the bus number, + cam_sim_bus(sim) + + initiator_id - the SCSI ID of the controller + itself + + base_transfer_speed - nominal transfer speed in + KB/s for asynchronous narrow transfers, equals to 3300 for + SCSI + + sim_vid - SIM driver's vendor id, a + zero-terminated string of maximal length SIM_IDLEN including + the terminating zero + + hba_vid - SCSI controller's vendor id, a + zero-terminated string of maximal length HBA_IDLEN including + the terminating zero + + dev_name - device driver name, a zero-terminated + string of maximal length DEV_IDLEN including the terminating + zero, equal to cam_sim_name(sim) + + + + The recommended way of setting the string fields is using + strncpy, like: + + + strncpy(cpi->dev_name, cam_sim_name(sim), DEV_IDLEN); + + + After setting the values set the status to CAM_REQ_CMP and mark the +CCB as done. + + + + + + + Polling + + + static void + xxx_poll + + + struct cam_sim *sim + + + + The poll function is used to simulate the interrupts when + the interrupt subsystem is not functioning (for example, when + the system has crashed and is creating the system dump). The CAM + subsystem sets the proper interrupt level before calling the + poll routine. So all it needs to do is to call the interrupt + routine (or the other way around, the poll routine may be doing + the real action and the interrupt routine would just call the + poll routine). Why bother about a separate function then ? + Because of different calling conventions. The + xxx_poll routine gets the struct cam_sim + pointer as its argument when the PCI interrupt routine by common + convention gets pointer to the struct + xxx_softc and the ISA interrupt routine + gets just the the device unit number. So the poll routine would + normally look as: + + +static void +xxx_poll(struct cam_sim *sim) +{ + xxx_intr((struct xxx_softc *)cam_sim_softc(sim)); /* for PCI device */ +} + + + or + + +static void +xxx_poll(struct cam_sim *sim) +{ + xxx_intr(cam_sim_unit(sim)); /* for ISA device */ +} + + + + + + Asynchronous Events + + If an asynchronous event callback has been set up then the + callback function should be defined. + + +static void +ahc_async(void *callback_arg, u_int32_t code, struct cam_path *path, void *arg) + + + + callback_arg - the value supplied when registering the + callback + + code - identifies the type of event + + path - identifies the devices to which the event + applies + + arg - event-specific argument + + + Implementation for a single type of event, AC_LOST_DEVICE, + looks like: + + + struct xxx_softc *softc; + struct cam_sim *sim; + int targ; + struct ccb_trans_settings neg; + + sim = (struct cam_sim *)callback_arg; + softc = (struct xxx_softc *)cam_sim_softc(sim); + switch (code) { + case AC_LOST_DEVICE: + targ = xpt_path_target_id(path); + if(targ <= OUR_MAX_SUPPORTED_TARGET) { + clean_negotiations(softc, targ); + /* send indication to CAM */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + xpt_async(AC_TRANSFER_NEG, path, &neg); + } + break; + default: + break; + } + + + + + + Interrupts + + The exact type of the interrupt routine depends on the type + of the peripheral bus (PCI, ISA and so on) to which the SCSI + controller is connected. + + The interrupt routines of the SIM drivers run at the + interrupt level splcam. So splcam() should + be used in the driver to synchronize activity between the + interrupt routine and the rest of the driver (for a + multiprocessor-aware driver things get yet more interesting but + we ignore this case here). The pseudo-code in this document + happily ignores the problems of synchronization. The real code + must not ignore them. A simple-minded approach is to set + splcam() on the entry to the other routines + and reset it on return thus protecting them by one big critical + section. To make sure that the interrupt level will be always + restored a wrapper function can be defined, like: + + + static void + xxx_action(struct cam_sim *sim, union ccb *ccb) + { + int s; + s = splcam(); + xxx_action1(sim, ccb); + splx(s); + } + + static void + xxx_action1(struct cam_sim *sim, union ccb *ccb) + { + ... process the request ... + } + + + This approach is simple and robust but the problem with it + is that interrupts may get blocked for a relatively long time + and this would negatively affect the system's performance. On + the other hand the functions of the spl() + family have rather high overhead, so vast amount of tiny + critical sections may not be good either. + + The conditions handled by the interrupt routine and the + details depend very much on the hardware. We consider the set of + "typical" conditions. + + First, we check if a SCSI reset was encountered on the bus + (probably caused by another SCSI controller on the same SCSI + bus). If so we drop all the enqueued and disconnected requests, + report the events and re-initialize our SCSI controller. It is + important that during this initialization the controller won't + issue another reset or else two controllers on the same SCSI bus + could ping-pong resets forever. The case of fatal controller + error/hang could be handled in the same place, but it will + probably need also sending RESET signal to the SCSI bus to reset + the status of the connections with the SCSI devices. + + + int fatal=0; + struct ccb_trans_settings neg; + struct cam_path *path; + + if( detected_scsi_reset(softc) + || (fatal = detected_fatal_controller_error(softc)) ) { + int targ, lun; + struct xxx_hcb *h, *hh; + + /* drop all enqueued CCBs */ + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* the clean values of negotiations to report */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + + /* drop all disconnected CCBs and clean negotiations */ + for(targ=0; targ <= OUR_MAX_SUPPORTED_TARGET; targ++) { + clean_negotiations(softc, targ); + + /* report the event if possible */ + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), targ, + CAM_LUN_WILDCARD) == CAM_REQ_CMP) { + xpt_async(AC_TRANSFER_NEG, path, &neg); + xpt_free_path(path); + } + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[targ][lun]; h != NULL; h = hh) { + hh=h->next; + if(fatal) + free_hcb_and_ccb_done(h, h->ccb, CAM_UNREC_HBA_ERROR); + else + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + } + + /* report the event */ + xpt_async(AC_BUS_RESET, softc->wpath, NULL); + + /* re-initialization may take a lot of time, in such case + * its completion should be signaled by another interrupt or + * checked on timeout - but for simplicity we assume here that + * it's really fast + */ + if(!fatal) { + reinitialize_controller_without_scsi_reset(softc); + } else { + reinitialize_controller_with_scsi_reset(softc); + } + schedule_next_hcb(softc); + return; + } + + + If interrupt is not caused by a controller-wide condition + then probably something has happened to the current hardware + control block. Depending on the hardware there may be other + non-HCB-related events, we just do not consider them here. Then + we analyze what happened to this HCB: + + + struct xxx_hcb *hcb, *h, *hh; + int hcb_status, scsi_status; + int ccb_status; + int targ; + int lun_to_freeze; + + hcb = get_current_hcb(softc); + if(hcb == NULL) { + /* either stray interrupt or something went very wrong + * or this is something hardware-dependent + */ + handle as necessary; + return; + } + + targ = hcb->target; + hcb_status = get_status_of_current_hcb(softc); + + + First we check if the HCB has completed and if so we check + the returned SCSI status. + + + if(hcb_status == COMPLETED) { + scsi_status = get_completion_status(hcb); + + + Then look if this status is related to the REQUEST SENSE + command and if so handle it in a simple way. + + + if(hcb->flags & DOING_AUTOSENSE) { + if(scsi_status == GOOD) { /* autosense was successful */ + hcb->ccb->ccb_h.status |= CAM_AUTOSNS_VALID; + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_SCSI_STATUS_ERROR); + } else { + autosense_failed: + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_AUTOSENSE_FAIL); + } + schedule_next_hcb(softc); + return; + } + + + Else the command itself has completed, pay more attention to + details. If auto-sense is not disabled for this CCB and the + command has failed with sense data then run REQUEST SENSE + command to receive that data. + + + hcb->ccb->csio.scsi_status = scsi_status; + calculate_residue(hcb); + + if( (hcb->ccb->ccb_h.flags & CAM_DIS_AUTOSENSE)==0 + && ( scsi_status == CHECK_CONDITION + || scsi_status == COMMAND_TERMINATED) ) { + /* start auto-SENSE */ + hcb->flags |= DOING_AUTOSENSE; + setup_autosense_command_in_hcb(hcb); + restart_current_hcb(softc); + return; + } + if(scsi_status == GOOD) + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_REQ_CMP); + else + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_SCSI_STATUS_ERROR); + schedule_next_hcb(softc); + return; + } + + + One typical thing would be negotiation events: negotiation + messages received from a SCSI target (in answer to our + negotiation attempt or by target's initiative) or the target is + unable to negotiate (rejects our negotiation messages or does + not answer them). + + + switch(hcb_status) { + case TARGET_REJECTED_WIDE_NEG: + /* revert to 8-bit bus */ + softc->current_bus_width[targ] = softc->goal_bus_width[targ] = 8; + /* report the event */ + neg.bus_width = 8; + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + continue_current_hcb(softc); + return; + case TARGET_ANSWERED_WIDE_NEG: + { + int wd; + + wd = get_target_bus_width_request(softc); + if(wd <= softc->goal_bus_width[targ]) { + /* answer is acceptable */ + softc->current_bus_width[targ] = + softc->goal_bus_width[targ] = neg.bus_width = wd; + + /* report the event */ + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + } else { + prepare_reject_message(hcb); + } + } + continue_current_hcb(softc); + return; + case TARGET_REQUESTED_WIDE_NEG: + { + int wd; + + wd = get_target_bus_width_request(softc); + wd = min (wd, OUR_BUS_WIDTH); + wd = min (wd, softc->user_bus_width[targ]); + + if(wd != softc->current_bus_width[targ]) { + /* the bus width has changed */ + softc->current_bus_width[targ] = + softc->goal_bus_width[targ] = neg.bus_width = wd; + + /* report the event */ + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + } + prepare_width_nego_rsponse(hcb, wd); + } + continue_current_hcb(softc); + return; + } + + + Then we handle any errors that could have happened during + auto-sense in the same simple-minded way as before. Otherwise we + look closer at the details again. + + + if(hcb->flags & DOING_AUTOSENSE) + goto autosense_failed; + + switch(hcb_status) { + + + The next event we consider is unexpected disconnect. Which + is considered normal after an ABORT or BUS DEVICE RESET message + and abnormal in other cases. + + + case UNEXPECTED_DISCONNECT: + if(requested_abort(hcb)) { + /* abort affects all commands on that target+LUN, so + * mark all disconnected HCBs on that target+LUN as aborted too + */ + for(h = softc->first_discon_hcb[hcb->target][hcb->lun]; + h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_REQ_ABORTED); + } + ccb_status = CAM_REQ_ABORTED; + } else if(requested_bus_device_reset(hcb)) { + int lun; + + /* reset affects all commands on that target, so + * mark all disconnected HCBs on that target+LUN as reset + */ + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[hcb->target][lun]; + h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* send event */ + xpt_async(AC_SENT_BDR, hcb->ccb->ccb_h.path_id, NULL); + + /* this was the CAM_RESET_DEV request itself, it's completed */ + ccb_status = CAM_REQ_CMP; + } else { + calculate_residue(hcb); + ccb_status = CAM_UNEXP_BUSFREE; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = hcb->lun; + } + break; + + + If the target refuses to accept tags we notify CAM about + that and return back all commands for this LUN: + + + case TAGS_REJECTED: + /* report the event */ + neg.flags = 0 & ~CCB_TRANS_TAG_ENB; + neg.valid = CCB_TRANS_TQ_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + + ccb_status = CAM_MSG_REJECT_REC; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = hcb->lun; + break; + + + Then we check a number of other conditions, with processing + basically limited to setting the CCB status: + + + case SELECTION_TIMEOUT: + ccb_status = CAM_SEL_TIMEOUT; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = CAM_LUN_WILDCARD; + break; + case PARITY_ERROR: + ccb_status = CAM_UNCOR_PARITY; + break; + case DATA_OVERRUN: + case ODD_WIDE_TRANSFER: + ccb_status = CAM_DATA_RUN_ERR; + break; + default: + /* all other errors are handled in a generic way */ + ccb_status = CAM_REQ_CMP_ERR; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = CAM_LUN_WILDCARD; + break; + } + + + Then we check if the error was serious enough to freeze the + input queue until it gets proceeded and do so if it is: + + + if(hcb->ccb->ccb_h.status & CAM_DEV_QFRZN) { + /* freeze the queue */ + xpt_freeze_devq(ccb->ccb_h.path, /*count*/1); + + /* re-queue all commands for this target/LUN back to CAM */ + + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + + if(targ == h->targ + && (lun_to_freeze == CAM_LUN_WILDCARD || lun_to_freeze == h->lun) ) + free_hcb_and_ccb_done(h, h->ccb, CAM_REQUEUE_REQ); + } + } + free_hcb_and_ccb_done(hcb, hcb->ccb, ccb_status); + schedule_next_hcb(softc); + return; + + + This concludes the generic interrupt handling although + specific controllers may require some additions. + + + + + Errors Summary + + When executing an I/O request many things may go wrong. The + reason of error can be reported in the CCB status with great + detail. Examples of use are spread throughout this document. For + completeness here is the summary of recommended responses for + the typical error conditions: + + + + CAM_RESRC_UNAVAIL - some + resource is temporarily unavailable and the SIM driver can not + generate an event when it will become available. An example of + this resource would be some intra-controller hardware resource + for which the controller does not generate an interrupt when + it becomes available. + + CAM_UNCOR_PARITY - + unrecovered parity error occurred + + CAM_DATA_RUN_ERR - data + overrun or unexpected data phase (going in other direction + than specified in CAM_DIR_MASK) or odd transfer length for + wide transfer + + CAM_SEL_TIMEOUT - selection + timeout occurred (target does not respond) + + CAM_CMD_TIMEOUT - command + timeout occurred (the timeout function ran) + + CAM_SCSI_STATUS_ERROR - the + device returned error + + CAM_AUTOSENSE_FAIL - the + device returned error and the REQUEST SENSE COMMAND + failed + + CAM_MSG_REJECT_REC - MESSAGE + REJECT message was received + + CAM_SCSI_BUS_RESET - received + SCSI bus reset + + CAM_REQ_CMP_ERR - + "impossible" SCSI phase occurred or something else as weird or + just a generic error if further detail is not + available + + CAM_UNEXP_BUSFREE - + unexpected disconnect occurred + + CAM_BDR_SENT - BUS DEVICE + RESET message was sent to the target + + CAM_UNREC_HBA_ERROR - + unrecoverable Host Bus Adapter Error + + CAM_REQ_TOO_BIG - the request + was too large for this controller + + CAM_REQUEUE_REQ - this + request should be re-queued to preserve transaction ordering. + This typically occurs when the SIM recognizes an error that + should freeze the queue and must place other queued requests + for the target at the sim level back into the XPT + queue. Typical cases of such errors are selection timeouts, + command timeouts and other like conditions. In such cases the + troublesome command returns the status indicating the error, + the and the other commands which have not be sent to the bus + yet get re-queued. + + CAM_LUN_INVALID - the LUN + ID in the request is not supported by the SCSI + controller + + CAM_TID_INVALID - the + target ID in the request is not supported by the SCSI + controller + + + + + Timeout Handling + + When the timeout for an HCB expires that request should be + aborted, just like with an XPT_ABORT request. The only + difference is that the returned status of aborted request should + be CAM_CMD_TIMEOUT instead of CAM_REQ_ABORTED (that's why + implementation of the abort better be done as a function). But + there is one more possible problem: what if the abort request + itself will get stuck? In this case the SCSI bus should be + reset, just like with an XPT_RESET_BUS request (and the idea + about implementing it as a function called from both places + applies here too). Also we should reset the whole SCSI bus if a + device reset request got stuck. So after all the timeout + function would look like: + + +static void +xxx_timeout(void *arg) +{ + struct xxx_hcb *hcb = (struct xxx_hcb *)arg; + struct xxx_softc *softc; + struct ccb_hdr *ccb_h; + + softc = hcb->softc; + ccb_h = &hcb->ccb->ccb_h; + + if(hcb->flags & HCB_BEING_ABORTED + || ccb_h->func_code == XPT_RESET_DEV) { + xxx_reset_bus(softc); + } else { + xxx_abort_ccb(hcb->ccb, CAM_CMD_TIMEOUT); + } +} + + + When we abort a request all the other disconnected requests + to the same target/LUN get aborted too. So there appears a + question, should we return them with status CAM_REQ_ABORTED or + CAM_CMD_TIMEOUT ? The current drivers use CAM_CMD_TIMEOUT. This + seems logical because if one request got timed out then probably + something really bad is happening to the device, so if they + would not be disturbed they would time out by themselves. + + + + diff --git a/en_US.ISO8859-1/books/developers-handbook/book.sgml b/en_US.ISO8859-1/books/developers-handbook/book.sgml index 1006c9f5d2..2cf7d725aa 100644 --- a/en_US.ISO8859-1/books/developers-handbook/book.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/book.sgml @@ -1,520 +1,523 @@ %bookinfo; %man; %chapters; + %authors; ]> FreeBSD Developers' Handbook The FreeBSD Documentation Project
doc@FreeBSD.org
August 2000 2000 + 2001 The FreeBSD Documentation Project &bookinfo.legalnotice; Welcome to the Developers' Handbook. Introduction Developing on FreeBSD This will need to discuss FreeBSD as a development platform, the vision of BSD, architectural overview, layout of /usr/src, history, etc. Thank you for considering FreeBSD as your development platform! We hope it will not let you down. The BSD Vision Architectural Overview The Layout of /usr/src The complete source code to FreeBSD is available from our public CVS repository. The source code is normally installed in /usr/src which contains the following subdirectories. Directory Description bin/ Source for files in /bin contrib/ Source for files from contribued software. crypto/ DES source etc/ Source for files in /etc games/ Source for files in /usr/games gnu/ Utilities covered by the GNU Public License include/ Source for files in /usr/include kerberosIV/ Source for Kerbereros version IV kerberos5/ Source for Kerbereros version 5 lib/ Source for files in /usr/lib libexec/ Source for files in /usr/libexec release/ Files required to produce a FreeBSD release sbin/ Source for files in /sbin secure/ FreeSec sources share/ Source for files in /sbin sys/ Kernel source files tools/ Tools used for maintenance and testing of FreeBSD usr.bin/ Source for files in /usr/bin usr.sbin/ Source for files in /usr/sbin Basics &chap.tools; &chap.secure; Kernel History of the Unix Kernel Some history of the Unix/BSD kernel, system calls, how do processes work, blocking, scheduling, threads (kernel), context switching, signals, interrupts, modules, etc. &chap.locking; Memory and Virtual Memory Virtual Memory VM, paging, swapping, allocating memory, testing for memory leaks, mmap, vnodes, etc. I/O System UFS UFS, FFS, Ext2FS, JFS, inodes, buffer cache, labeling, locking, metadata, soft-updates, LFS, portalfs, procfs, vnodes, memory sharing, memory objects, TLBs, caching Interprocess Communication Signals Signals, pipes, semaphores, message queues, shared memory, ports, sockets, doors Networking Sockets Sockets, bpf, IP, TCP, UDP, ICMP, OSI, bridging, firewalling, NAT, switching, etc Network Filesystems AFS AFS, NFS, SANs etc] Terminal Handling Syscons Syscons, tty, PCVT, serial console, screen savers, etc Sound OSS OSS, waveforms, etc Device Drivers &chap.driverbasics; &chap.pci; + &chap.scsi; USB Devices This chapter will talk about the FreeBSD mechanisms for writing a device driver for a device on a USB bus. NewBus This chapter will talk about the FreeBSD NewBus architecture. Architectures IA-32 Talk about the architectural specifics of FreeBSD/x86. Alpha Talk about the architectural specifics of FreeBSD/alpha. Explanation of allignment errors, how to fix, how to ignore. Example assembly language code for FreeBSD/alpha. IA-64 Talk about the architectural specifics of FreeBSD/ia64. Debugging Truss various descriptions on how to debug certain aspects of the system using truss, ktrace, gdb, kgdb, etc Compatibility Layers Linux Linux, SVR4, etc Appendices Dave A Patterson John L Hennessy 1998Morgan Kaufmann Publishers, Inc. 1-55860-428-6 Morgan Kaufmann Publishers, Inc. Computer Organization and Design The Hardware / Software Interface 1-2 W. Richard Stevens 1993Addison Wesley Longman, Inc. 0-201-56317-7 Addison Wesley Longman, Inc. Advanced Programming in the Unix Environment 1-2 Marshall Kirk McKusick Keith Bostic Michael J Karels John S Quarterman 1996Addison-Wesley Publishing Company, Inc. 0-201-54979-4 Addison-Wesley Publishing Company, Inc. The Design and Implementation of the 4.4 BSD Operating System 1-2 Aleph One Phrack 49; "Smashing the Stack for Fun and Profit" Chrispin Cowan Calton Pu Dave Maier StackGuard; Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks Todd Miller Theo de Raadt strlcpy and strlcat -- consistent, safe string copy and concatenation. diff --git a/en_US.ISO8859-1/books/developers-handbook/chapters.ent b/en_US.ISO8859-1/books/developers-handbook/chapters.ent index c2149880e2..a367f40239 100644 --- a/en_US.ISO8859-1/books/developers-handbook/chapters.ent +++ b/en_US.ISO8859-1/books/developers-handbook/chapters.ent @@ -1,58 +1,59 @@ + diff --git a/en_US.ISO8859-1/books/developers-handbook/scsi/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/scsi/chapter.sgml new file mode 100644 index 0000000000..837a8fd60d --- /dev/null +++ b/en_US.ISO8859-1/books/developers-handbook/scsi/chapter.sgml @@ -0,0 +1,2079 @@ + + + + Common Access Method SCSI Controllers + + This chapter was written by &a.babkin; + Modifications for the handbook made by + &a.murray;. + + + Synopsis + + This document assumes that the reader has a general + understanding of device drivers in FreeBSD and of the SCSI + protocol. Much of the information in this document was + extracted from the drivers : + + + + ncr (/sys/pci/ncr.c) by + Wolfgang Stanglmeier and Stefan Esser + + sym (/sys/pci/sym.c) by + Gerard Roudier + + aic7xxx + (/sys/dev/aic7xxx/aic7xxx.c) by Justin + T. Gibbs + + + + and from the CAM code itself (by Justing T. Gibbs, see + /sys/cam/*). When some solution looked the + most logical and was essentially verbatim extracted from the code + by Justin Gibbs, I marked it as "recommended". + + The document is illustrated with examples in + pseudo-code. Although sometimes the examples have many details + and look like real code, it's still pseudo-code. It was written + to demonstrate the concepts in an understandable way. For a real + driver other approaches may be more modular and efficient. It + also abstracts from the hardware details, as well as issues that + would cloud the demonstrated concepts or that are supposed to be + described in the other chapters of the developers handbook. Such + details are commonly shown as calls to functions with descriptive + names, comments or pseudo-statements. Fortunately real life + full-size examples with all the details can be found in the real + drivers. + + + + + General architecture + + CAM stands for Common Access Method. It's a generic way to + address the I/O buses in a SCSI-like way. This allows a + separation of the generic device drivers from the drivers + controlling the I/O bus: for example the disk driver becomes able + to control disks on both SCSI, IDE, and/or any other bus so the + disk driver portion does not have to be rewritten (or copied and + modified) for every new I/O bus. Thus the two most important + active entities are: + + + Peripheral Modules - a + driver for peripheral devices (disk, tape, CDROM, + etc.) + SCSI Interface Modules (SIM) + - a Host Bus Adapter drivers for connecting to an I/O bus such + as SCSI or IDE. + + + A peripheral driver receives requests from the OS, converts + them to a sequence of SCSI commands and passes these SCSI + commands to a SCSI Interface Module. The SCSI Interface Module + is responsible for passing these commands to the actual hardware + (or if the actual hardware is not SCSI but, for example, IDE + then also converting the SCSI commands to the native commands of + the hardware). + + Because we are interested in writing a SCSI adapter driver + here, from this point on we will consider everything from the + SIM standpoint. + + A typical SIM driver needs to include the following + CAM-related header files: + + +#include <cam/cam.h> +#include <cam/cam_ccb.h> +#include <cam/cam_sim.h> +#include <cam/cam_xpt_sim.h> +#include <cam/cam_debug.h> +#include <cam/scsi/scsi_all.h> + + + The first thing each SIM driver must do is register itself + with the CAM subsystem. This is done during the driver's + xxx_attach() function (here and further + xxx_ is used to denote the unique driver name prefix). The + xxx_attach() function itself is called by + the system bus auto-configuration code which we don't describe + here. + + This is achieved in multiple steps: first it's necessary to + allocate the queue of requests associated with this SIM: + + + struct cam_devq *devq; + + if(( devq = cam_simq_alloc(SIZE) )==NULL) { + error; /* some code to handle the error */ + } + + + Here SIZE is the size of the queue to be allocated, maximal + number of requests it could contain. It's the number of requests + that the SIM driver can handle in parallel on one SCSI + card. Commonly it can be calculated as: + + +SIZE = NUMBER_OF_SUPPORTED_TARGETS * MAX_SIMULTANEOUS_COMMANDS_PER_TARGET + + + Next we create a descriptor of our SIM: + + + struct cam_sim *sim; + + if(( sim = cam_sim_alloc(action_func, poll_func, driver_name, + softc, unit, max_dev_transactions, + max_tagged_dev_transactions, devq) )==NULL) { + cam_simq_free(devq); + error; /* some code to handle the error */ + } + + + Note that if we are not able to create a SIM descriptor we + free the devq also because we can do + nothing else with it and we want to conserve memory. + + If a SCSI card has multiple SCSI buses on it then each bus + requires its own cam_sim + structure. + + An interesting question is what to do if a SCSI card has + more than one SCSI bus, do we need one + devq structure per card or per SCSI + bus? The answer given in the comments to the CAM code is: + either way, as the driver's author prefers. + + The arguments are : + + + action_func - pointer to + the driver's xxx_action function. + + static void + xxx_action + + + struct cam_sim *sim, + union ccb *ccb + + + + + poll_func - pointer to + the driver's xxx_poll() + + static void + xxx_poll + + + struct cam_sim *sim + + + + + driver_name - the name of the actual driver, + such as "ncr" or "wds" + + softc - pointer to the + driver's internal descriptor for this SCSI card. This + pointer will be used by the driver in future to get private + data. + + unit - the controller unit number, for example + for controller "wds0" this number will be + 0 + + max_dev_transactions - maximal number of + simultaneous transactions per SCSI target in the non-tagged + mode. This value will be almost universally equal to 1, with + possible exceptions only for the non-SCSI cards. Also the + drivers that hope to take advantage by preparing one + transaction while another one is executed may set it to 2 + but this does not seem to be worth the + complexity. + + max_tagged_dev_transactions - the same thing, + but in the tagged mode. Tags are the SCSI way to initiate + multiple transactions on a device: each transaction is + assigned a unique tag and the transaction is sent to the + device. When the device completes some transaction it sends + back the result together with the tag so that the SCSI + adapter (and the driver) can tell which transaction was + completed. This argument is also known as the maximal tag + depth. It depends on the abilities of the SCSI + adapter. + + + + Finally we register the SCSI buses associated with our SCSI + adapter: + + + if(xpt_bus_register(sim, bus_number) != CAM_SUCCESS) { + cam_sim_free(sim, /*free_devq*/ TRUE); + error; /* some code to handle the error */ + } + + + If there is one devq structure per + SCSI bus (i.e. we consider a card with multiple buses as + multiple cards with one bus each) then the bus number will + always be 0, otherwise each bus on the SCSI card should be get a + distinct number. Each bus needs its own separate structure + cam_sim. + + After that our controller is completely hooked to the CAM + system. The value of devq can be + discarded now: sim will be passed as an argument in all further + calls from CAM and devq can be derived from it. + + CAM provides the framework for such asynchronous + events. Some events originate from the lower levels (the SIM + drivers), some events originate from the peripheral drivers, + some events originate from the CAM subsystem itself. Any driver + can register callbacks for some types of the asynchronous + events, so that it would be notified if these events + occur. + + A typical example of such an event is a device reset. Each + transaction and event identifies the devices to which it applies + by the means of "path". The target-specific events normally + occur during a transaction with this device. So the path from + that transaction may be re-used to report this event (this is + safe because the event path is copied in the event reporting + routine but not deallocated nor passed anywhere further). Also + it's safe to allocate paths dynamically at any time including + the interrupt routines, although that incurs certain overhead, + and a possible problem with this approach is that there may be + no free memory at that time. For a bus reset event we need to + define a wildcard path including all devices on the bus. So we + can create the path for the future bus reset events in advance + and avoid problems with the future memory shortage: + + + struct cam_path *path; + + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), CAM_TARGET_WILDCARD, + CAM_LUN_WILDCARD) != CAM_REQ_CMP) { + xpt_bus_deregister(cam_sim_path(sim)); + cam_sim_free(sim, /*free_devq*/TRUE); + error; /* some code to handle the error */ + } + + softc->wpath = path; + softc->sim = sim; + + + As you can see the path includes: + + + ID of the peripheral driver (NULL here because we have + none) + + ID of the SIM driver + (cam_sim_path(sim)) + + SCSI target number of the device (CAM_TARGET_WILDCARD + means "all devices") + + SCSI LUN number of the subdevice (CAM_LUN_WILDCARD means + "all LUNs") + + + If the driver can't allocate this path it won't be able to + work normally, so in that case we dismantle that SCSI + bus. + + And we save the path pointer in the + softc structure for future use. After + that we save the value of sim (or we can also discard it on the + exit from xxx_probe() if we wish). + + That's all for a minimalistic initialization. To do things + right there is one more issue left. + + For a SIM driver there is one particularly interesting + event: when a target device is considered lost. In this case + resetting the SCSI negotiations with this device may be a good + idea. So we register a callback for this event with CAM. The + request is passed to CAM by requesting CAM action on a CAM + control block for this type of request: + + + struct ccb_setasync csa; + + xpt_setup_ccb(&csa.ccb_h, path, /*priority*/5); + csa.ccb_h.func_code = XPT_SASYNC_CB; + csa.event_enable = AC_LOST_DEVICE; + csa.callback = xxx_async; + csa.callback_arg = sim; + xpt_action((union ccb *)&csa); + + + Now we take a look at the xxx_action() + and xxx_poll() driver entry points. + + + + static void + xxx_action + + + struct cam_sim *sim, + union ccb *ccb + + + + + Do some action on request of the CAM subsystem. Sim + describes the SIM for the request, CCB is the request + itself. CCB stands for "CAM Control Block". It is a union of + many specific instances, each describing arguments for some type + of transactions. All of these instances share the CCB header + where the common part of arguments is stored. + + CAM supports the SCSI controllers working in both initiator + ("normal") mode and target (simulating a SCSI device) mode. Here + we only consider the part relevant to the initiator mode. + + There are a few function and macros (in other words, + methods) defined to access the public data in the struct sim: + + + cam_sim_path(sim) - the + path ID (see above) + + cam_sim_name(sim) - the + name of the sim + + cam_sim_softc(sim) - the + pointer to the softc (driver private data) + structure + + cam_sim_unit(sim) - the + unit number + + cam_sim_bus(sim) - the bus + ID + + + To identify the device, xxx_action() can + get the unit number and pointer to its structure softc using + these functions. + + The type of request is stored in + ccb->ccb_h.func_code. So generally + xxx_action() consists of a big + switch: + + + struct xxx_softc *softc = (struct xxx_softc *) cam_sim_softc(sim); + struct ccb_hdr *ccb_h = &ccb->ccb_h; + int unit = cam_sim_unit(sim); + int bus = cam_sim_bus(sim); + + switch(ccb_h->func_code) { + case ...: + ... + default: + ccb_h->status = CAM_REQ_INVALID; + xpt_done(ccb); + break; + } + + + As can be seen from the default case (if an unknown command + was received) the return code of the command is set into + ccb->ccb_h.status and the completed + CCB is returned back to CAM by calling + xpt_done(ccb). + + xpt_done() does not have to be called + from xxx_action(): For example an I/O + request may be enqueued inside the SIM driver and/or its SCSI + controller. Then when the device would post an interrupt + signaling that the processing of this request is complete + xpt_done() may be called from the interrupt + handling routine. + + Actually, the CCB status is not only assigned as a return + code but a CCB has some status all the time. Before CCB is + passed to the xxx_action() routine it gets + the status CCB_REQ_INPROG meaning that it's in progress. There + are a surprising number of status values defined in + /sys/cam/cam.h which should be able to + represent the status of a request in great detail. More + interesting yet, the status is in fact a "bitwise or" of an + enumerated status value (the lower 6 bits) and possible + additional flag-like bits (the upper bits). The enumerated + values will be discussed later in more detail. The summary of + them can be found in the Errors Summary section. The possible + status flags are: + + + + CAM_DEV_QFRZN - if the + SIM driver gets a serious error (for example, the device does + not respond to the selection or breaks the SCSI protocol) when + processing a CCB it should freeze the request queue by calling + xpt_freeze_simq(), return the other + enqueued but not processed yet CCBs for this device back to + the CAM queue, then set this flag for the troublesome CCB and + call xpt_done(). This flag causes the CAM + subsystem to unfreeze the queue after it handles the + error. + + CAM_AUTOSNS_VALID - if + the device returned an error condition and the flag + CAM_DIS_AUTOSENSE is not set in CCB the SIM driver must + execute the REQUEST SENSE command automatically to extract the + sense (extended error information) data from the device. If + this attempt was successful the sense data should be saved in + the CCB and this flag set. + + CAM_RELEASE_SIMQ - like + CAM_DEV_QFRZN but used in case there is some problem (or + resource shortage) with the SCSI controller itself. Then all + the future requests to the controller should be stopped by + xpt_freeze_simq(). The controller queue + will be restarted after the SIM driver overcomes the shortage + and informs CAM by returning some CCB with this flag + set. + + CAM_SIM_QUEUED - when SIM + puts a CCB into its request queue this flag should be set (and + removed when this CCB gets dequeued before being returned back + to CAM). This flag is not used anywhere in the CAM code now, + so its purpose is purely diagnostic. + + + + The function xxx_action() is not + allowed to sleep, so all the synchronization for resource access + must be done using SIM or device queue freezing. Besides the + aforementioned flags the CAM subsystem provides functions + xpt_selease_simq() and + xpt_release_devq() to unfreeze the queues + directly, without passing a CCB to CAM. + + The CCB header contains the following fields: + + + + path - path ID for the + request + + target_id - target device + ID for the request + + target_lun - LUN ID of + the target device + + timeout - timeout + interval for this command, in milliseconds + + timeout_ch - a + convenience place for the SIM driver to store the timeout handle + (the CAM subsystem itself does not make any assumptions about + it) + + flags - various bits of + information about the request spriv_ptr0, spriv_ptr1 - fields + reserved for private use by the SIM driver (such as linking to + the SIM queues or SIM private control blocks); actually, they + exist as unions: spriv_ptr0 and spriv_ptr1 have the type (void + *), spriv_field0 and spriv_field1 have the type unsigned long, + sim_priv.entries[0].bytes and sim_priv.entries[1].bytes are byte + arrays of the size consistent with the other incarnations of the + union and sim_priv.bytes is one array, twice + bigger. + + + + The recommended way of using the SIM private fields of CCB + is to define some meaningful names for them and use these + meaningful names in the driver, like: + + +#define ccb_some_meaningful_name sim_priv.entries[0].bytes +#define ccb_hcb spriv_ptr1 /* for hardware control block */ + + + The most common initiator mode requests are: + + XPT_SCSI_IO - execute an + I/O transaction + + The instance "struct ccb_scsiio csio" of the union ccb is + used to transfer the arguments. They are: + + + cdb_io - pointer to + the SCSI command buffer or the buffer + itself + + cdb_len - SCSI + command length + + data_ptr - pointer to + the data buffer (gets a bit complicated if scatter/gather is + used) + + dxfer_len - length of + the data to transfer + + sglist_cnt - counter + of the scatter/gather segments + + scsi_status - place + to return the SCSI status + + sense_data - buffer + for the SCSI sense information if the command returns an + error (the SIM driver is supposed to run the REQUEST SENSE + command automatically in this case if the CCB flag + CAM_DIS_AUTOSENSE is not set) + + sense_len - the + length of that buffer (if it happens to be higher than size + of sense_data the SIM driver must silently assume the + smaller value) resid, sense_resid - if the transfer of data + or SCSI sense returned an error these are the returned + counters of the residual (not transferred) data. They do not + seem to be especially meaningful, so in a case when they are + difficult to compute (say, counting bytes in the SCSI + controller's FIFO buffer) an approximate value will do as + well. For a successfully completed transfer they must be set + to zero. + + tag_action - the kind + of tag to use: + + + CAM_TAG_ACTION_NONE - don't use tags for this + transaction + MSG_SIMPLE_Q_TAG, MSG_HEAD_OF_Q_TAG, + MSG_ORDERED_Q_TAG - value equal to the appropriate tag + message (see /sys/cam/scsi/scsi_message.h); this gives only + the tag type, the SIM driver must assign the tag value + itself + + + + + + + The general logic of handling this request is the + following: + + The first thing to do is to check for possible races, to + make sure that the command did not get aborted when it was + sitting in the queue: + + + struct ccb_scsiio *csio = &ccb->csio; + + if ((ccb_h->status & CAM_STATUS_MASK) != CAM_REQ_INPROG) { + xpt_done(ccb); + return; + } + + + Also we check that the device is supported at all by our + controller: + + + if(ccb_h->target_id > OUR_MAX_SUPPORTED_TARGET_ID + || cch_h->target_id == OUR_SCSI_CONTROLLERS_OWN_ID) { + ccb_h->status = CAM_TID_INVALID; + xpt_done(ccb); + return; + } + if(ccb_h->target_lun > OUR_MAX_SUPPORTED_LUN) { + ccb_h->status = CAM_LUN_INVALID; + xpt_done(ccb); + return; + } + + + Then allocate whatever data structures (such as + card-dependent hardware control block) we need to process this + request. If we can't then freeze the SIM queue and remember + that we have a pending operation, return the CCB back and ask + CAM to re-queue it. Later when the resources become available + the SIM queue must be unfrozen by returning a ccb with the + CAM_SIMQ_RELEASE bit set in its status. Otherwise, if all went + well, link the CCB with the hardware control block (HCB) and + mark it as queued. + + + struct xxx_hcb *hcb = allocate_hcb(softc, unit, bus); + + if(hcb == NULL) { + softc->flags |= RESOURCE_SHORTAGE; + xpt_freeze_simq(sim, /*count*/1); + ccb_h->status = CAM_REQUEUE_REQ; + xpt_done(ccb); + return; + } + + hcb->ccb = ccb; ccb_h->ccb_hcb = (void *)hcb; + ccb_h->status |= CAM_SIM_QUEUED; + + + Extract the target data from CCB into the hardware control + block. Check if we are asked to assign a tag and if yes then + generate an unique tag and build the SCSI tag messages. The + SIM driver is also responsible for negotiations with the + devices to set the maximal mutually supported bus width, + synchronous rate and offset. + + + hcb->target = ccb_h->target_id; hcb->lun = ccb_h->target_lun; + generate_identify_message(hcb); + if( ccb_h->tag_action != CAM_TAG_ACTION_NONE ) + generate_unique_tag_message(hcb, ccb_h->tag_action); + if( !target_negotiated(hcb) ) + generate_negotiation_messages(hcb); + + + Then set up the SCSI command. The command storage may be + specified in the CCB in many interesting ways, specified by + the CCB flags. The command buffer can be contained in CCB or + pointed to, in the latter case the pointer may be physical or + virtual. Since the hardware commonly needs physical address we + always convert the address to the physical one. + + A NOT-QUITE RELATED NOTE: Normally this is done by a call + to vtophys(), but for the PCI device (which account for most + of the SCSI controllers now) drivers' portability to the Alpha + architecture the conversion must be done by vtobus() instead + due to special Alpha quirks. [IMHO it would be much better to + have two separate functions, vtop() and ptobus() then vtobus() + would be a simple superposition of them.] In case if a + physical address is requested it's OK to return the CCB with + the status CAM_REQ_INVALID, the current drivers do that. But + it's also possible to compile the Alpha-specific piece of + code, as in this example (there should be a more direct way to + do that, without conditional compilation in the drivers). If + necessary a physical address can be also converted or mapped + back to a virtual address but with big pain, so we don't do + that. + + + if(ccb_h->flags & CAM_CDB_POINTER) { + /* CDB is a pointer */ + if(!(ccb_h->flags & CAM_CDB_PHYS)) { + /* CDB pointer is virtual */ + hcb->cmd = vtobus(csio->cdb_io.cdb_ptr); + } else { + /* CDB pointer is physical */ +#if defined(__alpha__) + hcb->cmd = csio->cdb_io.cdb_ptr | alpha_XXX_dmamap_or ; +#else + hcb->cmd = csio->cdb_io.cdb_ptr ; +#endif + } + } else { + /* CDB is in the ccb (buffer) */ + hcb->cmd = vtobus(csio->cdb_io.cdb_bytes); + } + hcb->cmdlen = csio->cdb_len; + + + Now it's time to set up the data. Again, the data storage + may be specified in the CCB in many interesting ways, + specified by the CCB flags. First we get the direction of the + data transfer. The simplest case is if there is no data to + transfer: + + + int dir = (ccb_h->flags & CAM_DIR_MASK); + + if (dir == CAM_DIR_NONE) + goto end_data; + + + Then we check if the data is in one chunk or in a + scatter-gather list, and the addresses are physical or + virtual. The SCSI controller may be able to handle only a + limited number of chunks of limited length. If the request + hits this limitation we return an error. We use a special + function to return the CCB to handle in one place the HCB + resource shortages. The functions to add chunks are + driver-dependent, and here we leave them without detailed + implementation. See description of the SCSI command (CDB) + handling for the details on the address-translation issues. + If some variation is too difficult or impossible to implement + with a particular card it's OK to return the status + CAM_REQ_INVALID. Actually, it seems like the scatter-gather + ability is not used anywhere in the CAM code now. But at least + the case for a single non-scattered virtual buffer must be + implemented, it's actively used by CAM. + + + int rv; + + initialize_hcb_for_data(hcb); + + if((!(ccb_h->flags & CAM_SCATTER_VALID)) { + /* single buffer */ + if(!(ccb_h->flags & CAM_DATA_PHYS)) { + rv = add_virtual_chunk(hcb, csio->data_ptr, csio->dxfer_len, dir); + } + } else { + rv = add_physical_chunk(hcb, csio->data_ptr, csio->dxfer_len, dir); + } + } else { + int i; + struct bus_dma_segment *segs; + segs = (struct bus_dma_segment *)csio->data_ptr; + + if ((ccb_h->flags & CAM_SG_LIST_PHYS) != 0) { + /* The SG list pointer is physical */ + rv = setup_hcb_for_physical_sg_list(hcb, segs, csio->sglist_cnt); + } else if (!(ccb_h->flags & CAM_DATA_PHYS)) { + /* SG buffer pointers are virtual */ + for (i = 0; i < csio->sglist_cnt; i++) { + rv = add_virtual_chunk(hcb, segs[i].ds_addr, + segs[i].ds_len, dir); + if (rv != CAM_REQ_CMP) + break; + } + } else { + /* SG buffer pointers are physical */ + for (i = 0; i < csio->sglist_cnt; i++) { + rv = add_physical_chunk(hcb, segs[i].ds_addr, + segs[i].ds_len, dir); + if (rv != CAM_REQ_CMP) + break; + } + } + } + if(rv != CAM_REQ_CMP) { + /* we expect that add_*_chunk() functions return CAM_REQ_CMP + * if they added a chunk successfully, CAM_REQ_TOO_BIG if + * the request is too big (too many bytes or too many chunks), + * CAM_REQ_INVALID in case of other troubles + */ + free_hcb_and_ccb_done(hcb, ccb, rv); + return; + } + end_data: + + + If disconnection is disabled for this CCB we pass this + information to the hcb: + + + if(ccb_h->flags & CAM_DIS_DISCONNECT) + hcb_disable_disconnect(hcb); + + + If the controller is able to run REQUEST SENSE command all + by itself then the value of the flag CAM_DIS_AUTOSENSE should + also be passed to it, to prevent automatic REQUEST SENSE if the + CAM subsystem does not want it. + + The only thing left is to set up the timeout, pass our hcb + to the hardware and return, the rest will be done by the + interrupt handler (or timeout handler). + + + ccb_h->timeout_ch = timeout(xxx_timeout, (caddr_t) hcb, + (ccb_h->timeout * hz) / 1000); /* convert milliseconds to ticks */ + put_hcb_into_hardware_queue(hcb); + return; + + + And here is a possible implementation of the function + returning CCB: + + + static void + free_hcb_and_ccb_done(struct xxx_hcb *hcb, union ccb *ccb, u_int32_t status) + { + struct xxx_softc *softc = hcb->softc; + + ccb->ccb_h.ccb_hcb = 0; + if(hcb != NULL) { + untimeout(xxx_timeout, (caddr_t) hcb, ccb->ccb_h.timeout_ch); + /* we're about to free a hcb, so the shortage has ended */ + if(softc->flags & RESOURCE_SHORTAGE) { + softc->flags &= ~RESOURCE_SHORTAGE; + status |= CAM_RELEASE_SIMQ; + } + free_hcb(hcb); /* also removes hcb from any internal lists */ + } + ccb->ccb_h.status = status | + (ccb->ccb_h.status & ~(CAM_STATUS_MASK|CAM_SIM_QUEUED)); + xpt_done(ccb); + } + + + + XPT_RESET_DEV - send the SCSI "BUS + DEVICE RESET" message to a device + + There is no data transferred in CCB except the header and + the most interesting argument of it is target_id. Depending on + the controller hardware a hardware control block just like for + the XPT_SCSI_IO request may be constructed (see XPT_SCSI_IO + request description) and sent to the controller or the SCSI + controller may be immediately programmed to send this RESET + message to the device or this request may be just not supported + (and return the status CAM_REQ_INVALID). Also on completion of + the request all the disconnected transactions for this target + must be aborted (probably in the interrupt routine). + + Also all the current negotiations for the target are lost on + reset, so they might be cleaned too. Or they clearing may be + deferred, because anyway the target would request re-negotiation + on the next transaction. + + XPT_RESET_BUS - send the RESET signal + to the SCSI bus + + No arguments are passed in the CCB, the only interesting + argument is the SCSI bus indicated by the struct sim + pointer. + + A minimalistic implementation would forget the SCSI + negotiations for all the devices on the bus and return the + status CAM_REQ_CMP. + + The proper implementation would in addition actually reset + the SCSI bus (possible also reset the SCSI controller) and mark + all the CCBs being processed, both those in the hardware queue + and those being disconnected, as done with the status + CAM_SCSI_BUS_RESET. Like: + + + int targ, lun; + struct xxx_hcb *h, *hh; + struct ccb_trans_settings neg; + struct cam_path *path; + + /* The SCSI bus reset may take a long time, in this case its completion + * should be checked by interrupt or timeout. But for simplicity + * we assume here that it's really fast. + */ + reset_scsi_bus(softc); + + /* drop all enqueued CCBs */ + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* the clean values of negotiations to report */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + + /* drop all disconnected CCBs and clean negotiations */ + for(targ=0; targ <= OUR_MAX_SUPPORTED_TARGET; targ++) { + clean_negotiations(softc, targ); + + /* report the event if possible */ + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), targ, + CAM_LUN_WILDCARD) == CAM_REQ_CMP) { + xpt_async(AC_TRANSFER_NEG, path, &neg); + xpt_free_path(path); + } + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[targ][lun]; h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + } + + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + + /* report the event */ + xpt_async(AC_BUS_RESET, softc->wpath, NULL); + return; + + + Implementing the SCSI bus reset as a function may be a good + idea because it would be re-used by the timeout function as a + last resort if the things go wrong. + + XPT_ABORT - abort the specified + CCB + + The arguments are transferred in the instance "struct + ccb_abort cab" of the union ccb. The only argument field in it + is: + + abort_ccb - pointer to the CCB to be + aborted + + If the abort is not supported just return the status + CAM_UA_ABORT. This is also the easy way to minimally implement + this call, return CAM_UA_ABORT in any case. + + The hard way is to implement this request honestly. First + check that abort applies to a SCSI transaction: + + + struct ccb *abort_ccb; + abort_ccb = ccb->cab.abort_ccb; + + if(abort_ccb->ccb_h.func_code != XPT_SCSI_IO) { + ccb->ccb_h.status = CAM_UA_ABORT; + xpt_done(ccb); + return; + } + + + Then it's necessary to find this CCB in our queue. This can + be done by walking the list of all our hardware control blocks + in search for one associated with this CCB: + + + struct xxx_hcb *hcb, *h; + + hcb = NULL; + + /* We assume that softc->first_hcb is the head of the list of all + * HCBs associated with this bus, including those enqueued for + * processing, being processed by hardware and disconnected ones. + */ + for(h = softc->first_hcb; h != NULL; h = h->next) { + if(h->ccb == abort_ccb) { + hcb = h; + break; + } + } + + if(hcb == NULL) { + /* no such CCB in our queue */ + ccb->ccb_h.status = CAM_PATH_INVALID; + xpt_done(ccb); + return; + } + + hcb=found_hcb; + + + Now we look at the current processing status of the HCB. It + may be either sitting in the queue waiting to be sent to the + SCSI bus, being transferred right now, or disconnected and + waiting for the result of the command, or actually completed by + hardware but not yet marked as done by software. To make sure + that we don't get in any races with hardware we mark the HCB as + being aborted, so that if this HCB is about to be sent to the + SCSI bus the SCSI controller will see this flag and skip + it. + + + int hstatus; + + /* shown as a function, in case special action is needed to make + * this flag visible to hardware + */ + set_hcb_flags(hcb, HCB_BEING_ABORTED); + + abort_again: + + hstatus = get_hcb_status(hcb); + switch(hstatus) { + case HCB_SITTING_IN_QUEUE: + remove_hcb_from_hardware_queue(hcb); + /* FALLTHROUGH */ + case HCB_COMPLETED: + /* this is an easy case */ + free_hcb_and_ccb_done(hcb, abort_ccb, CAM_REQ_ABORTED); + break; + + + If the CCB is being transferred right now we would like to + signal to the SCSI controller in some hardware-dependent way + that we want to abort the current transfer. The SCSI controller + would set the SCSI ATTENTION signal and when the target responds + to it send an ABORT message. We also reset the timeout to make + sure that the target is not sleeping forever. If the command + would not get aborted in some reasonable time like 10 seconds + the timeout routine would go ahead and reset the whole SCSI bus. + Because the command will be aborted in some reasonable time we + can just return the abort request now as successfully completed, + and mark the aborted CCB as aborted (but not mark it as done + yet). + + + case HCB_BEING_TRANSFERRED: + untimeout(xxx_timeout, (caddr_t) hcb, abort_ccb->ccb_h.timeout_ch); + abort_ccb->ccb_h.timeout_ch = + timeout(xxx_timeout, (caddr_t) hcb, 10 * hz); + abort_ccb->ccb_h.status = CAM_REQ_ABORTED; + /* ask the controller to abort that HCB, then generate + * an interrupt and stop + */ + if(signal_hardware_to_abort_hcb_and_stop(hcb) < 0) { + /* oops, we missed the race with hardware, this transaction + * got off the bus before we aborted it, try again */ + goto abort_again; + } + + break; + + + If the CCB is in the list of disconnected then set it up as + an abort request and re-queue it at the front of hardware + queue. Reset the timeout and report the abort request to be + completed. + + + case HCB_DISCONNECTED: + untimeout(xxx_timeout, (caddr_t) hcb, abort_ccb->ccb_h.timeout_ch); + abort_ccb->ccb_h.timeout_ch = + timeout(xxx_timeout, (caddr_t) hcb, 10 * hz); + put_abort_message_into_hcb(hcb); + put_hcb_at_the_front_of_hardware_queue(hcb); + break; + } + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + That's all for the ABORT request, although there is one more + issue. Because the ABORT message cleans all the ongoing + transactions on a LUN we have to mark all the other active + transactions on this LUN as aborted. That should be done in the + interrupt routine, after the transaction gets aborted. + + Implementing the CCB abort as a function may be quite a good + idea, this function can be re-used if an I/O transaction times + out. The only difference would be that the timed out transaction + would return the status CAM_CMD_TIMEOUT for the timed out + request. Then the case XPT_ABORT would be small, like + that: + + + case XPT_ABORT: + struct ccb *abort_ccb; + abort_ccb = ccb->cab.abort_ccb; + + if(abort_ccb->ccb_h.func_code != XPT_SCSI_IO) { + ccb->ccb_h.status = CAM_UA_ABORT; + xpt_done(ccb); + return; + } + if(xxx_abort_ccb(abort_ccb, CAM_REQ_ABORTED) < 0) + /* no such CCB in our queue */ + ccb->ccb_h.status = CAM_PATH_INVALID; + else + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + + XPT_SET_TRAN_SETTINGS - explicitly + set values of SCSI transfer settings + + The arguments are transferred in the instance "struct ccb_trans_setting cts" +of the union ccb: + + + valid - a bitmask showing + which settings should be updated: + + CCB_TRANS_SYNC_RATE_VALID + - synchronous transfer rate + + CCB_TRANS_SYNC_OFFSET_VALID + - synchronous offset + + CCB_TRANS_BUS_WIDTH_VALID + - bus width + + CCB_TRANS_DISC_VALID - + set enable/disable disconnection + + CCB_TRANS_TQ_VALID - set + enable/disable tagged queuing + + flags - consists of two + parts, binary arguments and identification of + sub-operations. The binary arguments are : + + CCB_TRANS_DISC_ENB - enable disconnection + CCB_TRANS_TAG_ENB - + enable tagged queuing + + + + the sub-operations are: + + CCB_TRANS_CURRENT_SETTINGS + - change the current negotiations + + CCB_TRANS_USER_SETTINGS + - remember the desired user values sync_period, sync_offset - + self-explanatory, if sync_offset==0 then the asynchronous mode + is requested bus_width - bus width, in bits (not + bytes) + + + + + + Two sets of negotiated parameters are supported, the user + settings and the current settings. The user settings are not + really used much in the SIM drivers, this is mostly just a piece + of memory where the upper levels can store (and later recall) + its ideas about the parameters. Setting the user parameters + does not cause re-negotiation of the transfer rates. But when + the SCSI controller does a negotiation it must never set the + values higher than the user parameters, so it's essentially the + top boundary. + + The current settings are, as the name says, + current. Changing them means that the parameters must be + re-negotiated on the next transfer. Again, these "new current + settings" are not supposed to be forced on the device, just they + are used as the initial step of negotiations. Also they must be + limited by actual capabilities of the SCSI controller: for + example, if the SCSI controller has 8-bit bus and the request + asks to set 16-bit wide transfers this parameter must be + silently truncated to 8-bit transfers before sending it to the + device. + + One caveat is that the bus width and synchronous parameters + are per target while the disconnection and tag enabling + parameters are per lun. + + The recommended implementation is to keep 3 sets of + negotiated (bus width and synchronous transfer) + parameters: + + + user - the user set, as + above + + current - those actually + in effect + + goal - those requested by + setting of the "current" parameters + + + The code looks like: + + + struct ccb_trans_settings *cts; + int targ, lun; + int flags; + + cts = &ccb->cts; + targ = ccb_h->target_id; + lun = ccb_h->target_lun; + flags = cts->flags; + if(flags & CCB_TRANS_USER_SETTINGS) { + if(flags & CCB_TRANS_SYNC_RATE_VALID) + softc->user_sync_period[targ] = cts->sync_period; + if(flags & CCB_TRANS_SYNC_OFFSET_VALID) + softc->user_sync_offset[targ] = cts->sync_offset; + if(flags & CCB_TRANS_BUS_WIDTH_VALID) + softc->user_bus_width[targ] = cts->bus_width; + + if(flags & CCB_TRANS_DISC_VALID) { + softc->user_tflags[targ][lun] &= ~CCB_TRANS_DISC_ENB; + softc->user_tflags[targ][lun] |= flags & CCB_TRANS_DISC_ENB; + } + if(flags & CCB_TRANS_TQ_VALID) { + softc->user_tflags[targ][lun] &= ~CCB_TRANS_TQ_ENB; + softc->user_tflags[targ][lun] |= flags & CCB_TRANS_TQ_ENB; + } + } + if(flags & CCB_TRANS_CURRENT_SETTINGS) { + if(flags & CCB_TRANS_SYNC_RATE_VALID) + softc->goal_sync_period[targ] = + max(cts->sync_period, OUR_MIN_SUPPORTED_PERIOD); + if(flags & CCB_TRANS_SYNC_OFFSET_VALID) + softc->goal_sync_offset[targ] = + min(cts->sync_offset, OUR_MAX_SUPPORTED_OFFSET); + if(flags & CCB_TRANS_BUS_WIDTH_VALID) + softc->goal_bus_width[targ] = min(cts->bus_width, OUR_BUS_WIDTH); + + if(flags & CCB_TRANS_DISC_VALID) { + softc->current_tflags[targ][lun] &= ~CCB_TRANS_DISC_ENB; + softc->current_tflags[targ][lun] |= flags & CCB_TRANS_DISC_ENB; + } + if(flags & CCB_TRANS_TQ_VALID) { + softc->current_tflags[targ][lun] &= ~CCB_TRANS_TQ_ENB; + softc->current_tflags[targ][lun] |= flags & CCB_TRANS_TQ_ENB; + } + } + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + Then when the next I/O request will be processed it will + check if it has to re-negotiate, for example by calling the + function target_negotiated(hcb). It can be implemented like + this: + + + int + target_negotiated(struct xxx_hcb *hcb) + { + struct softc *softc = hcb->softc; + int targ = hcb->targ; + + if( softc->current_sync_period[targ] != softc->goal_sync_period[targ] + || softc->current_sync_offset[targ] != softc->goal_sync_offset[targ] + || softc->current_bus_width[targ] != softc->goal_bus_width[targ] ) + return 0; /* FALSE */ + else + return 1; /* TRUE */ + } + + + After the values are re-negotiated the resulting values must + be assigned to both current and goal parameters, so for future + I/O transactions the current and goal parameters would be the + same and target_negotiated() would return + TRUE. When the card is initialized (in + xxx_attach()) the current negotiation + values must be initialized to narrow asynchronous mode, the goal + and current values must be initialized to the maximal values + supported by controller. + + XPT_GET_TRAN_SETTINGS - get values of + SCSI transfer settings + + This operations is the reverse of + XPT_SET_TRAN_SETTINGS. Fill up the CCB instance "struct + ccb_trans_setting cts" with data as requested by the flags + CCB_TRANS_CURRENT_SETTINGS or CCB_TRANS_USER_SETTINGS (if both + are set then the existing drivers return the current + settings). Set all the bits in the valid field. + + XPT_CALC_GEOMETRY - calculate logical + (BIOS) geometry of the disk + + The arguments are transferred in the instance "struct + ccb_calc_geometry ccg" of the union ccb: + + + + block_size - input, block + (A.K.A sector) size in bytes + + volume_size - input, + volume size in bytes + + cylinders - output, + logical cylinders + + heads - output, logical + heads + + secs_per_track - output, + logical sectors per track + + + + If the returned geometry differs much enough from what the + SCSI controller BIOS thinks and a disk on this SCSI controller + is used as bootable the system may not be able to boot. The + typical calculation example taken from the aic7xxx driver + is: + + + struct ccb_calc_geometry *ccg; + u_int32_t size_mb; + u_int32_t secs_per_cylinder; + int extended; + + ccg = &ccb->ccg; + size_mb = ccg->volume_size + / ((1024L * 1024L) / ccg->block_size); + extended = check_cards_EEPROM_for_extended_geometry(softc); + + if (size_mb > 1024 && extended) { + ccg->heads = 255; + ccg->secs_per_track = 63; + } else { + ccg->heads = 64; + ccg->secs_per_track = 32; + } + secs_per_cylinder = ccg->heads * ccg->secs_per_track; + ccg->cylinders = ccg->volume_size / secs_per_cylinder; + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + This gives the general idea, the exact calculation depends + on the quirks of the particular BIOS. If BIOS provides no way + set the "extended translation" flag in EEPROM this flag should + normally be assumed equal to 1. Other popular geometries + are: + + + 128 heads, 63 sectors - Symbios controllers + 16 heads, 63 sectors - old controllers + + + Some system BIOSes and SCSI BIOSes fight with each other + with variable success, for example a combination of Symbios + 875/895 SCSI and Phoenix BIOS can give geometry 128/63 after + power up and 255/63 after a hard reset or soft reboot. + + + XPT_PATH_INQ - path inquiry, in other + words get the SIM driver and SCSI controller (also known as HBA + - Host Bus Adapter) properties + + The properties are returned in the instance "struct +ccb_pathinq cpi" of the union ccb: + + + + version_num - the SIM driver version number, now + all drivers use 1 + + hba_inquiry - bitmask of features supported by + the controller: + + PI_MDP_ABLE - supports MDP message (something + from SCSI3?) + + PI_WIDE_32 - supports 32 bit wide + SCSI + + PI_WIDE_16 - supports 16 bit wide + SCSI + + PI_SDTR_ABLE - can negotiate synchronous + transfer rate + + PI_LINKED_CDB - supports linked + commands + + PI_TAG_ABLE - supports tagged + commands + + PI_SOFT_RST - supports soft reset alternative + (hard reset and soft reset are mutually exclusive within a + SCSI bus) + + target_sprt - flags for target mode support, 0 + if unsupported + + hba_misc - miscellaneous controller + features: + + PIM_SCANHILO - bus scans from high ID to low + ID + + PIM_NOREMOVE - removable devices not included in + scan + + PIM_NOINITIATOR - initiator role not + supported + + PIM_NOBUSRESET - user has disabled initial BUS + RESET + + hba_eng_cnt - mysterious HBA engine count, + something related to compression, now is always set to + 0 + + vuhba_flags - vendor-unique flags, unused + now + + max_target - maximal supported target ID (7 for + 8-bit bus, 15 for 16-bit bus, 127 for Fibre + Channel) + + max_lun - maximal supported LUN ID (7 for older + SCSI controllers, 63 for newer ones) + + async_flags - bitmask of installed Async + handler, unused now + + hpath_id - highest Path ID in the subsystem, + unused now + + unit_number - the controller unit number, + cam_sim_unit(sim) + + bus_id - the bus number, + cam_sim_bus(sim) + + initiator_id - the SCSI ID of the controller + itself + + base_transfer_speed - nominal transfer speed in + KB/s for asynchronous narrow transfers, equals to 3300 for + SCSI + + sim_vid - SIM driver's vendor id, a + zero-terminated string of maximal length SIM_IDLEN including + the terminating zero + + hba_vid - SCSI controller's vendor id, a + zero-terminated string of maximal length HBA_IDLEN including + the terminating zero + + dev_name - device driver name, a zero-terminated + string of maximal length DEV_IDLEN including the terminating + zero, equal to cam_sim_name(sim) + + + + The recommended way of setting the string fields is using + strncpy, like: + + + strncpy(cpi->dev_name, cam_sim_name(sim), DEV_IDLEN); + + + After setting the values set the status to CAM_REQ_CMP and mark the +CCB as done. + + + + + + + Polling + + + static void + xxx_poll + + + struct cam_sim *sim + + + + The poll function is used to simulate the interrupts when + the interrupt subsystem is not functioning (for example, when + the system has crashed and is creating the system dump). The CAM + subsystem sets the proper interrupt level before calling the + poll routine. So all it needs to do is to call the interrupt + routine (or the other way around, the poll routine may be doing + the real action and the interrupt routine would just call the + poll routine). Why bother about a separate function then ? + Because of different calling conventions. The + xxx_poll routine gets the struct cam_sim + pointer as its argument when the PCI interrupt routine by common + convention gets pointer to the struct + xxx_softc and the ISA interrupt routine + gets just the the device unit number. So the poll routine would + normally look as: + + +static void +xxx_poll(struct cam_sim *sim) +{ + xxx_intr((struct xxx_softc *)cam_sim_softc(sim)); /* for PCI device */ +} + + + or + + +static void +xxx_poll(struct cam_sim *sim) +{ + xxx_intr(cam_sim_unit(sim)); /* for ISA device */ +} + + + + + + Asynchronous Events + + If an asynchronous event callback has been set up then the + callback function should be defined. + + +static void +ahc_async(void *callback_arg, u_int32_t code, struct cam_path *path, void *arg) + + + + callback_arg - the value supplied when registering the + callback + + code - identifies the type of event + + path - identifies the devices to which the event + applies + + arg - event-specific argument + + + Implementation for a single type of event, AC_LOST_DEVICE, + looks like: + + + struct xxx_softc *softc; + struct cam_sim *sim; + int targ; + struct ccb_trans_settings neg; + + sim = (struct cam_sim *)callback_arg; + softc = (struct xxx_softc *)cam_sim_softc(sim); + switch (code) { + case AC_LOST_DEVICE: + targ = xpt_path_target_id(path); + if(targ <= OUR_MAX_SUPPORTED_TARGET) { + clean_negotiations(softc, targ); + /* send indication to CAM */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + xpt_async(AC_TRANSFER_NEG, path, &neg); + } + break; + default: + break; + } + + + + + + Interrupts + + The exact type of the interrupt routine depends on the type + of the peripheral bus (PCI, ISA and so on) to which the SCSI + controller is connected. + + The interrupt routines of the SIM drivers run at the + interrupt level splcam. So splcam() should + be used in the driver to synchronize activity between the + interrupt routine and the rest of the driver (for a + multiprocessor-aware driver things get yet more interesting but + we ignore this case here). The pseudo-code in this document + happily ignores the problems of synchronization. The real code + must not ignore them. A simple-minded approach is to set + splcam() on the entry to the other routines + and reset it on return thus protecting them by one big critical + section. To make sure that the interrupt level will be always + restored a wrapper function can be defined, like: + + + static void + xxx_action(struct cam_sim *sim, union ccb *ccb) + { + int s; + s = splcam(); + xxx_action1(sim, ccb); + splx(s); + } + + static void + xxx_action1(struct cam_sim *sim, union ccb *ccb) + { + ... process the request ... + } + + + This approach is simple and robust but the problem with it + is that interrupts may get blocked for a relatively long time + and this would negatively affect the system's performance. On + the other hand the functions of the spl() + family have rather high overhead, so vast amount of tiny + critical sections may not be good either. + + The conditions handled by the interrupt routine and the + details depend very much on the hardware. We consider the set of + "typical" conditions. + + First, we check if a SCSI reset was encountered on the bus + (probably caused by another SCSI controller on the same SCSI + bus). If so we drop all the enqueued and disconnected requests, + report the events and re-initialize our SCSI controller. It is + important that during this initialization the controller won't + issue another reset or else two controllers on the same SCSI bus + could ping-pong resets forever. The case of fatal controller + error/hang could be handled in the same place, but it will + probably need also sending RESET signal to the SCSI bus to reset + the status of the connections with the SCSI devices. + + + int fatal=0; + struct ccb_trans_settings neg; + struct cam_path *path; + + if( detected_scsi_reset(softc) + || (fatal = detected_fatal_controller_error(softc)) ) { + int targ, lun; + struct xxx_hcb *h, *hh; + + /* drop all enqueued CCBs */ + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* the clean values of negotiations to report */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + + /* drop all disconnected CCBs and clean negotiations */ + for(targ=0; targ <= OUR_MAX_SUPPORTED_TARGET; targ++) { + clean_negotiations(softc, targ); + + /* report the event if possible */ + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), targ, + CAM_LUN_WILDCARD) == CAM_REQ_CMP) { + xpt_async(AC_TRANSFER_NEG, path, &neg); + xpt_free_path(path); + } + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[targ][lun]; h != NULL; h = hh) { + hh=h->next; + if(fatal) + free_hcb_and_ccb_done(h, h->ccb, CAM_UNREC_HBA_ERROR); + else + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + } + + /* report the event */ + xpt_async(AC_BUS_RESET, softc->wpath, NULL); + + /* re-initialization may take a lot of time, in such case + * its completion should be signaled by another interrupt or + * checked on timeout - but for simplicity we assume here that + * it's really fast + */ + if(!fatal) { + reinitialize_controller_without_scsi_reset(softc); + } else { + reinitialize_controller_with_scsi_reset(softc); + } + schedule_next_hcb(softc); + return; + } + + + If interrupt is not caused by a controller-wide condition + then probably something has happened to the current hardware + control block. Depending on the hardware there may be other + non-HCB-related events, we just do not consider them here. Then + we analyze what happened to this HCB: + + + struct xxx_hcb *hcb, *h, *hh; + int hcb_status, scsi_status; + int ccb_status; + int targ; + int lun_to_freeze; + + hcb = get_current_hcb(softc); + if(hcb == NULL) { + /* either stray interrupt or something went very wrong + * or this is something hardware-dependent + */ + handle as necessary; + return; + } + + targ = hcb->target; + hcb_status = get_status_of_current_hcb(softc); + + + First we check if the HCB has completed and if so we check + the returned SCSI status. + + + if(hcb_status == COMPLETED) { + scsi_status = get_completion_status(hcb); + + + Then look if this status is related to the REQUEST SENSE + command and if so handle it in a simple way. + + + if(hcb->flags & DOING_AUTOSENSE) { + if(scsi_status == GOOD) { /* autosense was successful */ + hcb->ccb->ccb_h.status |= CAM_AUTOSNS_VALID; + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_SCSI_STATUS_ERROR); + } else { + autosense_failed: + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_AUTOSENSE_FAIL); + } + schedule_next_hcb(softc); + return; + } + + + Else the command itself has completed, pay more attention to + details. If auto-sense is not disabled for this CCB and the + command has failed with sense data then run REQUEST SENSE + command to receive that data. + + + hcb->ccb->csio.scsi_status = scsi_status; + calculate_residue(hcb); + + if( (hcb->ccb->ccb_h.flags & CAM_DIS_AUTOSENSE)==0 + && ( scsi_status == CHECK_CONDITION + || scsi_status == COMMAND_TERMINATED) ) { + /* start auto-SENSE */ + hcb->flags |= DOING_AUTOSENSE; + setup_autosense_command_in_hcb(hcb); + restart_current_hcb(softc); + return; + } + if(scsi_status == GOOD) + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_REQ_CMP); + else + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_SCSI_STATUS_ERROR); + schedule_next_hcb(softc); + return; + } + + + One typical thing would be negotiation events: negotiation + messages received from a SCSI target (in answer to our + negotiation attempt or by target's initiative) or the target is + unable to negotiate (rejects our negotiation messages or does + not answer them). + + + switch(hcb_status) { + case TARGET_REJECTED_WIDE_NEG: + /* revert to 8-bit bus */ + softc->current_bus_width[targ] = softc->goal_bus_width[targ] = 8; + /* report the event */ + neg.bus_width = 8; + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + continue_current_hcb(softc); + return; + case TARGET_ANSWERED_WIDE_NEG: + { + int wd; + + wd = get_target_bus_width_request(softc); + if(wd <= softc->goal_bus_width[targ]) { + /* answer is acceptable */ + softc->current_bus_width[targ] = + softc->goal_bus_width[targ] = neg.bus_width = wd; + + /* report the event */ + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + } else { + prepare_reject_message(hcb); + } + } + continue_current_hcb(softc); + return; + case TARGET_REQUESTED_WIDE_NEG: + { + int wd; + + wd = get_target_bus_width_request(softc); + wd = min (wd, OUR_BUS_WIDTH); + wd = min (wd, softc->user_bus_width[targ]); + + if(wd != softc->current_bus_width[targ]) { + /* the bus width has changed */ + softc->current_bus_width[targ] = + softc->goal_bus_width[targ] = neg.bus_width = wd; + + /* report the event */ + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + } + prepare_width_nego_rsponse(hcb, wd); + } + continue_current_hcb(softc); + return; + } + + + Then we handle any errors that could have happened during + auto-sense in the same simple-minded way as before. Otherwise we + look closer at the details again. + + + if(hcb->flags & DOING_AUTOSENSE) + goto autosense_failed; + + switch(hcb_status) { + + + The next event we consider is unexpected disconnect. Which + is considered normal after an ABORT or BUS DEVICE RESET message + and abnormal in other cases. + + + case UNEXPECTED_DISCONNECT: + if(requested_abort(hcb)) { + /* abort affects all commands on that target+LUN, so + * mark all disconnected HCBs on that target+LUN as aborted too + */ + for(h = softc->first_discon_hcb[hcb->target][hcb->lun]; + h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_REQ_ABORTED); + } + ccb_status = CAM_REQ_ABORTED; + } else if(requested_bus_device_reset(hcb)) { + int lun; + + /* reset affects all commands on that target, so + * mark all disconnected HCBs on that target+LUN as reset + */ + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[hcb->target][lun]; + h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* send event */ + xpt_async(AC_SENT_BDR, hcb->ccb->ccb_h.path_id, NULL); + + /* this was the CAM_RESET_DEV request itself, it's completed */ + ccb_status = CAM_REQ_CMP; + } else { + calculate_residue(hcb); + ccb_status = CAM_UNEXP_BUSFREE; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = hcb->lun; + } + break; + + + If the target refuses to accept tags we notify CAM about + that and return back all commands for this LUN: + + + case TAGS_REJECTED: + /* report the event */ + neg.flags = 0 & ~CCB_TRANS_TAG_ENB; + neg.valid = CCB_TRANS_TQ_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + + ccb_status = CAM_MSG_REJECT_REC; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = hcb->lun; + break; + + + Then we check a number of other conditions, with processing + basically limited to setting the CCB status: + + + case SELECTION_TIMEOUT: + ccb_status = CAM_SEL_TIMEOUT; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = CAM_LUN_WILDCARD; + break; + case PARITY_ERROR: + ccb_status = CAM_UNCOR_PARITY; + break; + case DATA_OVERRUN: + case ODD_WIDE_TRANSFER: + ccb_status = CAM_DATA_RUN_ERR; + break; + default: + /* all other errors are handled in a generic way */ + ccb_status = CAM_REQ_CMP_ERR; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = CAM_LUN_WILDCARD; + break; + } + + + Then we check if the error was serious enough to freeze the + input queue until it gets proceeded and do so if it is: + + + if(hcb->ccb->ccb_h.status & CAM_DEV_QFRZN) { + /* freeze the queue */ + xpt_freeze_devq(ccb->ccb_h.path, /*count*/1); + + /* re-queue all commands for this target/LUN back to CAM */ + + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + + if(targ == h->targ + && (lun_to_freeze == CAM_LUN_WILDCARD || lun_to_freeze == h->lun) ) + free_hcb_and_ccb_done(h, h->ccb, CAM_REQUEUE_REQ); + } + } + free_hcb_and_ccb_done(hcb, hcb->ccb, ccb_status); + schedule_next_hcb(softc); + return; + + + This concludes the generic interrupt handling although + specific controllers may require some additions. + + + + + Errors Summary + + When executing an I/O request many things may go wrong. The + reason of error can be reported in the CCB status with great + detail. Examples of use are spread throughout this document. For + completeness here is the summary of recommended responses for + the typical error conditions: + + + + CAM_RESRC_UNAVAIL - some + resource is temporarily unavailable and the SIM driver can not + generate an event when it will become available. An example of + this resource would be some intra-controller hardware resource + for which the controller does not generate an interrupt when + it becomes available. + + CAM_UNCOR_PARITY - + unrecovered parity error occurred + + CAM_DATA_RUN_ERR - data + overrun or unexpected data phase (going in other direction + than specified in CAM_DIR_MASK) or odd transfer length for + wide transfer + + CAM_SEL_TIMEOUT - selection + timeout occurred (target does not respond) + + CAM_CMD_TIMEOUT - command + timeout occurred (the timeout function ran) + + CAM_SCSI_STATUS_ERROR - the + device returned error + + CAM_AUTOSENSE_FAIL - the + device returned error and the REQUEST SENSE COMMAND + failed + + CAM_MSG_REJECT_REC - MESSAGE + REJECT message was received + + CAM_SCSI_BUS_RESET - received + SCSI bus reset + + CAM_REQ_CMP_ERR - + "impossible" SCSI phase occurred or something else as weird or + just a generic error if further detail is not + available + + CAM_UNEXP_BUSFREE - + unexpected disconnect occurred + + CAM_BDR_SENT - BUS DEVICE + RESET message was sent to the target + + CAM_UNREC_HBA_ERROR - + unrecoverable Host Bus Adapter Error + + CAM_REQ_TOO_BIG - the request + was too large for this controller + + CAM_REQUEUE_REQ - this + request should be re-queued to preserve transaction ordering. + This typically occurs when the SIM recognizes an error that + should freeze the queue and must place other queued requests + for the target at the sim level back into the XPT + queue. Typical cases of such errors are selection timeouts, + command timeouts and other like conditions. In such cases the + troublesome command returns the status indicating the error, + the and the other commands which have not be sent to the bus + yet get re-queued. + + CAM_LUN_INVALID - the LUN + ID in the request is not supported by the SCSI + controller + + CAM_TID_INVALID - the + target ID in the request is not supported by the SCSI + controller + + + + + Timeout Handling + + When the timeout for an HCB expires that request should be + aborted, just like with an XPT_ABORT request. The only + difference is that the returned status of aborted request should + be CAM_CMD_TIMEOUT instead of CAM_REQ_ABORTED (that's why + implementation of the abort better be done as a function). But + there is one more possible problem: what if the abort request + itself will get stuck? In this case the SCSI bus should be + reset, just like with an XPT_RESET_BUS request (and the idea + about implementing it as a function called from both places + applies here too). Also we should reset the whole SCSI bus if a + device reset request got stuck. So after all the timeout + function would look like: + + +static void +xxx_timeout(void *arg) +{ + struct xxx_hcb *hcb = (struct xxx_hcb *)arg; + struct xxx_softc *softc; + struct ccb_hdr *ccb_h; + + softc = hcb->softc; + ccb_h = &hcb->ccb->ccb_h; + + if(hcb->flags & HCB_BEING_ABORTED + || ccb_h->func_code == XPT_RESET_DEV) { + xxx_reset_bus(softc); + } else { + xxx_abort_ccb(hcb->ccb, CAM_CMD_TIMEOUT); + } +} + + + When we abort a request all the other disconnected requests + to the same target/LUN get aborted too. So there appears a + question, should we return them with status CAM_REQ_ABORTED or + CAM_CMD_TIMEOUT ? The current drivers use CAM_CMD_TIMEOUT. This + seems logical because if one request got timed out then probably + something really bad is happening to the device, so if they + would not be disturbed they would time out by themselves. + + + + diff --git a/en_US.ISO_8859-1/books/developers-handbook/book.sgml b/en_US.ISO_8859-1/books/developers-handbook/book.sgml index 1006c9f5d2..2cf7d725aa 100644 --- a/en_US.ISO_8859-1/books/developers-handbook/book.sgml +++ b/en_US.ISO_8859-1/books/developers-handbook/book.sgml @@ -1,520 +1,523 @@ %bookinfo; %man; %chapters; + %authors; ]> FreeBSD Developers' Handbook The FreeBSD Documentation Project
doc@FreeBSD.org
August 2000 2000 + 2001 The FreeBSD Documentation Project &bookinfo.legalnotice; Welcome to the Developers' Handbook. Introduction Developing on FreeBSD This will need to discuss FreeBSD as a development platform, the vision of BSD, architectural overview, layout of /usr/src, history, etc. Thank you for considering FreeBSD as your development platform! We hope it will not let you down. The BSD Vision Architectural Overview The Layout of /usr/src The complete source code to FreeBSD is available from our public CVS repository. The source code is normally installed in /usr/src which contains the following subdirectories. Directory Description bin/ Source for files in /bin contrib/ Source for files from contribued software. crypto/ DES source etc/ Source for files in /etc games/ Source for files in /usr/games gnu/ Utilities covered by the GNU Public License include/ Source for files in /usr/include kerberosIV/ Source for Kerbereros version IV kerberos5/ Source for Kerbereros version 5 lib/ Source for files in /usr/lib libexec/ Source for files in /usr/libexec release/ Files required to produce a FreeBSD release sbin/ Source for files in /sbin secure/ FreeSec sources share/ Source for files in /sbin sys/ Kernel source files tools/ Tools used for maintenance and testing of FreeBSD usr.bin/ Source for files in /usr/bin usr.sbin/ Source for files in /usr/sbin Basics &chap.tools; &chap.secure; Kernel History of the Unix Kernel Some history of the Unix/BSD kernel, system calls, how do processes work, blocking, scheduling, threads (kernel), context switching, signals, interrupts, modules, etc. &chap.locking; Memory and Virtual Memory Virtual Memory VM, paging, swapping, allocating memory, testing for memory leaks, mmap, vnodes, etc. I/O System UFS UFS, FFS, Ext2FS, JFS, inodes, buffer cache, labeling, locking, metadata, soft-updates, LFS, portalfs, procfs, vnodes, memory sharing, memory objects, TLBs, caching Interprocess Communication Signals Signals, pipes, semaphores, message queues, shared memory, ports, sockets, doors Networking Sockets Sockets, bpf, IP, TCP, UDP, ICMP, OSI, bridging, firewalling, NAT, switching, etc Network Filesystems AFS AFS, NFS, SANs etc] Terminal Handling Syscons Syscons, tty, PCVT, serial console, screen savers, etc Sound OSS OSS, waveforms, etc Device Drivers &chap.driverbasics; &chap.pci; + &chap.scsi; USB Devices This chapter will talk about the FreeBSD mechanisms for writing a device driver for a device on a USB bus. NewBus This chapter will talk about the FreeBSD NewBus architecture. Architectures IA-32 Talk about the architectural specifics of FreeBSD/x86. Alpha Talk about the architectural specifics of FreeBSD/alpha. Explanation of allignment errors, how to fix, how to ignore. Example assembly language code for FreeBSD/alpha. IA-64 Talk about the architectural specifics of FreeBSD/ia64. Debugging Truss various descriptions on how to debug certain aspects of the system using truss, ktrace, gdb, kgdb, etc Compatibility Layers Linux Linux, SVR4, etc Appendices Dave A Patterson John L Hennessy 1998Morgan Kaufmann Publishers, Inc. 1-55860-428-6 Morgan Kaufmann Publishers, Inc. Computer Organization and Design The Hardware / Software Interface 1-2 W. Richard Stevens 1993Addison Wesley Longman, Inc. 0-201-56317-7 Addison Wesley Longman, Inc. Advanced Programming in the Unix Environment 1-2 Marshall Kirk McKusick Keith Bostic Michael J Karels John S Quarterman 1996Addison-Wesley Publishing Company, Inc. 0-201-54979-4 Addison-Wesley Publishing Company, Inc. The Design and Implementation of the 4.4 BSD Operating System 1-2 Aleph One Phrack 49; "Smashing the Stack for Fun and Profit" Chrispin Cowan Calton Pu Dave Maier StackGuard; Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks Todd Miller Theo de Raadt strlcpy and strlcat -- consistent, safe string copy and concatenation. diff --git a/en_US.ISO_8859-1/books/developers-handbook/chapters.ent b/en_US.ISO_8859-1/books/developers-handbook/chapters.ent index c2149880e2..a367f40239 100644 --- a/en_US.ISO_8859-1/books/developers-handbook/chapters.ent +++ b/en_US.ISO_8859-1/books/developers-handbook/chapters.ent @@ -1,58 +1,59 @@ + diff --git a/en_US.ISO_8859-1/books/developers-handbook/scsi/chapter.sgml b/en_US.ISO_8859-1/books/developers-handbook/scsi/chapter.sgml new file mode 100644 index 0000000000..837a8fd60d --- /dev/null +++ b/en_US.ISO_8859-1/books/developers-handbook/scsi/chapter.sgml @@ -0,0 +1,2079 @@ + + + + Common Access Method SCSI Controllers + + This chapter was written by &a.babkin; + Modifications for the handbook made by + &a.murray;. + + + Synopsis + + This document assumes that the reader has a general + understanding of device drivers in FreeBSD and of the SCSI + protocol. Much of the information in this document was + extracted from the drivers : + + + + ncr (/sys/pci/ncr.c) by + Wolfgang Stanglmeier and Stefan Esser + + sym (/sys/pci/sym.c) by + Gerard Roudier + + aic7xxx + (/sys/dev/aic7xxx/aic7xxx.c) by Justin + T. Gibbs + + + + and from the CAM code itself (by Justing T. Gibbs, see + /sys/cam/*). When some solution looked the + most logical and was essentially verbatim extracted from the code + by Justin Gibbs, I marked it as "recommended". + + The document is illustrated with examples in + pseudo-code. Although sometimes the examples have many details + and look like real code, it's still pseudo-code. It was written + to demonstrate the concepts in an understandable way. For a real + driver other approaches may be more modular and efficient. It + also abstracts from the hardware details, as well as issues that + would cloud the demonstrated concepts or that are supposed to be + described in the other chapters of the developers handbook. Such + details are commonly shown as calls to functions with descriptive + names, comments or pseudo-statements. Fortunately real life + full-size examples with all the details can be found in the real + drivers. + + + + + General architecture + + CAM stands for Common Access Method. It's a generic way to + address the I/O buses in a SCSI-like way. This allows a + separation of the generic device drivers from the drivers + controlling the I/O bus: for example the disk driver becomes able + to control disks on both SCSI, IDE, and/or any other bus so the + disk driver portion does not have to be rewritten (or copied and + modified) for every new I/O bus. Thus the two most important + active entities are: + + + Peripheral Modules - a + driver for peripheral devices (disk, tape, CDROM, + etc.) + SCSI Interface Modules (SIM) + - a Host Bus Adapter drivers for connecting to an I/O bus such + as SCSI or IDE. + + + A peripheral driver receives requests from the OS, converts + them to a sequence of SCSI commands and passes these SCSI + commands to a SCSI Interface Module. The SCSI Interface Module + is responsible for passing these commands to the actual hardware + (or if the actual hardware is not SCSI but, for example, IDE + then also converting the SCSI commands to the native commands of + the hardware). + + Because we are interested in writing a SCSI adapter driver + here, from this point on we will consider everything from the + SIM standpoint. + + A typical SIM driver needs to include the following + CAM-related header files: + + +#include <cam/cam.h> +#include <cam/cam_ccb.h> +#include <cam/cam_sim.h> +#include <cam/cam_xpt_sim.h> +#include <cam/cam_debug.h> +#include <cam/scsi/scsi_all.h> + + + The first thing each SIM driver must do is register itself + with the CAM subsystem. This is done during the driver's + xxx_attach() function (here and further + xxx_ is used to denote the unique driver name prefix). The + xxx_attach() function itself is called by + the system bus auto-configuration code which we don't describe + here. + + This is achieved in multiple steps: first it's necessary to + allocate the queue of requests associated with this SIM: + + + struct cam_devq *devq; + + if(( devq = cam_simq_alloc(SIZE) )==NULL) { + error; /* some code to handle the error */ + } + + + Here SIZE is the size of the queue to be allocated, maximal + number of requests it could contain. It's the number of requests + that the SIM driver can handle in parallel on one SCSI + card. Commonly it can be calculated as: + + +SIZE = NUMBER_OF_SUPPORTED_TARGETS * MAX_SIMULTANEOUS_COMMANDS_PER_TARGET + + + Next we create a descriptor of our SIM: + + + struct cam_sim *sim; + + if(( sim = cam_sim_alloc(action_func, poll_func, driver_name, + softc, unit, max_dev_transactions, + max_tagged_dev_transactions, devq) )==NULL) { + cam_simq_free(devq); + error; /* some code to handle the error */ + } + + + Note that if we are not able to create a SIM descriptor we + free the devq also because we can do + nothing else with it and we want to conserve memory. + + If a SCSI card has multiple SCSI buses on it then each bus + requires its own cam_sim + structure. + + An interesting question is what to do if a SCSI card has + more than one SCSI bus, do we need one + devq structure per card or per SCSI + bus? The answer given in the comments to the CAM code is: + either way, as the driver's author prefers. + + The arguments are : + + + action_func - pointer to + the driver's xxx_action function. + + static void + xxx_action + + + struct cam_sim *sim, + union ccb *ccb + + + + + poll_func - pointer to + the driver's xxx_poll() + + static void + xxx_poll + + + struct cam_sim *sim + + + + + driver_name - the name of the actual driver, + such as "ncr" or "wds" + + softc - pointer to the + driver's internal descriptor for this SCSI card. This + pointer will be used by the driver in future to get private + data. + + unit - the controller unit number, for example + for controller "wds0" this number will be + 0 + + max_dev_transactions - maximal number of + simultaneous transactions per SCSI target in the non-tagged + mode. This value will be almost universally equal to 1, with + possible exceptions only for the non-SCSI cards. Also the + drivers that hope to take advantage by preparing one + transaction while another one is executed may set it to 2 + but this does not seem to be worth the + complexity. + + max_tagged_dev_transactions - the same thing, + but in the tagged mode. Tags are the SCSI way to initiate + multiple transactions on a device: each transaction is + assigned a unique tag and the transaction is sent to the + device. When the device completes some transaction it sends + back the result together with the tag so that the SCSI + adapter (and the driver) can tell which transaction was + completed. This argument is also known as the maximal tag + depth. It depends on the abilities of the SCSI + adapter. + + + + Finally we register the SCSI buses associated with our SCSI + adapter: + + + if(xpt_bus_register(sim, bus_number) != CAM_SUCCESS) { + cam_sim_free(sim, /*free_devq*/ TRUE); + error; /* some code to handle the error */ + } + + + If there is one devq structure per + SCSI bus (i.e. we consider a card with multiple buses as + multiple cards with one bus each) then the bus number will + always be 0, otherwise each bus on the SCSI card should be get a + distinct number. Each bus needs its own separate structure + cam_sim. + + After that our controller is completely hooked to the CAM + system. The value of devq can be + discarded now: sim will be passed as an argument in all further + calls from CAM and devq can be derived from it. + + CAM provides the framework for such asynchronous + events. Some events originate from the lower levels (the SIM + drivers), some events originate from the peripheral drivers, + some events originate from the CAM subsystem itself. Any driver + can register callbacks for some types of the asynchronous + events, so that it would be notified if these events + occur. + + A typical example of such an event is a device reset. Each + transaction and event identifies the devices to which it applies + by the means of "path". The target-specific events normally + occur during a transaction with this device. So the path from + that transaction may be re-used to report this event (this is + safe because the event path is copied in the event reporting + routine but not deallocated nor passed anywhere further). Also + it's safe to allocate paths dynamically at any time including + the interrupt routines, although that incurs certain overhead, + and a possible problem with this approach is that there may be + no free memory at that time. For a bus reset event we need to + define a wildcard path including all devices on the bus. So we + can create the path for the future bus reset events in advance + and avoid problems with the future memory shortage: + + + struct cam_path *path; + + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), CAM_TARGET_WILDCARD, + CAM_LUN_WILDCARD) != CAM_REQ_CMP) { + xpt_bus_deregister(cam_sim_path(sim)); + cam_sim_free(sim, /*free_devq*/TRUE); + error; /* some code to handle the error */ + } + + softc->wpath = path; + softc->sim = sim; + + + As you can see the path includes: + + + ID of the peripheral driver (NULL here because we have + none) + + ID of the SIM driver + (cam_sim_path(sim)) + + SCSI target number of the device (CAM_TARGET_WILDCARD + means "all devices") + + SCSI LUN number of the subdevice (CAM_LUN_WILDCARD means + "all LUNs") + + + If the driver can't allocate this path it won't be able to + work normally, so in that case we dismantle that SCSI + bus. + + And we save the path pointer in the + softc structure for future use. After + that we save the value of sim (or we can also discard it on the + exit from xxx_probe() if we wish). + + That's all for a minimalistic initialization. To do things + right there is one more issue left. + + For a SIM driver there is one particularly interesting + event: when a target device is considered lost. In this case + resetting the SCSI negotiations with this device may be a good + idea. So we register a callback for this event with CAM. The + request is passed to CAM by requesting CAM action on a CAM + control block for this type of request: + + + struct ccb_setasync csa; + + xpt_setup_ccb(&csa.ccb_h, path, /*priority*/5); + csa.ccb_h.func_code = XPT_SASYNC_CB; + csa.event_enable = AC_LOST_DEVICE; + csa.callback = xxx_async; + csa.callback_arg = sim; + xpt_action((union ccb *)&csa); + + + Now we take a look at the xxx_action() + and xxx_poll() driver entry points. + + + + static void + xxx_action + + + struct cam_sim *sim, + union ccb *ccb + + + + + Do some action on request of the CAM subsystem. Sim + describes the SIM for the request, CCB is the request + itself. CCB stands for "CAM Control Block". It is a union of + many specific instances, each describing arguments for some type + of transactions. All of these instances share the CCB header + where the common part of arguments is stored. + + CAM supports the SCSI controllers working in both initiator + ("normal") mode and target (simulating a SCSI device) mode. Here + we only consider the part relevant to the initiator mode. + + There are a few function and macros (in other words, + methods) defined to access the public data in the struct sim: + + + cam_sim_path(sim) - the + path ID (see above) + + cam_sim_name(sim) - the + name of the sim + + cam_sim_softc(sim) - the + pointer to the softc (driver private data) + structure + + cam_sim_unit(sim) - the + unit number + + cam_sim_bus(sim) - the bus + ID + + + To identify the device, xxx_action() can + get the unit number and pointer to its structure softc using + these functions. + + The type of request is stored in + ccb->ccb_h.func_code. So generally + xxx_action() consists of a big + switch: + + + struct xxx_softc *softc = (struct xxx_softc *) cam_sim_softc(sim); + struct ccb_hdr *ccb_h = &ccb->ccb_h; + int unit = cam_sim_unit(sim); + int bus = cam_sim_bus(sim); + + switch(ccb_h->func_code) { + case ...: + ... + default: + ccb_h->status = CAM_REQ_INVALID; + xpt_done(ccb); + break; + } + + + As can be seen from the default case (if an unknown command + was received) the return code of the command is set into + ccb->ccb_h.status and the completed + CCB is returned back to CAM by calling + xpt_done(ccb). + + xpt_done() does not have to be called + from xxx_action(): For example an I/O + request may be enqueued inside the SIM driver and/or its SCSI + controller. Then when the device would post an interrupt + signaling that the processing of this request is complete + xpt_done() may be called from the interrupt + handling routine. + + Actually, the CCB status is not only assigned as a return + code but a CCB has some status all the time. Before CCB is + passed to the xxx_action() routine it gets + the status CCB_REQ_INPROG meaning that it's in progress. There + are a surprising number of status values defined in + /sys/cam/cam.h which should be able to + represent the status of a request in great detail. More + interesting yet, the status is in fact a "bitwise or" of an + enumerated status value (the lower 6 bits) and possible + additional flag-like bits (the upper bits). The enumerated + values will be discussed later in more detail. The summary of + them can be found in the Errors Summary section. The possible + status flags are: + + + + CAM_DEV_QFRZN - if the + SIM driver gets a serious error (for example, the device does + not respond to the selection or breaks the SCSI protocol) when + processing a CCB it should freeze the request queue by calling + xpt_freeze_simq(), return the other + enqueued but not processed yet CCBs for this device back to + the CAM queue, then set this flag for the troublesome CCB and + call xpt_done(). This flag causes the CAM + subsystem to unfreeze the queue after it handles the + error. + + CAM_AUTOSNS_VALID - if + the device returned an error condition and the flag + CAM_DIS_AUTOSENSE is not set in CCB the SIM driver must + execute the REQUEST SENSE command automatically to extract the + sense (extended error information) data from the device. If + this attempt was successful the sense data should be saved in + the CCB and this flag set. + + CAM_RELEASE_SIMQ - like + CAM_DEV_QFRZN but used in case there is some problem (or + resource shortage) with the SCSI controller itself. Then all + the future requests to the controller should be stopped by + xpt_freeze_simq(). The controller queue + will be restarted after the SIM driver overcomes the shortage + and informs CAM by returning some CCB with this flag + set. + + CAM_SIM_QUEUED - when SIM + puts a CCB into its request queue this flag should be set (and + removed when this CCB gets dequeued before being returned back + to CAM). This flag is not used anywhere in the CAM code now, + so its purpose is purely diagnostic. + + + + The function xxx_action() is not + allowed to sleep, so all the synchronization for resource access + must be done using SIM or device queue freezing. Besides the + aforementioned flags the CAM subsystem provides functions + xpt_selease_simq() and + xpt_release_devq() to unfreeze the queues + directly, without passing a CCB to CAM. + + The CCB header contains the following fields: + + + + path - path ID for the + request + + target_id - target device + ID for the request + + target_lun - LUN ID of + the target device + + timeout - timeout + interval for this command, in milliseconds + + timeout_ch - a + convenience place for the SIM driver to store the timeout handle + (the CAM subsystem itself does not make any assumptions about + it) + + flags - various bits of + information about the request spriv_ptr0, spriv_ptr1 - fields + reserved for private use by the SIM driver (such as linking to + the SIM queues or SIM private control blocks); actually, they + exist as unions: spriv_ptr0 and spriv_ptr1 have the type (void + *), spriv_field0 and spriv_field1 have the type unsigned long, + sim_priv.entries[0].bytes and sim_priv.entries[1].bytes are byte + arrays of the size consistent with the other incarnations of the + union and sim_priv.bytes is one array, twice + bigger. + + + + The recommended way of using the SIM private fields of CCB + is to define some meaningful names for them and use these + meaningful names in the driver, like: + + +#define ccb_some_meaningful_name sim_priv.entries[0].bytes +#define ccb_hcb spriv_ptr1 /* for hardware control block */ + + + The most common initiator mode requests are: + + XPT_SCSI_IO - execute an + I/O transaction + + The instance "struct ccb_scsiio csio" of the union ccb is + used to transfer the arguments. They are: + + + cdb_io - pointer to + the SCSI command buffer or the buffer + itself + + cdb_len - SCSI + command length + + data_ptr - pointer to + the data buffer (gets a bit complicated if scatter/gather is + used) + + dxfer_len - length of + the data to transfer + + sglist_cnt - counter + of the scatter/gather segments + + scsi_status - place + to return the SCSI status + + sense_data - buffer + for the SCSI sense information if the command returns an + error (the SIM driver is supposed to run the REQUEST SENSE + command automatically in this case if the CCB flag + CAM_DIS_AUTOSENSE is not set) + + sense_len - the + length of that buffer (if it happens to be higher than size + of sense_data the SIM driver must silently assume the + smaller value) resid, sense_resid - if the transfer of data + or SCSI sense returned an error these are the returned + counters of the residual (not transferred) data. They do not + seem to be especially meaningful, so in a case when they are + difficult to compute (say, counting bytes in the SCSI + controller's FIFO buffer) an approximate value will do as + well. For a successfully completed transfer they must be set + to zero. + + tag_action - the kind + of tag to use: + + + CAM_TAG_ACTION_NONE - don't use tags for this + transaction + MSG_SIMPLE_Q_TAG, MSG_HEAD_OF_Q_TAG, + MSG_ORDERED_Q_TAG - value equal to the appropriate tag + message (see /sys/cam/scsi/scsi_message.h); this gives only + the tag type, the SIM driver must assign the tag value + itself + + + + + + + The general logic of handling this request is the + following: + + The first thing to do is to check for possible races, to + make sure that the command did not get aborted when it was + sitting in the queue: + + + struct ccb_scsiio *csio = &ccb->csio; + + if ((ccb_h->status & CAM_STATUS_MASK) != CAM_REQ_INPROG) { + xpt_done(ccb); + return; + } + + + Also we check that the device is supported at all by our + controller: + + + if(ccb_h->target_id > OUR_MAX_SUPPORTED_TARGET_ID + || cch_h->target_id == OUR_SCSI_CONTROLLERS_OWN_ID) { + ccb_h->status = CAM_TID_INVALID; + xpt_done(ccb); + return; + } + if(ccb_h->target_lun > OUR_MAX_SUPPORTED_LUN) { + ccb_h->status = CAM_LUN_INVALID; + xpt_done(ccb); + return; + } + + + Then allocate whatever data structures (such as + card-dependent hardware control block) we need to process this + request. If we can't then freeze the SIM queue and remember + that we have a pending operation, return the CCB back and ask + CAM to re-queue it. Later when the resources become available + the SIM queue must be unfrozen by returning a ccb with the + CAM_SIMQ_RELEASE bit set in its status. Otherwise, if all went + well, link the CCB with the hardware control block (HCB) and + mark it as queued. + + + struct xxx_hcb *hcb = allocate_hcb(softc, unit, bus); + + if(hcb == NULL) { + softc->flags |= RESOURCE_SHORTAGE; + xpt_freeze_simq(sim, /*count*/1); + ccb_h->status = CAM_REQUEUE_REQ; + xpt_done(ccb); + return; + } + + hcb->ccb = ccb; ccb_h->ccb_hcb = (void *)hcb; + ccb_h->status |= CAM_SIM_QUEUED; + + + Extract the target data from CCB into the hardware control + block. Check if we are asked to assign a tag and if yes then + generate an unique tag and build the SCSI tag messages. The + SIM driver is also responsible for negotiations with the + devices to set the maximal mutually supported bus width, + synchronous rate and offset. + + + hcb->target = ccb_h->target_id; hcb->lun = ccb_h->target_lun; + generate_identify_message(hcb); + if( ccb_h->tag_action != CAM_TAG_ACTION_NONE ) + generate_unique_tag_message(hcb, ccb_h->tag_action); + if( !target_negotiated(hcb) ) + generate_negotiation_messages(hcb); + + + Then set up the SCSI command. The command storage may be + specified in the CCB in many interesting ways, specified by + the CCB flags. The command buffer can be contained in CCB or + pointed to, in the latter case the pointer may be physical or + virtual. Since the hardware commonly needs physical address we + always convert the address to the physical one. + + A NOT-QUITE RELATED NOTE: Normally this is done by a call + to vtophys(), but for the PCI device (which account for most + of the SCSI controllers now) drivers' portability to the Alpha + architecture the conversion must be done by vtobus() instead + due to special Alpha quirks. [IMHO it would be much better to + have two separate functions, vtop() and ptobus() then vtobus() + would be a simple superposition of them.] In case if a + physical address is requested it's OK to return the CCB with + the status CAM_REQ_INVALID, the current drivers do that. But + it's also possible to compile the Alpha-specific piece of + code, as in this example (there should be a more direct way to + do that, without conditional compilation in the drivers). If + necessary a physical address can be also converted or mapped + back to a virtual address but with big pain, so we don't do + that. + + + if(ccb_h->flags & CAM_CDB_POINTER) { + /* CDB is a pointer */ + if(!(ccb_h->flags & CAM_CDB_PHYS)) { + /* CDB pointer is virtual */ + hcb->cmd = vtobus(csio->cdb_io.cdb_ptr); + } else { + /* CDB pointer is physical */ +#if defined(__alpha__) + hcb->cmd = csio->cdb_io.cdb_ptr | alpha_XXX_dmamap_or ; +#else + hcb->cmd = csio->cdb_io.cdb_ptr ; +#endif + } + } else { + /* CDB is in the ccb (buffer) */ + hcb->cmd = vtobus(csio->cdb_io.cdb_bytes); + } + hcb->cmdlen = csio->cdb_len; + + + Now it's time to set up the data. Again, the data storage + may be specified in the CCB in many interesting ways, + specified by the CCB flags. First we get the direction of the + data transfer. The simplest case is if there is no data to + transfer: + + + int dir = (ccb_h->flags & CAM_DIR_MASK); + + if (dir == CAM_DIR_NONE) + goto end_data; + + + Then we check if the data is in one chunk or in a + scatter-gather list, and the addresses are physical or + virtual. The SCSI controller may be able to handle only a + limited number of chunks of limited length. If the request + hits this limitation we return an error. We use a special + function to return the CCB to handle in one place the HCB + resource shortages. The functions to add chunks are + driver-dependent, and here we leave them without detailed + implementation. See description of the SCSI command (CDB) + handling for the details on the address-translation issues. + If some variation is too difficult or impossible to implement + with a particular card it's OK to return the status + CAM_REQ_INVALID. Actually, it seems like the scatter-gather + ability is not used anywhere in the CAM code now. But at least + the case for a single non-scattered virtual buffer must be + implemented, it's actively used by CAM. + + + int rv; + + initialize_hcb_for_data(hcb); + + if((!(ccb_h->flags & CAM_SCATTER_VALID)) { + /* single buffer */ + if(!(ccb_h->flags & CAM_DATA_PHYS)) { + rv = add_virtual_chunk(hcb, csio->data_ptr, csio->dxfer_len, dir); + } + } else { + rv = add_physical_chunk(hcb, csio->data_ptr, csio->dxfer_len, dir); + } + } else { + int i; + struct bus_dma_segment *segs; + segs = (struct bus_dma_segment *)csio->data_ptr; + + if ((ccb_h->flags & CAM_SG_LIST_PHYS) != 0) { + /* The SG list pointer is physical */ + rv = setup_hcb_for_physical_sg_list(hcb, segs, csio->sglist_cnt); + } else if (!(ccb_h->flags & CAM_DATA_PHYS)) { + /* SG buffer pointers are virtual */ + for (i = 0; i < csio->sglist_cnt; i++) { + rv = add_virtual_chunk(hcb, segs[i].ds_addr, + segs[i].ds_len, dir); + if (rv != CAM_REQ_CMP) + break; + } + } else { + /* SG buffer pointers are physical */ + for (i = 0; i < csio->sglist_cnt; i++) { + rv = add_physical_chunk(hcb, segs[i].ds_addr, + segs[i].ds_len, dir); + if (rv != CAM_REQ_CMP) + break; + } + } + } + if(rv != CAM_REQ_CMP) { + /* we expect that add_*_chunk() functions return CAM_REQ_CMP + * if they added a chunk successfully, CAM_REQ_TOO_BIG if + * the request is too big (too many bytes or too many chunks), + * CAM_REQ_INVALID in case of other troubles + */ + free_hcb_and_ccb_done(hcb, ccb, rv); + return; + } + end_data: + + + If disconnection is disabled for this CCB we pass this + information to the hcb: + + + if(ccb_h->flags & CAM_DIS_DISCONNECT) + hcb_disable_disconnect(hcb); + + + If the controller is able to run REQUEST SENSE command all + by itself then the value of the flag CAM_DIS_AUTOSENSE should + also be passed to it, to prevent automatic REQUEST SENSE if the + CAM subsystem does not want it. + + The only thing left is to set up the timeout, pass our hcb + to the hardware and return, the rest will be done by the + interrupt handler (or timeout handler). + + + ccb_h->timeout_ch = timeout(xxx_timeout, (caddr_t) hcb, + (ccb_h->timeout * hz) / 1000); /* convert milliseconds to ticks */ + put_hcb_into_hardware_queue(hcb); + return; + + + And here is a possible implementation of the function + returning CCB: + + + static void + free_hcb_and_ccb_done(struct xxx_hcb *hcb, union ccb *ccb, u_int32_t status) + { + struct xxx_softc *softc = hcb->softc; + + ccb->ccb_h.ccb_hcb = 0; + if(hcb != NULL) { + untimeout(xxx_timeout, (caddr_t) hcb, ccb->ccb_h.timeout_ch); + /* we're about to free a hcb, so the shortage has ended */ + if(softc->flags & RESOURCE_SHORTAGE) { + softc->flags &= ~RESOURCE_SHORTAGE; + status |= CAM_RELEASE_SIMQ; + } + free_hcb(hcb); /* also removes hcb from any internal lists */ + } + ccb->ccb_h.status = status | + (ccb->ccb_h.status & ~(CAM_STATUS_MASK|CAM_SIM_QUEUED)); + xpt_done(ccb); + } + + + + XPT_RESET_DEV - send the SCSI "BUS + DEVICE RESET" message to a device + + There is no data transferred in CCB except the header and + the most interesting argument of it is target_id. Depending on + the controller hardware a hardware control block just like for + the XPT_SCSI_IO request may be constructed (see XPT_SCSI_IO + request description) and sent to the controller or the SCSI + controller may be immediately programmed to send this RESET + message to the device or this request may be just not supported + (and return the status CAM_REQ_INVALID). Also on completion of + the request all the disconnected transactions for this target + must be aborted (probably in the interrupt routine). + + Also all the current negotiations for the target are lost on + reset, so they might be cleaned too. Or they clearing may be + deferred, because anyway the target would request re-negotiation + on the next transaction. + + XPT_RESET_BUS - send the RESET signal + to the SCSI bus + + No arguments are passed in the CCB, the only interesting + argument is the SCSI bus indicated by the struct sim + pointer. + + A minimalistic implementation would forget the SCSI + negotiations for all the devices on the bus and return the + status CAM_REQ_CMP. + + The proper implementation would in addition actually reset + the SCSI bus (possible also reset the SCSI controller) and mark + all the CCBs being processed, both those in the hardware queue + and those being disconnected, as done with the status + CAM_SCSI_BUS_RESET. Like: + + + int targ, lun; + struct xxx_hcb *h, *hh; + struct ccb_trans_settings neg; + struct cam_path *path; + + /* The SCSI bus reset may take a long time, in this case its completion + * should be checked by interrupt or timeout. But for simplicity + * we assume here that it's really fast. + */ + reset_scsi_bus(softc); + + /* drop all enqueued CCBs */ + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* the clean values of negotiations to report */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + + /* drop all disconnected CCBs and clean negotiations */ + for(targ=0; targ <= OUR_MAX_SUPPORTED_TARGET; targ++) { + clean_negotiations(softc, targ); + + /* report the event if possible */ + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), targ, + CAM_LUN_WILDCARD) == CAM_REQ_CMP) { + xpt_async(AC_TRANSFER_NEG, path, &neg); + xpt_free_path(path); + } + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[targ][lun]; h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + } + + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + + /* report the event */ + xpt_async(AC_BUS_RESET, softc->wpath, NULL); + return; + + + Implementing the SCSI bus reset as a function may be a good + idea because it would be re-used by the timeout function as a + last resort if the things go wrong. + + XPT_ABORT - abort the specified + CCB + + The arguments are transferred in the instance "struct + ccb_abort cab" of the union ccb. The only argument field in it + is: + + abort_ccb - pointer to the CCB to be + aborted + + If the abort is not supported just return the status + CAM_UA_ABORT. This is also the easy way to minimally implement + this call, return CAM_UA_ABORT in any case. + + The hard way is to implement this request honestly. First + check that abort applies to a SCSI transaction: + + + struct ccb *abort_ccb; + abort_ccb = ccb->cab.abort_ccb; + + if(abort_ccb->ccb_h.func_code != XPT_SCSI_IO) { + ccb->ccb_h.status = CAM_UA_ABORT; + xpt_done(ccb); + return; + } + + + Then it's necessary to find this CCB in our queue. This can + be done by walking the list of all our hardware control blocks + in search for one associated with this CCB: + + + struct xxx_hcb *hcb, *h; + + hcb = NULL; + + /* We assume that softc->first_hcb is the head of the list of all + * HCBs associated with this bus, including those enqueued for + * processing, being processed by hardware and disconnected ones. + */ + for(h = softc->first_hcb; h != NULL; h = h->next) { + if(h->ccb == abort_ccb) { + hcb = h; + break; + } + } + + if(hcb == NULL) { + /* no such CCB in our queue */ + ccb->ccb_h.status = CAM_PATH_INVALID; + xpt_done(ccb); + return; + } + + hcb=found_hcb; + + + Now we look at the current processing status of the HCB. It + may be either sitting in the queue waiting to be sent to the + SCSI bus, being transferred right now, or disconnected and + waiting for the result of the command, or actually completed by + hardware but not yet marked as done by software. To make sure + that we don't get in any races with hardware we mark the HCB as + being aborted, so that if this HCB is about to be sent to the + SCSI bus the SCSI controller will see this flag and skip + it. + + + int hstatus; + + /* shown as a function, in case special action is needed to make + * this flag visible to hardware + */ + set_hcb_flags(hcb, HCB_BEING_ABORTED); + + abort_again: + + hstatus = get_hcb_status(hcb); + switch(hstatus) { + case HCB_SITTING_IN_QUEUE: + remove_hcb_from_hardware_queue(hcb); + /* FALLTHROUGH */ + case HCB_COMPLETED: + /* this is an easy case */ + free_hcb_and_ccb_done(hcb, abort_ccb, CAM_REQ_ABORTED); + break; + + + If the CCB is being transferred right now we would like to + signal to the SCSI controller in some hardware-dependent way + that we want to abort the current transfer. The SCSI controller + would set the SCSI ATTENTION signal and when the target responds + to it send an ABORT message. We also reset the timeout to make + sure that the target is not sleeping forever. If the command + would not get aborted in some reasonable time like 10 seconds + the timeout routine would go ahead and reset the whole SCSI bus. + Because the command will be aborted in some reasonable time we + can just return the abort request now as successfully completed, + and mark the aborted CCB as aborted (but not mark it as done + yet). + + + case HCB_BEING_TRANSFERRED: + untimeout(xxx_timeout, (caddr_t) hcb, abort_ccb->ccb_h.timeout_ch); + abort_ccb->ccb_h.timeout_ch = + timeout(xxx_timeout, (caddr_t) hcb, 10 * hz); + abort_ccb->ccb_h.status = CAM_REQ_ABORTED; + /* ask the controller to abort that HCB, then generate + * an interrupt and stop + */ + if(signal_hardware_to_abort_hcb_and_stop(hcb) < 0) { + /* oops, we missed the race with hardware, this transaction + * got off the bus before we aborted it, try again */ + goto abort_again; + } + + break; + + + If the CCB is in the list of disconnected then set it up as + an abort request and re-queue it at the front of hardware + queue. Reset the timeout and report the abort request to be + completed. + + + case HCB_DISCONNECTED: + untimeout(xxx_timeout, (caddr_t) hcb, abort_ccb->ccb_h.timeout_ch); + abort_ccb->ccb_h.timeout_ch = + timeout(xxx_timeout, (caddr_t) hcb, 10 * hz); + put_abort_message_into_hcb(hcb); + put_hcb_at_the_front_of_hardware_queue(hcb); + break; + } + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + That's all for the ABORT request, although there is one more + issue. Because the ABORT message cleans all the ongoing + transactions on a LUN we have to mark all the other active + transactions on this LUN as aborted. That should be done in the + interrupt routine, after the transaction gets aborted. + + Implementing the CCB abort as a function may be quite a good + idea, this function can be re-used if an I/O transaction times + out. The only difference would be that the timed out transaction + would return the status CAM_CMD_TIMEOUT for the timed out + request. Then the case XPT_ABORT would be small, like + that: + + + case XPT_ABORT: + struct ccb *abort_ccb; + abort_ccb = ccb->cab.abort_ccb; + + if(abort_ccb->ccb_h.func_code != XPT_SCSI_IO) { + ccb->ccb_h.status = CAM_UA_ABORT; + xpt_done(ccb); + return; + } + if(xxx_abort_ccb(abort_ccb, CAM_REQ_ABORTED) < 0) + /* no such CCB in our queue */ + ccb->ccb_h.status = CAM_PATH_INVALID; + else + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + + XPT_SET_TRAN_SETTINGS - explicitly + set values of SCSI transfer settings + + The arguments are transferred in the instance "struct ccb_trans_setting cts" +of the union ccb: + + + valid - a bitmask showing + which settings should be updated: + + CCB_TRANS_SYNC_RATE_VALID + - synchronous transfer rate + + CCB_TRANS_SYNC_OFFSET_VALID + - synchronous offset + + CCB_TRANS_BUS_WIDTH_VALID + - bus width + + CCB_TRANS_DISC_VALID - + set enable/disable disconnection + + CCB_TRANS_TQ_VALID - set + enable/disable tagged queuing + + flags - consists of two + parts, binary arguments and identification of + sub-operations. The binary arguments are : + + CCB_TRANS_DISC_ENB - enable disconnection + CCB_TRANS_TAG_ENB - + enable tagged queuing + + + + the sub-operations are: + + CCB_TRANS_CURRENT_SETTINGS + - change the current negotiations + + CCB_TRANS_USER_SETTINGS + - remember the desired user values sync_period, sync_offset - + self-explanatory, if sync_offset==0 then the asynchronous mode + is requested bus_width - bus width, in bits (not + bytes) + + + + + + Two sets of negotiated parameters are supported, the user + settings and the current settings. The user settings are not + really used much in the SIM drivers, this is mostly just a piece + of memory where the upper levels can store (and later recall) + its ideas about the parameters. Setting the user parameters + does not cause re-negotiation of the transfer rates. But when + the SCSI controller does a negotiation it must never set the + values higher than the user parameters, so it's essentially the + top boundary. + + The current settings are, as the name says, + current. Changing them means that the parameters must be + re-negotiated on the next transfer. Again, these "new current + settings" are not supposed to be forced on the device, just they + are used as the initial step of negotiations. Also they must be + limited by actual capabilities of the SCSI controller: for + example, if the SCSI controller has 8-bit bus and the request + asks to set 16-bit wide transfers this parameter must be + silently truncated to 8-bit transfers before sending it to the + device. + + One caveat is that the bus width and synchronous parameters + are per target while the disconnection and tag enabling + parameters are per lun. + + The recommended implementation is to keep 3 sets of + negotiated (bus width and synchronous transfer) + parameters: + + + user - the user set, as + above + + current - those actually + in effect + + goal - those requested by + setting of the "current" parameters + + + The code looks like: + + + struct ccb_trans_settings *cts; + int targ, lun; + int flags; + + cts = &ccb->cts; + targ = ccb_h->target_id; + lun = ccb_h->target_lun; + flags = cts->flags; + if(flags & CCB_TRANS_USER_SETTINGS) { + if(flags & CCB_TRANS_SYNC_RATE_VALID) + softc->user_sync_period[targ] = cts->sync_period; + if(flags & CCB_TRANS_SYNC_OFFSET_VALID) + softc->user_sync_offset[targ] = cts->sync_offset; + if(flags & CCB_TRANS_BUS_WIDTH_VALID) + softc->user_bus_width[targ] = cts->bus_width; + + if(flags & CCB_TRANS_DISC_VALID) { + softc->user_tflags[targ][lun] &= ~CCB_TRANS_DISC_ENB; + softc->user_tflags[targ][lun] |= flags & CCB_TRANS_DISC_ENB; + } + if(flags & CCB_TRANS_TQ_VALID) { + softc->user_tflags[targ][lun] &= ~CCB_TRANS_TQ_ENB; + softc->user_tflags[targ][lun] |= flags & CCB_TRANS_TQ_ENB; + } + } + if(flags & CCB_TRANS_CURRENT_SETTINGS) { + if(flags & CCB_TRANS_SYNC_RATE_VALID) + softc->goal_sync_period[targ] = + max(cts->sync_period, OUR_MIN_SUPPORTED_PERIOD); + if(flags & CCB_TRANS_SYNC_OFFSET_VALID) + softc->goal_sync_offset[targ] = + min(cts->sync_offset, OUR_MAX_SUPPORTED_OFFSET); + if(flags & CCB_TRANS_BUS_WIDTH_VALID) + softc->goal_bus_width[targ] = min(cts->bus_width, OUR_BUS_WIDTH); + + if(flags & CCB_TRANS_DISC_VALID) { + softc->current_tflags[targ][lun] &= ~CCB_TRANS_DISC_ENB; + softc->current_tflags[targ][lun] |= flags & CCB_TRANS_DISC_ENB; + } + if(flags & CCB_TRANS_TQ_VALID) { + softc->current_tflags[targ][lun] &= ~CCB_TRANS_TQ_ENB; + softc->current_tflags[targ][lun] |= flags & CCB_TRANS_TQ_ENB; + } + } + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + Then when the next I/O request will be processed it will + check if it has to re-negotiate, for example by calling the + function target_negotiated(hcb). It can be implemented like + this: + + + int + target_negotiated(struct xxx_hcb *hcb) + { + struct softc *softc = hcb->softc; + int targ = hcb->targ; + + if( softc->current_sync_period[targ] != softc->goal_sync_period[targ] + || softc->current_sync_offset[targ] != softc->goal_sync_offset[targ] + || softc->current_bus_width[targ] != softc->goal_bus_width[targ] ) + return 0; /* FALSE */ + else + return 1; /* TRUE */ + } + + + After the values are re-negotiated the resulting values must + be assigned to both current and goal parameters, so for future + I/O transactions the current and goal parameters would be the + same and target_negotiated() would return + TRUE. When the card is initialized (in + xxx_attach()) the current negotiation + values must be initialized to narrow asynchronous mode, the goal + and current values must be initialized to the maximal values + supported by controller. + + XPT_GET_TRAN_SETTINGS - get values of + SCSI transfer settings + + This operations is the reverse of + XPT_SET_TRAN_SETTINGS. Fill up the CCB instance "struct + ccb_trans_setting cts" with data as requested by the flags + CCB_TRANS_CURRENT_SETTINGS or CCB_TRANS_USER_SETTINGS (if both + are set then the existing drivers return the current + settings). Set all the bits in the valid field. + + XPT_CALC_GEOMETRY - calculate logical + (BIOS) geometry of the disk + + The arguments are transferred in the instance "struct + ccb_calc_geometry ccg" of the union ccb: + + + + block_size - input, block + (A.K.A sector) size in bytes + + volume_size - input, + volume size in bytes + + cylinders - output, + logical cylinders + + heads - output, logical + heads + + secs_per_track - output, + logical sectors per track + + + + If the returned geometry differs much enough from what the + SCSI controller BIOS thinks and a disk on this SCSI controller + is used as bootable the system may not be able to boot. The + typical calculation example taken from the aic7xxx driver + is: + + + struct ccb_calc_geometry *ccg; + u_int32_t size_mb; + u_int32_t secs_per_cylinder; + int extended; + + ccg = &ccb->ccg; + size_mb = ccg->volume_size + / ((1024L * 1024L) / ccg->block_size); + extended = check_cards_EEPROM_for_extended_geometry(softc); + + if (size_mb > 1024 && extended) { + ccg->heads = 255; + ccg->secs_per_track = 63; + } else { + ccg->heads = 64; + ccg->secs_per_track = 32; + } + secs_per_cylinder = ccg->heads * ccg->secs_per_track; + ccg->cylinders = ccg->volume_size / secs_per_cylinder; + ccb->ccb_h.status = CAM_REQ_CMP; + xpt_done(ccb); + return; + + + This gives the general idea, the exact calculation depends + on the quirks of the particular BIOS. If BIOS provides no way + set the "extended translation" flag in EEPROM this flag should + normally be assumed equal to 1. Other popular geometries + are: + + + 128 heads, 63 sectors - Symbios controllers + 16 heads, 63 sectors - old controllers + + + Some system BIOSes and SCSI BIOSes fight with each other + with variable success, for example a combination of Symbios + 875/895 SCSI and Phoenix BIOS can give geometry 128/63 after + power up and 255/63 after a hard reset or soft reboot. + + + XPT_PATH_INQ - path inquiry, in other + words get the SIM driver and SCSI controller (also known as HBA + - Host Bus Adapter) properties + + The properties are returned in the instance "struct +ccb_pathinq cpi" of the union ccb: + + + + version_num - the SIM driver version number, now + all drivers use 1 + + hba_inquiry - bitmask of features supported by + the controller: + + PI_MDP_ABLE - supports MDP message (something + from SCSI3?) + + PI_WIDE_32 - supports 32 bit wide + SCSI + + PI_WIDE_16 - supports 16 bit wide + SCSI + + PI_SDTR_ABLE - can negotiate synchronous + transfer rate + + PI_LINKED_CDB - supports linked + commands + + PI_TAG_ABLE - supports tagged + commands + + PI_SOFT_RST - supports soft reset alternative + (hard reset and soft reset are mutually exclusive within a + SCSI bus) + + target_sprt - flags for target mode support, 0 + if unsupported + + hba_misc - miscellaneous controller + features: + + PIM_SCANHILO - bus scans from high ID to low + ID + + PIM_NOREMOVE - removable devices not included in + scan + + PIM_NOINITIATOR - initiator role not + supported + + PIM_NOBUSRESET - user has disabled initial BUS + RESET + + hba_eng_cnt - mysterious HBA engine count, + something related to compression, now is always set to + 0 + + vuhba_flags - vendor-unique flags, unused + now + + max_target - maximal supported target ID (7 for + 8-bit bus, 15 for 16-bit bus, 127 for Fibre + Channel) + + max_lun - maximal supported LUN ID (7 for older + SCSI controllers, 63 for newer ones) + + async_flags - bitmask of installed Async + handler, unused now + + hpath_id - highest Path ID in the subsystem, + unused now + + unit_number - the controller unit number, + cam_sim_unit(sim) + + bus_id - the bus number, + cam_sim_bus(sim) + + initiator_id - the SCSI ID of the controller + itself + + base_transfer_speed - nominal transfer speed in + KB/s for asynchronous narrow transfers, equals to 3300 for + SCSI + + sim_vid - SIM driver's vendor id, a + zero-terminated string of maximal length SIM_IDLEN including + the terminating zero + + hba_vid - SCSI controller's vendor id, a + zero-terminated string of maximal length HBA_IDLEN including + the terminating zero + + dev_name - device driver name, a zero-terminated + string of maximal length DEV_IDLEN including the terminating + zero, equal to cam_sim_name(sim) + + + + The recommended way of setting the string fields is using + strncpy, like: + + + strncpy(cpi->dev_name, cam_sim_name(sim), DEV_IDLEN); + + + After setting the values set the status to CAM_REQ_CMP and mark the +CCB as done. + + + + + + + Polling + + + static void + xxx_poll + + + struct cam_sim *sim + + + + The poll function is used to simulate the interrupts when + the interrupt subsystem is not functioning (for example, when + the system has crashed and is creating the system dump). The CAM + subsystem sets the proper interrupt level before calling the + poll routine. So all it needs to do is to call the interrupt + routine (or the other way around, the poll routine may be doing + the real action and the interrupt routine would just call the + poll routine). Why bother about a separate function then ? + Because of different calling conventions. The + xxx_poll routine gets the struct cam_sim + pointer as its argument when the PCI interrupt routine by common + convention gets pointer to the struct + xxx_softc and the ISA interrupt routine + gets just the the device unit number. So the poll routine would + normally look as: + + +static void +xxx_poll(struct cam_sim *sim) +{ + xxx_intr((struct xxx_softc *)cam_sim_softc(sim)); /* for PCI device */ +} + + + or + + +static void +xxx_poll(struct cam_sim *sim) +{ + xxx_intr(cam_sim_unit(sim)); /* for ISA device */ +} + + + + + + Asynchronous Events + + If an asynchronous event callback has been set up then the + callback function should be defined. + + +static void +ahc_async(void *callback_arg, u_int32_t code, struct cam_path *path, void *arg) + + + + callback_arg - the value supplied when registering the + callback + + code - identifies the type of event + + path - identifies the devices to which the event + applies + + arg - event-specific argument + + + Implementation for a single type of event, AC_LOST_DEVICE, + looks like: + + + struct xxx_softc *softc; + struct cam_sim *sim; + int targ; + struct ccb_trans_settings neg; + + sim = (struct cam_sim *)callback_arg; + softc = (struct xxx_softc *)cam_sim_softc(sim); + switch (code) { + case AC_LOST_DEVICE: + targ = xpt_path_target_id(path); + if(targ <= OUR_MAX_SUPPORTED_TARGET) { + clean_negotiations(softc, targ); + /* send indication to CAM */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + xpt_async(AC_TRANSFER_NEG, path, &neg); + } + break; + default: + break; + } + + + + + + Interrupts + + The exact type of the interrupt routine depends on the type + of the peripheral bus (PCI, ISA and so on) to which the SCSI + controller is connected. + + The interrupt routines of the SIM drivers run at the + interrupt level splcam. So splcam() should + be used in the driver to synchronize activity between the + interrupt routine and the rest of the driver (for a + multiprocessor-aware driver things get yet more interesting but + we ignore this case here). The pseudo-code in this document + happily ignores the problems of synchronization. The real code + must not ignore them. A simple-minded approach is to set + splcam() on the entry to the other routines + and reset it on return thus protecting them by one big critical + section. To make sure that the interrupt level will be always + restored a wrapper function can be defined, like: + + + static void + xxx_action(struct cam_sim *sim, union ccb *ccb) + { + int s; + s = splcam(); + xxx_action1(sim, ccb); + splx(s); + } + + static void + xxx_action1(struct cam_sim *sim, union ccb *ccb) + { + ... process the request ... + } + + + This approach is simple and robust but the problem with it + is that interrupts may get blocked for a relatively long time + and this would negatively affect the system's performance. On + the other hand the functions of the spl() + family have rather high overhead, so vast amount of tiny + critical sections may not be good either. + + The conditions handled by the interrupt routine and the + details depend very much on the hardware. We consider the set of + "typical" conditions. + + First, we check if a SCSI reset was encountered on the bus + (probably caused by another SCSI controller on the same SCSI + bus). If so we drop all the enqueued and disconnected requests, + report the events and re-initialize our SCSI controller. It is + important that during this initialization the controller won't + issue another reset or else two controllers on the same SCSI bus + could ping-pong resets forever. The case of fatal controller + error/hang could be handled in the same place, but it will + probably need also sending RESET signal to the SCSI bus to reset + the status of the connections with the SCSI devices. + + + int fatal=0; + struct ccb_trans_settings neg; + struct cam_path *path; + + if( detected_scsi_reset(softc) + || (fatal = detected_fatal_controller_error(softc)) ) { + int targ, lun; + struct xxx_hcb *h, *hh; + + /* drop all enqueued CCBs */ + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* the clean values of negotiations to report */ + neg.bus_width = 8; + neg.sync_period = neg.sync_offset = 0; + neg.valid = (CCB_TRANS_BUS_WIDTH_VALID + | CCB_TRANS_SYNC_RATE_VALID | CCB_TRANS_SYNC_OFFSET_VALID); + + /* drop all disconnected CCBs and clean negotiations */ + for(targ=0; targ <= OUR_MAX_SUPPORTED_TARGET; targ++) { + clean_negotiations(softc, targ); + + /* report the event if possible */ + if(xpt_create_path(&path, /*periph*/NULL, + cam_sim_path(sim), targ, + CAM_LUN_WILDCARD) == CAM_REQ_CMP) { + xpt_async(AC_TRANSFER_NEG, path, &neg); + xpt_free_path(path); + } + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[targ][lun]; h != NULL; h = hh) { + hh=h->next; + if(fatal) + free_hcb_and_ccb_done(h, h->ccb, CAM_UNREC_HBA_ERROR); + else + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + } + + /* report the event */ + xpt_async(AC_BUS_RESET, softc->wpath, NULL); + + /* re-initialization may take a lot of time, in such case + * its completion should be signaled by another interrupt or + * checked on timeout - but for simplicity we assume here that + * it's really fast + */ + if(!fatal) { + reinitialize_controller_without_scsi_reset(softc); + } else { + reinitialize_controller_with_scsi_reset(softc); + } + schedule_next_hcb(softc); + return; + } + + + If interrupt is not caused by a controller-wide condition + then probably something has happened to the current hardware + control block. Depending on the hardware there may be other + non-HCB-related events, we just do not consider them here. Then + we analyze what happened to this HCB: + + + struct xxx_hcb *hcb, *h, *hh; + int hcb_status, scsi_status; + int ccb_status; + int targ; + int lun_to_freeze; + + hcb = get_current_hcb(softc); + if(hcb == NULL) { + /* either stray interrupt or something went very wrong + * or this is something hardware-dependent + */ + handle as necessary; + return; + } + + targ = hcb->target; + hcb_status = get_status_of_current_hcb(softc); + + + First we check if the HCB has completed and if so we check + the returned SCSI status. + + + if(hcb_status == COMPLETED) { + scsi_status = get_completion_status(hcb); + + + Then look if this status is related to the REQUEST SENSE + command and if so handle it in a simple way. + + + if(hcb->flags & DOING_AUTOSENSE) { + if(scsi_status == GOOD) { /* autosense was successful */ + hcb->ccb->ccb_h.status |= CAM_AUTOSNS_VALID; + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_SCSI_STATUS_ERROR); + } else { + autosense_failed: + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_AUTOSENSE_FAIL); + } + schedule_next_hcb(softc); + return; + } + + + Else the command itself has completed, pay more attention to + details. If auto-sense is not disabled for this CCB and the + command has failed with sense data then run REQUEST SENSE + command to receive that data. + + + hcb->ccb->csio.scsi_status = scsi_status; + calculate_residue(hcb); + + if( (hcb->ccb->ccb_h.flags & CAM_DIS_AUTOSENSE)==0 + && ( scsi_status == CHECK_CONDITION + || scsi_status == COMMAND_TERMINATED) ) { + /* start auto-SENSE */ + hcb->flags |= DOING_AUTOSENSE; + setup_autosense_command_in_hcb(hcb); + restart_current_hcb(softc); + return; + } + if(scsi_status == GOOD) + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_REQ_CMP); + else + free_hcb_and_ccb_done(hcb, hcb->ccb, CAM_SCSI_STATUS_ERROR); + schedule_next_hcb(softc); + return; + } + + + One typical thing would be negotiation events: negotiation + messages received from a SCSI target (in answer to our + negotiation attempt or by target's initiative) or the target is + unable to negotiate (rejects our negotiation messages or does + not answer them). + + + switch(hcb_status) { + case TARGET_REJECTED_WIDE_NEG: + /* revert to 8-bit bus */ + softc->current_bus_width[targ] = softc->goal_bus_width[targ] = 8; + /* report the event */ + neg.bus_width = 8; + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + continue_current_hcb(softc); + return; + case TARGET_ANSWERED_WIDE_NEG: + { + int wd; + + wd = get_target_bus_width_request(softc); + if(wd <= softc->goal_bus_width[targ]) { + /* answer is acceptable */ + softc->current_bus_width[targ] = + softc->goal_bus_width[targ] = neg.bus_width = wd; + + /* report the event */ + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + } else { + prepare_reject_message(hcb); + } + } + continue_current_hcb(softc); + return; + case TARGET_REQUESTED_WIDE_NEG: + { + int wd; + + wd = get_target_bus_width_request(softc); + wd = min (wd, OUR_BUS_WIDTH); + wd = min (wd, softc->user_bus_width[targ]); + + if(wd != softc->current_bus_width[targ]) { + /* the bus width has changed */ + softc->current_bus_width[targ] = + softc->goal_bus_width[targ] = neg.bus_width = wd; + + /* report the event */ + neg.valid = CCB_TRANS_BUS_WIDTH_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + } + prepare_width_nego_rsponse(hcb, wd); + } + continue_current_hcb(softc); + return; + } + + + Then we handle any errors that could have happened during + auto-sense in the same simple-minded way as before. Otherwise we + look closer at the details again. + + + if(hcb->flags & DOING_AUTOSENSE) + goto autosense_failed; + + switch(hcb_status) { + + + The next event we consider is unexpected disconnect. Which + is considered normal after an ABORT or BUS DEVICE RESET message + and abnormal in other cases. + + + case UNEXPECTED_DISCONNECT: + if(requested_abort(hcb)) { + /* abort affects all commands on that target+LUN, so + * mark all disconnected HCBs on that target+LUN as aborted too + */ + for(h = softc->first_discon_hcb[hcb->target][hcb->lun]; + h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_REQ_ABORTED); + } + ccb_status = CAM_REQ_ABORTED; + } else if(requested_bus_device_reset(hcb)) { + int lun; + + /* reset affects all commands on that target, so + * mark all disconnected HCBs on that target+LUN as reset + */ + + for(lun=0; lun <= OUR_MAX_SUPPORTED_LUN; lun++) + for(h = softc->first_discon_hcb[hcb->target][lun]; + h != NULL; h = hh) { + hh=h->next; + free_hcb_and_ccb_done(h, h->ccb, CAM_SCSI_BUS_RESET); + } + + /* send event */ + xpt_async(AC_SENT_BDR, hcb->ccb->ccb_h.path_id, NULL); + + /* this was the CAM_RESET_DEV request itself, it's completed */ + ccb_status = CAM_REQ_CMP; + } else { + calculate_residue(hcb); + ccb_status = CAM_UNEXP_BUSFREE; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = hcb->lun; + } + break; + + + If the target refuses to accept tags we notify CAM about + that and return back all commands for this LUN: + + + case TAGS_REJECTED: + /* report the event */ + neg.flags = 0 & ~CCB_TRANS_TAG_ENB; + neg.valid = CCB_TRANS_TQ_VALID; + xpt_async(AC_TRANSFER_NEG, hcb->ccb.ccb_h.path_id, &neg); + + ccb_status = CAM_MSG_REJECT_REC; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = hcb->lun; + break; + + + Then we check a number of other conditions, with processing + basically limited to setting the CCB status: + + + case SELECTION_TIMEOUT: + ccb_status = CAM_SEL_TIMEOUT; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = CAM_LUN_WILDCARD; + break; + case PARITY_ERROR: + ccb_status = CAM_UNCOR_PARITY; + break; + case DATA_OVERRUN: + case ODD_WIDE_TRANSFER: + ccb_status = CAM_DATA_RUN_ERR; + break; + default: + /* all other errors are handled in a generic way */ + ccb_status = CAM_REQ_CMP_ERR; + /* request the further code to freeze the queue */ + hcb->ccb->ccb_h.status |= CAM_DEV_QFRZN; + lun_to_freeze = CAM_LUN_WILDCARD; + break; + } + + + Then we check if the error was serious enough to freeze the + input queue until it gets proceeded and do so if it is: + + + if(hcb->ccb->ccb_h.status & CAM_DEV_QFRZN) { + /* freeze the queue */ + xpt_freeze_devq(ccb->ccb_h.path, /*count*/1); + + /* re-queue all commands for this target/LUN back to CAM */ + + for(h = softc->first_queued_hcb; h != NULL; h = hh) { + hh = h->next; + + if(targ == h->targ + && (lun_to_freeze == CAM_LUN_WILDCARD || lun_to_freeze == h->lun) ) + free_hcb_and_ccb_done(h, h->ccb, CAM_REQUEUE_REQ); + } + } + free_hcb_and_ccb_done(hcb, hcb->ccb, ccb_status); + schedule_next_hcb(softc); + return; + + + This concludes the generic interrupt handling although + specific controllers may require some additions. + + + + + Errors Summary + + When executing an I/O request many things may go wrong. The + reason of error can be reported in the CCB status with great + detail. Examples of use are spread throughout this document. For + completeness here is the summary of recommended responses for + the typical error conditions: + + + + CAM_RESRC_UNAVAIL - some + resource is temporarily unavailable and the SIM driver can not + generate an event when it will become available. An example of + this resource would be some intra-controller hardware resource + for which the controller does not generate an interrupt when + it becomes available. + + CAM_UNCOR_PARITY - + unrecovered parity error occurred + + CAM_DATA_RUN_ERR - data + overrun or unexpected data phase (going in other direction + than specified in CAM_DIR_MASK) or odd transfer length for + wide transfer + + CAM_SEL_TIMEOUT - selection + timeout occurred (target does not respond) + + CAM_CMD_TIMEOUT - command + timeout occurred (the timeout function ran) + + CAM_SCSI_STATUS_ERROR - the + device returned error + + CAM_AUTOSENSE_FAIL - the + device returned error and the REQUEST SENSE COMMAND + failed + + CAM_MSG_REJECT_REC - MESSAGE + REJECT message was received + + CAM_SCSI_BUS_RESET - received + SCSI bus reset + + CAM_REQ_CMP_ERR - + "impossible" SCSI phase occurred or something else as weird or + just a generic error if further detail is not + available + + CAM_UNEXP_BUSFREE - + unexpected disconnect occurred + + CAM_BDR_SENT - BUS DEVICE + RESET message was sent to the target + + CAM_UNREC_HBA_ERROR - + unrecoverable Host Bus Adapter Error + + CAM_REQ_TOO_BIG - the request + was too large for this controller + + CAM_REQUEUE_REQ - this + request should be re-queued to preserve transaction ordering. + This typically occurs when the SIM recognizes an error that + should freeze the queue and must place other queued requests + for the target at the sim level back into the XPT + queue. Typical cases of such errors are selection timeouts, + command timeouts and other like conditions. In such cases the + troublesome command returns the status indicating the error, + the and the other commands which have not be sent to the bus + yet get re-queued. + + CAM_LUN_INVALID - the LUN + ID in the request is not supported by the SCSI + controller + + CAM_TID_INVALID - the + target ID in the request is not supported by the SCSI + controller + + + + + Timeout Handling + + When the timeout for an HCB expires that request should be + aborted, just like with an XPT_ABORT request. The only + difference is that the returned status of aborted request should + be CAM_CMD_TIMEOUT instead of CAM_REQ_ABORTED (that's why + implementation of the abort better be done as a function). But + there is one more possible problem: what if the abort request + itself will get stuck? In this case the SCSI bus should be + reset, just like with an XPT_RESET_BUS request (and the idea + about implementing it as a function called from both places + applies here too). Also we should reset the whole SCSI bus if a + device reset request got stuck. So after all the timeout + function would look like: + + +static void +xxx_timeout(void *arg) +{ + struct xxx_hcb *hcb = (struct xxx_hcb *)arg; + struct xxx_softc *softc; + struct ccb_hdr *ccb_h; + + softc = hcb->softc; + ccb_h = &hcb->ccb->ccb_h; + + if(hcb->flags & HCB_BEING_ABORTED + || ccb_h->func_code == XPT_RESET_DEV) { + xxx_reset_bus(softc); + } else { + xxx_abort_ccb(hcb->ccb, CAM_CMD_TIMEOUT); + } +} + + + When we abort a request all the other disconnected requests + to the same target/LUN get aborted too. So there appears a + question, should we return them with status CAM_REQ_ABORTED or + CAM_CMD_TIMEOUT ? The current drivers use CAM_CMD_TIMEOUT. This + seems logical because if one request got timed out then probably + something really bad is happening to the device, so if they + would not be disturbed they would time out by themselves. + + + +