D33114.1792553434.diff
No OneTemporary
Actions

Size

287 KB

Referenced Files

None

Subscribers

None

D33114.1792553434.diff
View Options

This file is larger than 256 KB, so syntax highlighting was skipped.

	Index: share/man/man4/Makefile
	===================================================================
	--- share/man/man4/Makefile
	+++ share/man/man4/Makefile
	@@ -552,7 +552,6 @@
	${_tpm.4} \
	tty.4 \
	tun.4 \
	- twa.4 \
	twe.4 \
	tws.4 \
	udp.4 \
	Index: share/man/man4/twa.4
	===================================================================
	--- share/man/man4/twa.4
	+++ /dev/null
	@@ -1,140 +0,0 @@
	-.\"
	-.\" Copyright (c) 2004 3ware, Inc.
	-.\" Copyright (c) 2000 BSDi
	-.\" All rights reserved.
	-.\"
	-.\" Redistribution and use in source and binary forms, with or without
	-.\" modification, are permitted provided that the following conditions
	-.\" are met:
	-.\" 1. Redistributions of source code must retain the above copyright
	-.\" notice, this list of conditions and the following disclaimer.
	-.\" 2. Redistributions in binary form must reproduce the above copyright
	-.\" notice, this list of conditions and the following disclaimer in the
	-.\" documentation and/or other materials provided with the distribution.
	-.\"
	-.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
	-.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
	-.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
	-.\" IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
	-.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
	-.\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
	-.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
	-.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
	-.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
	-.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
	-.\"
	-.\" $FreeBSD$
	-.\"
	-.Dd May 9, 2007
	-.Dt TWA 4
	-.Os
	-.Sh NAME
	-.Nm twa
	-.Nd 3ware 9000/9500/9550/9650 series SATA RAID controllers driver
	-.Sh DEPRECATION NOTICE
	-The
	-.Nm
	-driver is not present in
	-.Fx 13.0 .
	-.Sh SYNOPSIS
	-To compile this driver into the kernel,
	-place the following lines in your
	-kernel configuration file:
	-.Bd -ragged -offset indent
	-.Cd "device scbus"
	-.Cd "device twa"
	-.Ed
	-.Pp
	-Alternatively, to load the driver as a
	-module at boot time, place the following line in
	-.Xr loader.conf 5 :
	-.Bd -literal -offset indent
	-twa_load="YES"
	-.Ed
	-.Sh DESCRIPTION
	-The
	-.Nm
	-driver provides support for AMCC's 3ware 9000/9500/9550/9650 series
	-SATA controllers.
	-.Pp
	-These controllers are available in 4, 8, 12 or 16-port configurations,
	-and support the following RAID levels: 0, 1, 10, 5, 50.
	-The device nodes for the controllers are of the form
	-.Pa /dev/twa Ns Ar X ,
	-where
	-.Ar X
	-is the controller number.
	-The driver is implemented as a SCSI SIM
	-under CAM, and, as such, the logical units that it controls are accessible
	-via the device nodes,
	-.Pa /dev/da Ns Ar Y ,
	-where
	-.Ar Y
	-is the logical unit number.
	-.Sh HARDWARE
	-The
	-.Nm
	-driver supports the following SATA RAID controllers:
	-.Pp
	-.Bl -bullet -compact
	-.It
	-AMCC's 3ware 9500S-4LP
	-.It
	-AMCC's 3ware 9500S-8
	-.It
	-AMCC's 3ware 9500S-8MI
	-.It
	-AMCC's 3ware 9500S-12
	-.It
	-AMCC's 3ware 9500S-12MI
	-.It
	-AMCC's 3ware 9500SX-4LP
	-.It
	-AMCC's 3ware 9500SX-8LP
	-.It
	-AMCC's 3ware 9500SX-12
	-.It
	-AMCC's 3ware 9500SX-12MI
	-.It
	-AMCC's 3ware 9500SX-16ML
	-.It
	-AMCC's 3ware 9550SX-4LP
	-.It
	-AMCC's 3ware 9550SX-8LP
	-.It
	-AMCC's 3ware 9550SX-12
	-.It
	-AMCC's 3ware 9550SX-12MI
	-.It
	-AMCC's 3ware 9550SX-16ML
	-.It
	-AMCC's 3ware 9650SE-2LP
	-.It
	-AMCC's 3ware 9650SE-4LPML
	-.It
	-AMCC's 3ware 9650SE-8LPML
	-.It
	-AMCC's 3ware 9650SE-12ML
	-.It
	-AMCC's 3ware 9650SE-16ML
	-.It
	-AMCC's 3ware 9650SE-24M8
	-.El
	-.Sh DIAGNOSTICS
	-Whenever the driver encounters a command failure, it prints out an error code in
	-the format:
	-.Qq Li "ERROR: (<error source>: <error code>):" ,
	-followed by a text description of the error.
	-There are other error messages and warnings that the
	-driver prints out, depending on the kinds of errors that it encounters.
	-If the driver is compiled with
	-.Dv TWA_DEBUG
	-defined, it prints out a whole bunch of debug
	-messages, the quantity of which varies depending on the value assigned to
	-.Dv TWA_DEBUG
	-(0 to 10).
	-.Sh AUTHORS
	-The
	-.Nm
	-driver and manpage were written by
	-.An Vinod Kashyap Aq Mt vkashyap@FreeBSD.org .
	Index: sys/amd64/conf/GENERIC
	===================================================================
	--- sys/amd64/conf/GENERIC
	+++ sys/amd64/conf/GENERIC
	@@ -182,7 +182,6 @@
	device arcmsr # Areca SATA II RAID
	device ciss # Compaq Smart RAID 5*
	device ips # IBM (Adaptec) ServeRAID
	-device twa # 3ware 9000 series PATA/SATA RAID
	device smartpqi # Microsemi smartpqi driver
	device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller

	Index: sys/amd64/conf/NOTES
	===================================================================
	--- sys/amd64/conf/NOTES
	+++ sys/amd64/conf/NOTES
	@@ -406,13 +406,6 @@
	#
	device smartpqi

	-#
	-# 3ware 9000 series PATA/SATA RAID controller driver and options.
	-# The driver is implemented as a SIM, and so, needs the CAM infrastructure.
	-#
	-options TWA_DEBUG # 0-10; 10 prints the most messages.
	-device twa # 3ware 9000 series PATA/SATA RAID
	-
	#
	# Adaptec FSA RAID controllers, including integrated DELL controllers,
	# the Dell PERC 2/QC and the HP NetRAID-4M
	Index: sys/conf/files
	===================================================================
	--- sys/conf/files
	+++ sys/conf/files
	@@ -3205,18 +3205,6 @@
	dev/tcp_log/tcp_log_dev.c optional tcp_blackbox inet \| tcp_blackbox inet6
	dev/tdfx/tdfx_pci.c optional tdfx pci
	dev/ti/if_ti.c optional ti pci
	-dev/twa/tw_cl_init.c optional twa \
	- compile-with "${NORMAL_C} -I$S/dev/twa"
	-dev/twa/tw_cl_intr.c optional twa \
	- compile-with "${NORMAL_C} -I$S/dev/twa"
	-dev/twa/tw_cl_io.c optional twa \
	- compile-with "${NORMAL_C} -I$S/dev/twa"
	-dev/twa/tw_cl_misc.c optional twa \
	- compile-with "${NORMAL_C} -I$S/dev/twa"
	-dev/twa/tw_osl_cam.c optional twa \
	- compile-with "${NORMAL_C} -I$S/dev/twa"
	-dev/twa/tw_osl_freebsd.c optional twa \
	- compile-with "${NORMAL_C} -I$S/dev/twa"
	dev/twe/twe.c optional twe
	dev/twe/twe_freebsd.c optional twe
	dev/tws/tws.c optional tws
	Index: sys/conf/options
	===================================================================
	--- sys/conf/options
	+++ sys/conf/options
	@@ -43,8 +43,6 @@
	AHD_TMODE_ENABLE opt_aic79xx.h
	AHD_REG_PRETTY_PRINT opt_aic79xx.h

	-TWA_DEBUG opt_twa.h
	-
	# Debugging options.
	ALT_BREAK_TO_DEBUGGER opt_kdb.h
	BREAK_TO_DEBUGGER opt_kdb.h
	Index: sys/dev/twa/tw_cl.h
	===================================================================
	--- sys/dev/twa/tw_cl.h
	+++ /dev/null
	@@ -1,324 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_CL_H
	-
	-#define TW_CL_H
	-
	-/*
	- * Common Layer internal macros, structures and functions.
	- */
	-
	-#define TW_CLI_SECTOR_SIZE 0x200
	-#define TW_CLI_REQUEST_TIMEOUT_PERIOD 60 /* seconds */
	-#define TW_CLI_RESET_TIMEOUT_PERIOD 60 /* seconds */
	-#define TW_CLI_MAX_RESET_ATTEMPTS 2
	-
	-/* Possible values of ctlr->ioctl_lock.lock. */
	-#define TW_CLI_LOCK_FREE 0x0 /* lock is free */
	-#define TW_CLI_LOCK_HELD 0x1 /* lock is held */
	-
	-/* Possible values of req->state. */
	-#define TW_CLI_REQ_STATE_INIT 0x0 /* being initialized */
	-#define TW_CLI_REQ_STATE_BUSY 0x1 /* submitted to controller */
	-#define TW_CLI_REQ_STATE_PENDING 0x2 /* in pending queue */
	-#define TW_CLI_REQ_STATE_COMPLETE 0x3 /* completed by controller */
	-
	-/* Possible values of req->flags. */
	-#define TW_CLI_REQ_FLAGS_7K (1<<0) /* 7000 cmd pkt */
	-#define TW_CLI_REQ_FLAGS_9K (1<<1) /* 9000 cmd pkt */
	-#define TW_CLI_REQ_FLAGS_INTERNAL (1<<2) /* internal request */
	-#define TW_CLI_REQ_FLAGS_PASSTHRU (1<<3) /* passthru request */
	-#define TW_CLI_REQ_FLAGS_EXTERNAL (1<<4) /* external request */
	-
	-#ifdef TW_OSL_PCI_CONFIG_ACCESSIBLE
	-/* Register offsets in PCI config space. */
	-#define TW_CLI_PCI_CONFIG_COMMAND_OFFSET 0x4 /* cmd register offset */
	-#define TW_CLI_PCI_CONFIG_STATUS_OFFSET 0x6 /* status register offset */
	-#endif /* TW_OSL_PCI_CONFIG_ACCESSIBLE */
	-
	-#ifdef TW_OSL_DEBUG
	-struct tw_cli_q_stats {
	- TW_UINT32 cur_len;/* current # of entries in q */
	- TW_UINT32 max_len; /* max # of entries in q, ever reached */
	-};
	-#endif /* TW_OSL_DEBUG */
	-
	-/* Queues of CL internal request context packets. */
	-#define TW_CLI_FREE_Q 0 /* free q */
	-#define TW_CLI_BUSY_Q 1 /* q of reqs submitted to fw */
	-#define TW_CLI_PENDING_Q 2 /* q of reqs deferred due to 'q full' */
	-#define TW_CLI_COMPLETE_Q 3 /* q of reqs completed by fw */
	-#define TW_CLI_RESET_Q 4 /* q of reqs reset by timeout */
	-#define TW_CLI_Q_COUNT 5 /* total number of queues */
	-
	-/* CL's internal request context. */
	-struct tw_cli_req_context {
	- struct tw_cl_req_handle req_handle;/ handle to track requests between
	- OSL & CL */
	- struct tw_cli_ctlr_context ctlr; / ptr to CL's controller context */
	- struct tw_cl_command_packet cmd_pkt;/ ptr to ctlr cmd pkt */
	- TW_UINT64 cmd_pkt_phys; /* cmd pkt physical address */
	- TW_VOID data; / ptr to data being passed to fw */
	- TW_UINT32 length; /* length of data being passed to fw */
	- TW_UINT64 data_phys; /* physical address of data */
	-
	- TW_UINT32 state; /* request state */
	- TW_UINT32 flags; /* request flags */
	-
	- TW_UINT32 error_code; /* error encountered before submission
	- of request to fw, if any */
	-
	- TW_VOID orig_req; / ptr to original request for use
	- during callback */
	- TW_VOID (tw_cli_callback)(struct tw_cli_req_context req);
	- /* CL internal callback */
	- TW_UINT32 request_id; /* request id for tracking with fw */
	- struct tw_cl_link link; /* to link this request in a list */
	-};
	-
	-/* CL's internal controller context. */
	-struct tw_cli_ctlr_context {
	- struct tw_cl_ctlr_handle ctlr_handle; / handle to track ctlr between
	- OSL & CL. */
	- struct tw_cli_req_context req_ctxt_buf;/ pointer to the array of CL's
	- internal request context pkts */
	- struct tw_cl_command_packet cmd_pkt_buf;/ ptr to array of cmd pkts */
	-
	- TW_UINT64 cmd_pkt_phys; /* phys addr of cmd_pkt_buf */
	-
	- TW_UINT32 device_id; /* controller device id */
	- TW_UINT32 arch_id; /* controller architecture id */
	- TW_UINT8 active; /* Initialization done, and controller is active. */
	- TW_UINT8 interrupts_enabled; /* Interrupts on controller enabled. */
	- TW_UINT8 internal_req_busy; /* Data buffer for internal requests in use. */
	- TW_UINT8 get_more_aens; /* More AEN's need to be retrieved. */
	- TW_UINT8 reset_needed; /* Controller needs a soft reset. */
	- TW_UINT8 reset_in_progress; /* Controller is being reset. */
	- TW_UINT8 reset_phase1_in_progress; /* In 'phase 1' of reset. */
	- TW_UINT32 flags; /* controller settings */
	- TW_UINT32 sg_size_factor; /* SG element size should be a
	- multiple of this */
	-
	- /* Request queues and arrays. */
	- struct tw_cl_link req_q_head[TW_CLI_Q_COUNT];
	-
	- TW_UINT8 internal_req_data;/ internal req data buf */
	- TW_UINT64 internal_req_data_phys;/* phys addr of internal
	- req data buf */
	- TW_UINT32 max_simult_reqs; /* max simultaneous requests
	- supported */
	- TW_UINT32 max_aens_supported;/* max AEN's supported */
	- /* AEN handler fields. */
	- struct tw_cl_event_packet aen_queue; / circular queue of AENs from
	- firmware/CL/OSL */
	- TW_UINT32 aen_head; /* AEN queue head */
	- TW_UINT32 aen_tail; /* AEN queue tail */
	- TW_UINT32 aen_cur_seq_id; /* index of the last event+1 */
	- TW_UINT32 aen_q_overflow; /* indicates if unretrieved
	- events were overwritten */
	- TW_UINT32 aen_q_wrapped; /* indicates if AEN queue ever
	- wrapped */
	-
	- TW_UINT16 working_srl; /* driver & firmware negotiated
	- srl */
	- TW_UINT16 working_branch; /* branch # of the firmware
	- that the driver is compatible with */
	- TW_UINT16 working_build; /* build # of the firmware
	- that the driver is compatible with */
	- TW_UINT16 fw_on_ctlr_srl; /* srl of running firmware */
	- TW_UINT16 fw_on_ctlr_branch;/* branch # of running
	- firmware */
	- TW_UINT16 fw_on_ctlr_build;/* build # of running
	- firmware */
	- TW_UINT32 operating_mode; /* base mode/current mode */
	-
	- TW_INT32 host_intr_pending;/* host intr processing
	- needed */
	- TW_INT32 attn_intr_pending;/* attn intr processing
	- needed */
	- TW_INT32 cmd_intr_pending;/* cmd intr processing
	- needed */
	- TW_INT32 resp_intr_pending;/* resp intr processing
	- needed */
	-
	- TW_LOCK_HANDLE gen_lock_handle;/* general purpose lock */
	- TW_LOCK_HANDLE gen_lock;/ ptr to general purpose lock */
	- TW_LOCK_HANDLE io_lock_handle; /* lock held during cmd
	- submission */
	- TW_LOCK_HANDLE io_lock;/ ptr to lock held during cmd
	- submission */
	-
	-#ifdef TW_OSL_CAN_SLEEP
	- TW_SLEEP_HANDLE sleep_handle; /* handle to co-ordinate sleeps
	- & wakeups */
	-#endif /* TW_OSL_CAN_SLEEP */
	-
	- struct {
	- TW_UINT32 lock; /* lock state */
	- TW_TIME timeout; /* time at which the lock will
	- become available, even if not
	- explicitly released */
	- } ioctl_lock; /* lock for use by user applications, for
	- synchronization between ioctl calls */
	-#ifdef TW_OSL_DEBUG
	- struct tw_cli_q_stats q_stats[TW_CLI_Q_COUNT];/* queue statistics */
	-#endif /* TW_OSL_DEBUG */
	-};
	-
	-/*
	- * Queue primitives
	- */
	-
	-#ifdef TW_OSL_DEBUG
	-
	-#define TW_CLI_Q_INIT(ctlr, q_type) do { \
	- (ctlr)->q_stats[q_type].cur_len = 0; \
	- (ctlr)->q_stats[q_type].max_len = 0; \
	-} while (0)
	-
	-#define TW_CLI_Q_INSERT(ctlr, q_type) do { \
	- struct tw_cli_q_stats *q_stats = &((ctlr)->q_stats[q_type]); \
	- \
	- if (++(q_stats->cur_len) > q_stats->max_len) \
	- q_stats->max_len = q_stats->cur_len; \
	-} while (0)
	-
	-#define TW_CLI_Q_REMOVE(ctlr, q_type) \
	- (ctlr)->q_stats[q_type].cur_len--
	-
	-#else /* TW_OSL_DEBUG */
	-
	-#define TW_CLI_Q_INIT(ctlr, q_index)
	-#define TW_CLI_Q_INSERT(ctlr, q_index)
	-#define TW_CLI_Q_REMOVE(ctlr, q_index)
	-
	-#endif /* TW_OSL_DEBUG */
	-
	-/* Initialize a queue of requests. */
	-static __inline TW_VOID
	-tw_cli_req_q_init(struct tw_cli_ctlr_context *ctlr, TW_UINT8 q_type)
	-{
	- TW_CL_Q_INIT(&(ctlr->req_q_head[q_type]));
	- TW_CLI_Q_INIT(ctlr, q_type);
	-}
	-
	-/* Insert the given request at the head of the given queue (q_type). */
	-static __inline TW_VOID
	-tw_cli_req_q_insert_head(struct tw_cli_req_context *req, TW_UINT8 q_type)
	-{
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	-
	- tw_osl_get_lock(ctlr->ctlr_handle, ctlr->gen_lock);
	- TW_CL_Q_INSERT_HEAD(&(ctlr->req_q_head[q_type]), &(req->link));
	- TW_CLI_Q_INSERT(ctlr, q_type);
	- tw_osl_free_lock(ctlr->ctlr_handle, ctlr->gen_lock);
	-}
	-
	-/* Insert the given request at the tail of the given queue (q_type). */
	-static __inline TW_VOID
	-tw_cli_req_q_insert_tail(struct tw_cli_req_context *req, TW_UINT8 q_type)
	-{
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	-
	- tw_osl_get_lock(ctlr->ctlr_handle, ctlr->gen_lock);
	- TW_CL_Q_INSERT_TAIL(&(ctlr->req_q_head[q_type]), &(req->link));
	- TW_CLI_Q_INSERT(ctlr, q_type);
	- tw_osl_free_lock(ctlr->ctlr_handle, ctlr->gen_lock);
	-}
	-
	-/* Remove and return the request at the head of the given queue (q_type). */
	-static __inline struct tw_cli_req_context *
	-tw_cli_req_q_remove_head(struct tw_cli_ctlr_context *ctlr, TW_UINT8 q_type)
	-{
	- struct tw_cli_req_context *req = TW_CL_NULL;
	- struct tw_cl_link *link;
	-
	- tw_osl_get_lock(ctlr->ctlr_handle, ctlr->gen_lock);
	- if ((link = TW_CL_Q_FIRST_ITEM(&(ctlr->req_q_head[q_type]))) !=
	- TW_CL_NULL) {
	- req = TW_CL_STRUCT_HEAD(link,
	- struct tw_cli_req_context, link);
	- TW_CL_Q_REMOVE_ITEM(&(ctlr->req_q_head[q_type]), &(req->link));
	- TW_CLI_Q_REMOVE(ctlr, q_type);
	- }
	- tw_osl_free_lock(ctlr->ctlr_handle, ctlr->gen_lock);
	- return(req);
	-}
	-
	-/* Remove the given request from the given queue (q_type). */
	-static __inline TW_VOID
	-tw_cli_req_q_remove_item(struct tw_cli_req_context *req, TW_UINT8 q_type)
	-{
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	-
	- tw_osl_get_lock(ctlr->ctlr_handle, ctlr->gen_lock);
	- TW_CL_Q_REMOVE_ITEM(&(ctlr->req_q_head[q_type]), &(req->link));
	- TW_CLI_Q_REMOVE(ctlr, q_type);
	- tw_osl_free_lock(ctlr->ctlr_handle, ctlr->gen_lock);
	-}
	-
	-/* Create an event packet for an event/error posted by the controller. */
	-#define tw_cli_create_ctlr_event(ctlr, event_src, cmd_hdr) do { \
	- TW_UINT8 severity = \
	- GET_SEVERITY((cmd_hdr)->status_block.res__severity); \
	- \
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_TRUE, event_src, \
	- (cmd_hdr)->status_block.error, \
	- severity, \
	- tw_cli_severity_string_table[severity], \
	- (cmd_hdr)->err_specific_desc + \
	- tw_osl_strlen((cmd_hdr)->err_specific_desc) + 1, \
	- (cmd_hdr)->err_specific_desc); \
	- /* Print 18 bytes of sense information. */ \
	- tw_cli_dbg_printf(2, ctlr->ctlr_handle, \
	- tw_osl_cur_func(), \
	- "sense info: %x %x %x %x %x %x %x %x %x " \
	- "%x %x %x %x %x %x %x %x %x", \
	- (cmd_hdr)->sense_data[0], (cmd_hdr)->sense_data[1], \
	- (cmd_hdr)->sense_data[2], (cmd_hdr)->sense_data[3], \
	- (cmd_hdr)->sense_data[4], (cmd_hdr)->sense_data[5], \
	- (cmd_hdr)->sense_data[6], (cmd_hdr)->sense_data[7], \
	- (cmd_hdr)->sense_data[8], (cmd_hdr)->sense_data[9], \
	- (cmd_hdr)->sense_data[10], (cmd_hdr)->sense_data[11], \
	- (cmd_hdr)->sense_data[12], (cmd_hdr)->sense_data[13], \
	- (cmd_hdr)->sense_data[14], (cmd_hdr)->sense_data[15], \
	- (cmd_hdr)->sense_data[16], (cmd_hdr)->sense_data[17]); \
	-} while (0)
	-
	-#endif /* TW_CL_H */
	Index: sys/dev/twa/tw_cl_externs.h
	===================================================================
	--- sys/dev/twa/tw_cl_externs.h
	+++ /dev/null
	@@ -1,191 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_CL_EXTERNS_H
	-
	-#define TW_CL_EXTERNS_H
	-
	-/*
	- * Data structures and functions global to the Common Layer.
	- */
	-
	-extern TW_INT8 tw_cli_fw_img[];
	-extern TW_INT32 tw_cli_fw_img_size;
	-extern TW_INT8 *tw_cli_severity_string_table[];
	-
	-/* Do controller initialization. */
	-extern TW_INT32 tw_cli_start_ctlr(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Establish a logical connection with the firmware on the controller. */
	-extern TW_INT32 tw_cli_init_connection(struct tw_cli_ctlr_context *ctlr,
	- TW_UINT16 message_credits, TW_UINT32 set_features,
	- TW_UINT16 current_fw_srl, TW_UINT16 current_fw_arch_id,
	- TW_UINT16 current_fw_branch, TW_UINT16 current_fw_build,
	- TW_UINT16 fw_on_ctlr_srl, TW_UINT16 fw_on_ctlr_arch_id,
	- TW_UINT16 fw_on_ctlr_branch, TW_UINT16 fw_on_ctlr_build,
	- TW_UINT32 *init_connect_result);
	-
	-/* Functions in tw_cl_io.c */
	-
	-/* Submit a command packet to the firmware on the controller. */
	-extern TW_INT32 tw_cli_submit_cmd(struct tw_cli_req_context *req);
	-
	-/* Get a firmware parameter. */
	-extern TW_INT32 tw_cli_get_param(struct tw_cli_ctlr_context *ctlr,
	- TW_INT32 table_id, TW_INT32 parameter_id, TW_VOID *param_data,
	- TW_INT32 size, TW_VOID (* callback)(struct tw_cli_req_context *req));
	-
	-/* Set a firmware parameter. */
	-extern TW_INT32 tw_cli_set_param(struct tw_cli_ctlr_context *ctlr,
	- TW_INT32 table_id, TW_INT32 param_id, TW_INT32 param_size,
	- TW_VOID data, TW_VOID ( callback)(struct tw_cli_req_context *req));
	-
	-/* Submit a command to the firmware and poll for completion. */
	-extern TW_INT32 tw_cli_submit_and_poll_request(struct tw_cli_req_context *req,
	- TW_UINT32 timeout);
	-
	-/* Soft reset the controller. */
	-extern TW_INT32 tw_cli_soft_reset(struct tw_cli_ctlr_context *ctlr);
	-extern int twa_setup_intr(struct twa_softc *sc);
	-extern int twa_teardown_intr(struct twa_softc *sc);
	-
	-/* Send down a SCSI command to the firmware (usually, an internal Req Sense. */
	-extern TW_INT32 tw_cli_send_scsi_cmd(struct tw_cli_req_context *req,
	- TW_INT32 cmd);
	-
	-/* Get an AEN from the firmware (by sending down a Req Sense). */
	-extern TW_INT32 tw_cli_get_aen(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Fill in the scatter/gather list. */
	-extern TW_VOID tw_cli_fill_sg_list(struct tw_cli_ctlr_context *ctlr,
	- TW_VOID sgl_src, TW_VOID sgl_dest, TW_INT32 num_sgl_entries);
	-
	-/* Functions in tw_cl_intr.c */
	-
	-/* Process a host interrupt. */
	-extern TW_VOID tw_cli_process_host_intr(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Process an attention interrupt. */
	-extern TW_VOID tw_cli_process_attn_intr(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Process a command interrupt. */
	-extern TW_VOID tw_cli_process_cmd_intr(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Process a response interrupt from the controller. */
	-extern TW_INT32 tw_cli_process_resp_intr(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Submit any requests in the pending queue to the firmware. */
	-extern TW_INT32 tw_cli_submit_pending_queue(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Process all requests in the complete queue. */
	-extern TW_VOID tw_cli_process_complete_queue(struct tw_cli_ctlr_context *ctlr);
	-
	-/* CL internal callback for SCSI/fw passthru requests. */
	-extern TW_VOID tw_cli_complete_io(struct tw_cli_req_context *req);
	-
	-/* Completion routine for SCSI requests. */
	-extern TW_VOID tw_cli_scsi_complete(struct tw_cli_req_context *req);
	-
	-/* Callback for get/set param requests. */
	-extern TW_VOID tw_cli_param_callback(struct tw_cli_req_context *req);
	-
	-/* Callback for Req Sense commands to get AEN's. */
	-extern TW_VOID tw_cli_aen_callback(struct tw_cli_req_context *req);
	-
	-/* Decide what to do with a retrieved AEN. */
	-extern TW_UINT16 tw_cli_manage_aen(struct tw_cli_ctlr_context *ctlr,
	- struct tw_cli_req_context *req);
	-
	-/* Enable controller interrupts. */
	-extern TW_VOID
	- tw_cli_enable_interrupts(struct tw_cli_ctlr_context *ctlr_handle);
	-
	-/* Disable controller interrupts. */
	-extern TW_VOID
	- tw_cli_disable_interrupts(struct tw_cli_ctlr_context *ctlr_handle);
	-
	-/* Functions in tw_cl_misc.c */
	-
	-/* Print if dbg_level is appropriate (by calling OS Layer). */
	-extern TW_VOID tw_cli_dbg_printf(TW_UINT8 dbg_level,
	- struct tw_cl_ctlr_handle ctlr_handle, const TW_INT8 cur_func,
	- TW_INT8 *fmt, ...);
	-
	-/* Describe meaning of each set bit in the given register. */
	-extern TW_INT8 tw_cli_describe_bits(TW_UINT32 reg, TW_INT8 str);
	-
	-/* Complete all requests in the complete queue with a RESET status. */
	-extern TW_VOID tw_cli_drain_complete_queue(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Complete all requests in the busy queue with a RESET status. */
	-extern TW_VOID tw_cli_drain_busy_queue(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Complete all requests in the pending queue with a RESET status. */
	-extern TW_VOID tw_cli_drain_pending_queue(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Drain the controller response queue. */
	-extern TW_INT32 tw_cli_drain_response_queue(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Find a particular response in the controller response queue. */
	-extern TW_INT32 tw_cli_find_response(struct tw_cli_ctlr_context *ctlr,
	- TW_INT32 req_id);
	-
	-/* Drain the controller AEN queue. */
	-extern TW_INT32 tw_cli_drain_aen_queue(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Determine if a given AEN has been posted by the firmware. */
	-extern TW_INT32 tw_cli_find_aen(struct tw_cli_ctlr_context *ctlr,
	- TW_UINT16 aen_code);
	-
	-/* Poll for a given status to show up in the firmware status register. */
	-extern TW_INT32 tw_cli_poll_status(struct tw_cli_ctlr_context *ctlr,
	- TW_UINT32 status, TW_UINT32 timeout);
	-
	-/* Get a free CL internal request context packet. */
	-extern struct tw_cli_req_context *
	- tw_cli_get_request(struct tw_cli_ctlr_context *ctlr
	- );
	-
	-/* Notify OSL of controller info (fw/BIOS versions, etc.). */
	-extern TW_VOID tw_cli_notify_ctlr_info(struct tw_cli_ctlr_context *ctlr);
	-
	-/* Make sure that the firmware status register reports a proper status. */
	-extern TW_INT32 tw_cli_check_ctlr_state(struct tw_cli_ctlr_context *ctlr,
	- TW_UINT32 status_reg);
	-
	-#endif /* TW_CL_EXTERNS_H */
	Index: sys/dev/twa/tw_cl_fwif.h
	===================================================================
	--- sys/dev/twa/tw_cl_fwif.h
	+++ /dev/null
	@@ -1,418 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_CL_FWIF_H
	-
	-#define TW_CL_FWIF_H
	-
	-/*
	- * Macros and data structures for interfacing with the firmware.
	- */
	-
	-/* Register offsets from base address. */
	-#define TWA_CONTROL_REGISTER_OFFSET 0x0
	-#define TWA_STATUS_REGISTER_OFFSET 0x4
	-#define TWA_COMMAND_QUEUE_OFFSET 0x8
	-#define TWA_RESPONSE_QUEUE_OFFSET 0xC
	-#define TWA_COMMAND_QUEUE_OFFSET_LOW 0x20
	-#define TWA_COMMAND_QUEUE_OFFSET_HIGH 0x24
	-#define TWA_LARGE_RESPONSE_QUEUE_OFFSET 0x30
	-
	-/* Control register bit definitions. */
	-#define TWA_CONTROL_ISSUE_HOST_INTERRUPT 0x00000020
	-#define TWA_CONTROL_DISABLE_INTERRUPTS 0x00000040
	-#define TWA_CONTROL_ENABLE_INTERRUPTS 0x00000080
	-#define TWA_CONTROL_ISSUE_SOFT_RESET 0x00000100
	-#define TWA_CONTROL_UNMASK_RESPONSE_INTERRUPT 0x00004000
	-#define TWA_CONTROL_UNMASK_COMMAND_INTERRUPT 0x00008000
	-#define TWA_CONTROL_MASK_RESPONSE_INTERRUPT 0x00010000
	-#define TWA_CONTROL_MASK_COMMAND_INTERRUPT 0x00020000
	-#define TWA_CONTROL_CLEAR_ATTENTION_INTERRUPT 0x00040000
	-#define TWA_CONTROL_CLEAR_HOST_INTERRUPT 0x00080000
	-#define TWA_CONTROL_CLEAR_PCI_ABORT 0x00100000
	-#define TWA_CONTROL_CLEAR_QUEUE_ERROR 0x00400000
	-#define TWA_CONTROL_CLEAR_PARITY_ERROR 0x00800000
	-
	-/* Status register bit definitions. */
	-#define TWA_STATUS_ROM_BIOS_IN_SBUF 0x00000002
	-#define TWA_STATUS_COMMAND_QUEUE_EMPTY 0x00001000
	-#define TWA_STATUS_MICROCONTROLLER_READY 0x00002000
	-#define TWA_STATUS_RESPONSE_QUEUE_EMPTY 0x00004000
	-#define TWA_STATUS_COMMAND_QUEUE_FULL 0x00008000
	-#define TWA_STATUS_RESPONSE_INTERRUPT 0x00010000
	-#define TWA_STATUS_COMMAND_INTERRUPT 0x00020000
	-#define TWA_STATUS_ATTENTION_INTERRUPT 0x00040000
	-#define TWA_STATUS_HOST_INTERRUPT 0x00080000
	-#define TWA_STATUS_PCI_ABORT_INTERRUPT 0x00100000
	-#define TWA_STATUS_MICROCONTROLLER_ERROR 0x00200000
	-#define TWA_STATUS_QUEUE_ERROR_INTERRUPT 0x00400000
	-#define TWA_STATUS_PCI_PARITY_ERROR_INTERRUPT 0x00800000
	-#define TWA_STATUS_MINOR_VERSION_MASK 0x0F000000
	-#define TWA_STATUS_MAJOR_VERSION_MASK 0xF0000000
	-
	-#define TWA_STATUS_UNEXPECTED_BITS 0x00D00000
	-
	-/* PCI related defines. */
	-#define TWA_IO_CONFIG_REG 0x10
	-
	-#define TWA_PCI_CONFIG_CLEAR_PARITY_ERROR 0xc100
	-#define TWA_PCI_CONFIG_CLEAR_PCI_ABORT 0x2000
	-
	-#define TWA_RESET_PHASE1_NOTIFICATION_RESPONSE 0xFFFF
	-#define TWA_RESET_PHASE1_WAIT_TIME_MS 500
	-
	-/* Command packet opcodes. */
	-#define TWA_FW_CMD_NOP 0x00
	-#define TWA_FW_CMD_INIT_CONNECTION 0x01
	-#define TWA_FW_CMD_READ 0x02
	-#define TWA_FW_CMD_WRITE 0x03
	-#define TWA_FW_CMD_READVERIFY 0x04
	-#define TWA_FW_CMD_VERIFY 0x05
	-#define TWA_FW_CMD_ZEROUNIT 0x08
	-#define TWA_FW_CMD_REPLACEUNIT 0x09
	-#define TWA_FW_CMD_HOTSWAP 0x0A
	-#define TWA_FW_CMD_SELFTESTS 0x0B
	-#define TWA_FW_CMD_SYNC_PARAM 0x0C
	-#define TWA_FW_CMD_REORDER_UNITS 0x0D
	-
	-#define TWA_FW_CMD_EXECUTE_SCSI 0x10
	-#define TWA_FW_CMD_ATA_PASSTHROUGH 0x11
	-#define TWA_FW_CMD_GET_PARAM 0x12
	-#define TWA_FW_CMD_SET_PARAM 0x13
	-#define TWA_FW_CMD_CREATEUNIT 0x14
	-#define TWA_FW_CMD_DELETEUNIT 0x15
	-#define TWA_FW_CMD_DOWNLOAD_FIRMWARE 0x16
	-#define TWA_FW_CMD_REBUILDUNIT 0x17
	-#define TWA_FW_CMD_POWER_MANAGEMENT 0x18
	-
	-#define TWA_FW_CMD_REMOTE_PRINT 0x1B
	-#define TWA_FW_CMD_HARD_RESET_FIRMWARE 0x1C
	-#define TWA_FW_CMD_DEBUG 0x1D
	-
	-#define TWA_FW_CMD_DIAGNOSTICS 0x1F
	-
	-/* Misc defines. */
	-#define TWA_SHUTDOWN_MESSAGE_CREDITS 0x001
	-#define TWA_64BIT_SG_ADDRESSES 0x00000001
	-#define TWA_EXTENDED_INIT_CONNECT 0x00000002
	-#define TWA_BASE_MODE 1
	-#define TWA_BASE_FW_SRL 24
	-#define TWA_BASE_FW_BRANCH 0
	-#define TWA_BASE_FW_BUILD 1
	-#define TWA_CURRENT_FW_SRL 41
	-#define TWA_CURRENT_FW_BRANCH_9K 4
	-#define TWA_CURRENT_FW_BUILD_9K 8
	-#define TWA_CURRENT_FW_BRANCH_9K_X 8
	-#define TWA_CURRENT_FW_BUILD_9K_X 4
	-#define TWA_MULTI_LUN_FW_SRL 28
	-#define TWA_ARCH_ID_9K 0x5 /* 9000 PCI controllers */
	-#define TWA_ARCH_ID_9K_X 0x6 /* 9000 PCI-X controllers */
	-#define TWA_CTLR_FW_SAME_OR_NEWER 0x00000001
	-#define TWA_CTLR_FW_COMPATIBLE 0x00000002
	-#define TWA_SENSE_DATA_LENGTH 18
	-
	-#define TWA_ARCH_ID(device_id) \
	- (((device_id) == TW_CL_DEVICE_ID_9K) ? TWA_ARCH_ID_9K : \
	- TWA_ARCH_ID_9K_X)
	-#define TWA_CURRENT_FW_BRANCH(arch_id) \
	- (((arch_id) == TWA_ARCH_ID_9K) ? TWA_CURRENT_FW_BRANCH_9K : \
	- TWA_CURRENT_FW_BRANCH_9K_X)
	-#define TWA_CURRENT_FW_BUILD(arch_id) \
	- (((arch_id) == TWA_ARCH_ID_9K) ? TWA_CURRENT_FW_BUILD_9K : \
	- TWA_CURRENT_FW_BUILD_9K_X)
	-
	-/*
	- * All SG addresses and DMA'able memory allocated by the OSL should be
	- * TWA_ALIGNMENT bytes aligned, and have a size that is a multiple of
	- * TWA_SG_ELEMENT_SIZE_FACTOR.
	- */
	-#define TWA_ALIGNMENT(device_id) 0x4
	-#define TWA_SG_ELEMENT_SIZE_FACTOR(device_id) \
	- (((device_id) == TW_CL_DEVICE_ID_9K) ? 512 : 4)
	-
	-/*
	- * Some errors of interest (in cmd_hdr->status_block.error) when a command
	- * is completed by the firmware with a bad status.
	- */
	-#define TWA_ERROR_LOGICAL_UNIT_NOT_SUPPORTED 0x010a
	-#define TWA_ERROR_UNIT_OFFLINE 0x0128
	-#define TWA_ERROR_MORE_DATA 0x0231
	-
	-/* AEN codes of interest. */
	-#define TWA_AEN_QUEUE_EMPTY 0x00
	-#define TWA_AEN_SOFT_RESET 0x01
	-#define TWA_AEN_SYNC_TIME_WITH_HOST 0x31
	-
	-/* Table #'s and id's of parameters of interest in firmware's param table. */
	-#define TWA_PARAM_VERSION_TABLE 0x0402
	-#define TWA_PARAM_VERSION_FW 3 /* firmware version [16] */
	-#define TWA_PARAM_VERSION_BIOS 4 /* BIOSs version [16] */
	-#define TWA_PARAM_CTLR_MODEL 8 /* Controller model [16] */
	-
	-#define TWA_PARAM_CONTROLLER_TABLE 0x0403
	-#define TWA_PARAM_CONTROLLER_PORT_COUNT 3 /* number of ports [1] */
	-
	-#define TWA_PARAM_TIME_TABLE 0x40A
	-#define TWA_PARAM_TIME_SCHED_TIME 0x3
	-
	-#define TWA_9K_PARAM_DESCRIPTOR 0x8000
	-
	-#pragma pack(1)
	-/* 7000 structures. */
	-struct tw_cl_command_init_connect {
	- TW_UINT8 res1__opcode; /* 3:5 */
	- TW_UINT8 size;
	- TW_UINT8 request_id;
	- TW_UINT8 res2;
	- TW_UINT8 status;
	- TW_UINT8 flags;
	- TW_UINT16 message_credits;
	- TW_UINT32 features;
	- TW_UINT16 fw_srl;
	- TW_UINT16 fw_arch_id;
	- TW_UINT16 fw_branch;
	- TW_UINT16 fw_build;
	- TW_UINT32 result;
	-};
	-
	-/* Structure for downloading firmware onto the controller. */
	-struct tw_cl_command_download_firmware {
	- TW_UINT8 sgl_off__opcode;/* 3:5 */
	- TW_UINT8 size;
	- TW_UINT8 request_id;
	- TW_UINT8 unit;
	- TW_UINT8 status;
	- TW_UINT8 flags;
	- TW_UINT16 param;
	- TW_UINT8 sgl[1];
	-};
	-
	-/* Structure for hard resetting the controller. */
	-struct tw_cl_command_reset_firmware {
	- TW_UINT8 res1__opcode; /* 3:5 */
	- TW_UINT8 size;
	- TW_UINT8 request_id;
	- TW_UINT8 unit;
	- TW_UINT8 status;
	- TW_UINT8 flags;
	- TW_UINT8 res2;
	- TW_UINT8 param;
	-};
	-
	-/* Structure for sending get/set param commands. */
	-struct tw_cl_command_param {
	- TW_UINT8 sgl_off__opcode;/* 3:5 */
	- TW_UINT8 size;
	- TW_UINT8 request_id;
	- TW_UINT8 host_id__unit; /* 4:4 */
	- TW_UINT8 status;
	- TW_UINT8 flags;
	- TW_UINT16 param_count;
	- TW_UINT8 sgl[1];
	-};
	-
	-/* Generic command packet. */
	-struct tw_cl_command_generic {
	- TW_UINT8 sgl_off__opcode;/* 3:5 */
	- TW_UINT8 size;
	- TW_UINT8 request_id;
	- TW_UINT8 host_id__unit; /* 4:4 */
	- TW_UINT8 status;
	- TW_UINT8 flags;
	- TW_UINT16 count; /* block cnt, parameter cnt, message credits */
	-};
	-
	-/* Command packet header. */
	-struct tw_cl_command_header {
	- TW_UINT8 sense_data[TWA_SENSE_DATA_LENGTH];
	- struct {
	- TW_INT8 reserved[4];
	- TW_UINT16 error;
	- TW_UINT8 padding;
	- TW_UINT8 res__severity; /* 5:3 */
	- } status_block;
	- TW_UINT8 err_specific_desc[98];
	- struct {
	- TW_UINT8 size_header;
	- TW_UINT16 reserved;
	- TW_UINT8 size_sense;
	- } header_desc;
	-};
	-
	-/* 7000 Command packet. */
	-union tw_cl_command_7k {
	- struct tw_cl_command_init_connect init_connect;
	- struct tw_cl_command_download_firmware download_fw;
	- struct tw_cl_command_reset_firmware reset_fw;
	- struct tw_cl_command_param param;
	- struct tw_cl_command_generic generic;
	- TW_UINT8 padding[1024 - sizeof(struct tw_cl_command_header)];
	-};
	-
	-/* 9000 Command Packet. */
	-struct tw_cl_command_9k {
	- TW_UINT8 res__opcode; /* 3:5 */
	- TW_UINT8 unit;
	- TW_UINT16 lun_l4__req_id; /* 4:12 */
	- TW_UINT8 status;
	- TW_UINT8 sgl_offset; /* offset (in bytes) to sg_list, from the
	- end of sgl_entries */
	- TW_UINT16 lun_h4__sgl_entries;
	- TW_UINT8 cdb[16];
	- TW_UINT8 sg_list[872];/* total struct size =
	- 1024-sizeof(cmd_hdr) */
	-};
	-
	-/* Full command packet. */
	-struct tw_cl_command_packet {
	- struct tw_cl_command_header cmd_hdr;
	- union {
	- union tw_cl_command_7k cmd_pkt_7k;
	- struct tw_cl_command_9k cmd_pkt_9k;
	- } command;
	-};
	-
	-/* Structure describing payload for get/set param commands. */
	-struct tw_cl_param_9k {
	- TW_UINT16 table_id;
	- TW_UINT8 parameter_id;
	- TW_UINT8 reserved;
	- TW_UINT16 parameter_size_bytes;
	- TW_UINT16 parameter_actual_size_bytes;
	- TW_UINT8 data[1];
	-};
	-#pragma pack()
	-
	-/* Functions to read from, and write to registers */
	-#define TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle, value) \
	- tw_osl_write_reg(ctlr_handle, TWA_CONTROL_REGISTER_OFFSET, value, 4)
	-
	-#define TW_CLI_READ_STATUS_REGISTER(ctlr_handle) \
	- tw_osl_read_reg(ctlr_handle, TWA_STATUS_REGISTER_OFFSET, 4)
	-
	-#define TW_CLI_WRITE_COMMAND_QUEUE(ctlr_handle, value) do { \
	- if (ctlr->flags & TW_CL_64BIT_ADDRESSES) { \
	- /* First write the low 4 bytes, then the high 4. */ \
	- tw_osl_write_reg(ctlr_handle, TWA_COMMAND_QUEUE_OFFSET_LOW, \
	- (TW_UINT32)(value), 4); \
	- tw_osl_write_reg(ctlr_handle, TWA_COMMAND_QUEUE_OFFSET_HIGH,\
	- (TW_UINT32)(((TW_UINT64)value)>>32), 4); \
	- } else \
	- tw_osl_write_reg(ctlr_handle, TWA_COMMAND_QUEUE_OFFSET, \
	- (TW_UINT32)(value), 4); \
	-} while (0)
	-
	-#define TW_CLI_READ_RESPONSE_QUEUE(ctlr_handle) \
	- tw_osl_read_reg(ctlr_handle, TWA_RESPONSE_QUEUE_OFFSET, 4)
	-
	-#define TW_CLI_READ_LARGE_RESPONSE_QUEUE(ctlr_handle) \
	- tw_osl_read_reg(ctlr_handle, TWA_LARGE_RESPONSE_QUEUE_OFFSET, 4)
	-
	-#define TW_CLI_SOFT_RESET(ctlr) \
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr, \
	- TWA_CONTROL_ISSUE_SOFT_RESET \| \
	- TWA_CONTROL_CLEAR_HOST_INTERRUPT \| \
	- TWA_CONTROL_CLEAR_ATTENTION_INTERRUPT \| \
	- TWA_CONTROL_MASK_COMMAND_INTERRUPT \| \
	- TWA_CONTROL_MASK_RESPONSE_INTERRUPT \| \
	- TWA_CONTROL_DISABLE_INTERRUPTS)
	-
	-/* Detect inconsistencies in the status register. */
	-#define TW_CLI_STATUS_ERRORS(x) \
	- ((x & TWA_STATUS_UNEXPECTED_BITS) && \
	- (x & TWA_STATUS_MICROCONTROLLER_READY))
	-
	-/*
	- * Functions for making transparent, the bit fields in firmware
	- * interface structures.
	- */
	-#define BUILD_SGL_OFF__OPCODE(sgl_off, opcode) \
	- ((sgl_off << 5) & 0xE0) \| (opcode & 0x1F) /* 3:5 */
	-
	-#define BUILD_RES__OPCODE(res, opcode) \
	- ((res << 5) & 0xE0) \| (opcode & 0x1F) /* 3:5 */
	-
	-#define BUILD_HOST_ID__UNIT(host_id, unit) \
	- ((host_id << 4) & 0xF0) \| (unit & 0xF) /* 4:4 */
	-
	-#define BUILD_RES__SEVERITY(res, severity) \
	- ((res << 3) & 0xF8) \| (severity & 0x7) /* 5:3 */
	-
	-#define BUILD_LUN_L4__REQ_ID(lun, req_id) \
	- (((lun << 12) & 0xF000) \| (req_id & 0xFFF)) /* 4:12 */
	-
	-#define BUILD_LUN_H4__SGL_ENTRIES(lun, sgl_entries) \
	- (((lun << 8) & 0xF000) \| (sgl_entries & 0xFFF)) /* 4:12 */
	-
	-#define GET_OPCODE(sgl_off__opcode) \
	- (sgl_off__opcode & 0x1F) /* 3:5 */
	-
	-#define GET_SGL_OFF(sgl_off__opcode) \
	- ((sgl_off__opcode >> 5) & 0x7) /* 3:5 */
	-
	-#define GET_UNIT(host_id__unit) \
	- (host_id__unit & 0xF) /* 4:4 */
	-
	-#define GET_HOST_ID(host_id__unit) \
	- ((host_id__unit >> 4) & 0xF) /* 4:4 */
	-
	-#define GET_SEVERITY(res__severity) \
	- (res__severity & 0x7) /* 5:3 */
	-
	-#define GET_RESP_ID(undef2__resp_id__undef1) \
	- ((undef2__resp_id__undef1 >> 4) & 0xFF) /* 20:8:4 */
	-
	-#define GET_RESP_ID_9K_X(undef2__resp_id) \
	- ((undef2__resp_id) & 0xFFF) /* 20:12 */
	-
	-#define GET_LARGE_RESP_ID(misc__large_resp_id) \
	- ((misc__large_resp_id) & 0xFFFF) /* 16:16 */
	-
	-#define GET_REQ_ID(lun_l4__req_id) \
	- (lun_l4__req_id & 0xFFF) /* 4:12 */
	-
	-#define GET_LUN_L4(lun_l4__req_id) \
	- ((lun_l4__req_id >> 12) & 0xF) /* 4:12 */
	-
	-#define GET_SGL_ENTRIES(lun_h4__sgl_entries) \
	- (lun_h4__sgl_entries & 0xFFF) /* 4:12 */
	-
	-#define GET_LUN_H4(lun_h4__sgl_entries) \
	- ((lun_h4__sgl_entries >> 12) & 0xF) /* 4:12 */
	-
	-#endif /* TW_CL_FWIF_H */
	Index: sys/dev/twa/tw_cl_init.c
	===================================================================
	--- sys/dev/twa/tw_cl_init.c
	+++ /dev/null
	@@ -1,696 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- * Modifications by: Manjunath Ranganathaiah
	- */
	-
	-/*
	- * Common Layer initialization functions.
	- */
	-
	-#include "tw_osl_share.h"
	-#include "tw_cl_share.h"
	-#include "tw_cl_fwif.h"
	-#include "tw_cl_ioctl.h"
	-#include "tw_cl.h"
	-#include "tw_cl_externs.h"
	-#include "tw_osl_ioctl.h"
	-
	-/*
	- * Function name: tw_cl_ctlr_supported
	- * Description: Determines if a controller is supported.
	- *
	- * Input: vendor_id -- vendor id of the controller
	- * device_id -- device id of the controller
	- * Output: None
	- * Return value: TW_CL_TRUE-- controller supported
	- * TW_CL_FALSE-- controller not supported
	- */
	-TW_INT32
	-tw_cl_ctlr_supported(TW_INT32 vendor_id, TW_INT32 device_id)
	-{
	- if ((vendor_id == TW_CL_VENDOR_ID) &&
	- ((device_id == TW_CL_DEVICE_ID_9K) \|\|
	- (device_id == TW_CL_DEVICE_ID_9K_X) \|\|
	- (device_id == TW_CL_DEVICE_ID_9K_E) \|\|
	- (device_id == TW_CL_DEVICE_ID_9K_SA)))
	- return(TW_CL_TRUE);
	- return(TW_CL_FALSE);
	-}
	-
	-/*
	- * Function name: tw_cl_get_pci_bar_info
	- * Description: Returns PCI BAR info.
	- *
	- * Input: device_id -- device id of the controller
	- * bar_type -- type of PCI BAR in question
	- * Output: bar_num -- PCI BAR number corresponding to bar_type
	- * bar0_offset -- byte offset from BAR 0 (0x10 in
	- * PCI config space)
	- * bar_size -- size, in bytes, of the BAR in question
	- * Return value: 0 -- success
	- * non-zero -- failure
	- */
	-TW_INT32
	-tw_cl_get_pci_bar_info(TW_INT32 device_id, TW_INT32 bar_type,
	- TW_INT32 bar_num, TW_INT32 bar0_offset, TW_INT32 *bar_size)
	-{
	- TW_INT32 error = TW_OSL_ESUCCESS;
	-
	- switch(device_id) {
	- case TW_CL_DEVICE_ID_9K:
	- switch(bar_type) {
	- case TW_CL_BAR_TYPE_IO:
	- *bar_num = 0;
	- *bar0_offset = 0;
	- *bar_size = 4;
	- break;
	-
	- case TW_CL_BAR_TYPE_MEM:
	- *bar_num = 1;
	- *bar0_offset = 0x4;
	- *bar_size = 8;
	- break;
	-
	- case TW_CL_BAR_TYPE_SBUF:
	- *bar_num = 2;
	- *bar0_offset = 0xC;
	- *bar_size = 8;
	- break;
	- }
	- break;
	-
	- case TW_CL_DEVICE_ID_9K_X:
	- case TW_CL_DEVICE_ID_9K_E:
	- case TW_CL_DEVICE_ID_9K_SA:
	- switch(bar_type) {
	- case TW_CL_BAR_TYPE_IO:
	- *bar_num = 2;
	- *bar0_offset = 0x10;
	- *bar_size = 4;
	- break;
	-
	- case TW_CL_BAR_TYPE_MEM:
	- *bar_num = 1;
	- *bar0_offset = 0x8;
	- *bar_size = 8;
	- break;
	-
	- case TW_CL_BAR_TYPE_SBUF:
	- *bar_num = 0;
	- *bar0_offset = 0;
	- *bar_size = 8;
	- break;
	- }
	- break;
	-
	- default:
	- error = TW_OSL_ENOTTY;
	- break;
	- }
	-
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cl_get_mem_requirements
	- * Description: Provides info about Common Layer requirements for a
	- * controller, given the controller type (in 'flags').
	- * Input: ctlr_handle -- controller handle
	- * flags -- more info passed by the OS Layer
	- * device_id -- device id of the controller
	- * max_simult_reqs -- maximum # of simultaneous
	- * requests that the OS Layer expects
	- * the Common Layer to support
	- * max_aens -- maximun # of AEN's needed to be supported
	- * Output: alignment -- alignment needed for all DMA'able
	- * buffers
	- * sg_size_factor -- every SG element should have a size
	- * that's a multiple of this number
	- * non_dma_mem_size -- # of bytes of memory needed for
	- * non-DMA purposes
	- * dma_mem_size -- # of bytes of DMA'able memory needed
	- * per_req_dma_mem_size -- # of bytes of DMA'able memory
	- * needed per request, if applicable
	- * per_req_non_dma_mem_size -- # of bytes of memory needed
	- * per request for non-DMA purposes,
	- * if applicable
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cl_get_mem_requirements(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_UINT32 flags, TW_INT32 device_id, TW_INT32 max_simult_reqs,
	- TW_INT32 max_aens, TW_UINT32 alignment, TW_UINT32 sg_size_factor,
	- TW_UINT32 non_dma_mem_size, TW_UINT32 dma_mem_size
	- )
	-{
	- if (device_id == 0)
	- device_id = TW_CL_DEVICE_ID_9K;
	-
	- if (max_simult_reqs > TW_CL_MAX_SIMULTANEOUS_REQUESTS) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1000, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Too many simultaneous requests to support!",
	- "requested = %d, supported = %d, error = %d\n",
	- max_simult_reqs, TW_CL_MAX_SIMULTANEOUS_REQUESTS,
	- TW_OSL_EBIG);
	- return(TW_OSL_EBIG);
	- }
	-
	- *alignment = TWA_ALIGNMENT(device_id);
	- *sg_size_factor = TWA_SG_ELEMENT_SIZE_FACTOR(device_id);
	-
	- /*
	- * Total non-DMA memory needed is the sum total of memory needed for
	- * the controller context, request packets (including the 1 needed for
	- * CL internal requests), and event packets.
	- */
	-
	- *non_dma_mem_size = sizeof(struct tw_cli_ctlr_context) +
	- (sizeof(struct tw_cli_req_context) * max_simult_reqs) +
	- (sizeof(struct tw_cl_event_packet) * max_aens);
	-
	- /*
	- * Total DMA'able memory needed is the sum total of memory needed for
	- * all command packets (including the 1 needed for CL internal
	- * requests), and memory needed to hold the payload for internal
	- * requests.
	- */
	-
	- dma_mem_size = (sizeof(struct tw_cl_command_packet)
	- (max_simult_reqs)) + (TW_CLI_SECTOR_SIZE);
	-
	- return(0);
	-}
	-
	-/*
	- * Function name: tw_cl_init_ctlr
	- * Description: Initializes driver data structures for the controller.
	- *
	- * Input: ctlr_handle -- controller handle
	- * flags -- more info passed by the OS Layer
	- * device_id -- device id of the controller
	- * max_simult_reqs -- maximum # of simultaneous requests
	- * that the OS Layer expects the Common
	- * Layer to support
	- * max_aens -- maximun # of AEN's needed to be supported
	- * non_dma_mem -- ptr to allocated non-DMA memory
	- * dma_mem -- ptr to allocated DMA'able memory
	- * dma_mem_phys -- physical address of dma_mem
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cl_init_ctlr(struct tw_cl_ctlr_handle *ctlr_handle, TW_UINT32 flags,
	- TW_INT32 device_id, TW_INT32 max_simult_reqs, TW_INT32 max_aens,
	- TW_VOID non_dma_mem, TW_VOID dma_mem, TW_UINT64 dma_mem_phys
	- )
	-{
	- struct tw_cli_ctlr_context *ctlr;
	- struct tw_cli_req_context *req;
	- TW_UINT8 *free_non_dma_mem;
	- TW_INT32 error = TW_OSL_ESUCCESS;
	- TW_INT32 i;
	-
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- if (flags & TW_CL_START_CTLR_ONLY) {
	- ctlr = (struct tw_cli_ctlr_context *)
	- (ctlr_handle->cl_ctlr_ctxt);
	- goto start_ctlr;
	- }
	-
	- if (max_simult_reqs > TW_CL_MAX_SIMULTANEOUS_REQUESTS) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1000, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Too many simultaneous requests to support!",
	- "requested = %d, supported = %d, error = %d\n",
	- max_simult_reqs, TW_CL_MAX_SIMULTANEOUS_REQUESTS,
	- TW_OSL_EBIG);
	- return(TW_OSL_EBIG);
	- }
	-
	- if ((non_dma_mem == TW_CL_NULL) \|\| (dma_mem == TW_CL_NULL)
	- ) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1001, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Insufficient memory for Common Layer's internal usage",
	- "error = %d\n", TW_OSL_ENOMEM);
	- return(TW_OSL_ENOMEM);
	- }
	-
	- tw_osl_memzero(non_dma_mem, sizeof(struct tw_cli_ctlr_context) +
	- (sizeof(struct tw_cli_req_context) * max_simult_reqs) +
	- (sizeof(struct tw_cl_event_packet) * max_aens));
	-
	- tw_osl_memzero(dma_mem,
	- (sizeof(struct tw_cl_command_packet) *
	- max_simult_reqs) +
	- TW_CLI_SECTOR_SIZE);
	-
	- free_non_dma_mem = (TW_UINT8 *)non_dma_mem;
	-
	- ctlr = (struct tw_cli_ctlr_context *)free_non_dma_mem;
	- free_non_dma_mem += sizeof(struct tw_cli_ctlr_context);
	-
	- ctlr_handle->cl_ctlr_ctxt = ctlr;
	- ctlr->ctlr_handle = ctlr_handle;
	-
	- ctlr->device_id = (TW_UINT32)device_id;
	- ctlr->arch_id = TWA_ARCH_ID(device_id);
	- ctlr->flags = flags;
	- ctlr->sg_size_factor = TWA_SG_ELEMENT_SIZE_FACTOR(device_id);
	- ctlr->max_simult_reqs = max_simult_reqs;
	- ctlr->max_aens_supported = max_aens;
	-
	- /* Initialize queues of CL internal request context packets. */
	- tw_cli_req_q_init(ctlr, TW_CLI_FREE_Q);
	- tw_cli_req_q_init(ctlr, TW_CLI_BUSY_Q);
	- tw_cli_req_q_init(ctlr, TW_CLI_PENDING_Q);
	- tw_cli_req_q_init(ctlr, TW_CLI_COMPLETE_Q);
	- tw_cli_req_q_init(ctlr, TW_CLI_RESET_Q);
	-
	- /* Initialize all locks used by CL. */
	- ctlr->gen_lock = &(ctlr->gen_lock_handle);
	- tw_osl_init_lock(ctlr_handle, "tw_cl_gen_lock", ctlr->gen_lock);
	- ctlr->io_lock = &(ctlr->io_lock_handle);
	- tw_osl_init_lock(ctlr_handle, "tw_cl_io_lock", ctlr->io_lock);
	-
	- /* Initialize CL internal request context packets. */
	- ctlr->req_ctxt_buf = (struct tw_cli_req_context *)free_non_dma_mem;
	- free_non_dma_mem += (sizeof(struct tw_cli_req_context) *
	- max_simult_reqs);
	-
	- ctlr->cmd_pkt_buf = (struct tw_cl_command_packet *)dma_mem;
	- ctlr->cmd_pkt_phys = dma_mem_phys;
	-
	- ctlr->internal_req_data = (TW_UINT8 *)
	- (ctlr->cmd_pkt_buf +
	- max_simult_reqs);
	- ctlr->internal_req_data_phys = ctlr->cmd_pkt_phys +
	- (sizeof(struct tw_cl_command_packet) *
	- max_simult_reqs);
	-
	- for (i = 0; i < max_simult_reqs; i++) {
	- req = &(ctlr->req_ctxt_buf[i]);
	-
	- req->cmd_pkt = &(ctlr->cmd_pkt_buf[i]);
	- req->cmd_pkt_phys = ctlr->cmd_pkt_phys +
	- (i * sizeof(struct tw_cl_command_packet));
	-
	- req->request_id = i;
	- req->ctlr = ctlr;
	-
	- /* Insert request into the free queue. */
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- }
	-
	- /* Initialize the AEN queue. */
	- ctlr->aen_queue = (struct tw_cl_event_packet *)free_non_dma_mem;
	-
	-start_ctlr:
	- /*
	- * Disable interrupts. Interrupts will be enabled in tw_cli_start_ctlr
	- * (only) if initialization succeeded.
	- */
	- tw_cli_disable_interrupts(ctlr);
	-
	- /* Initialize the controller. */
	- if ((error = tw_cli_start_ctlr(ctlr))) {
	- /* Soft reset the controller, and try one more time. */
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1002, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Controller initialization failed. Retrying...",
	- "error = %d\n", error);
	- if ((error = tw_cli_soft_reset(ctlr))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1003, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Controller soft reset failed",
	- "error = %d\n", error);
	- return(error);
	- } else if ((error = tw_cli_start_ctlr(ctlr))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1004, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Controller initialization retry failed",
	- "error = %d\n", error);
	- return(error);
	- }
	- }
	- /* Notify some info about the controller to the OSL. */
	- tw_cli_notify_ctlr_info(ctlr);
	-
	- /* Mark the controller active. */
	- ctlr->active = TW_CL_TRUE;
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_start_ctlr
	- * Description: Establishes a logical connection with the controller.
	- * Determines whether or not the driver is compatible
	- * with the firmware on the controller, before proceeding
	- * to work with it.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_start_ctlr(struct tw_cli_ctlr_context *ctlr)
	-{
	- TW_UINT16 fw_on_ctlr_srl = 0;
	- TW_UINT16 fw_on_ctlr_arch_id = 0;
	- TW_UINT16 fw_on_ctlr_branch = 0;
	- TW_UINT16 fw_on_ctlr_build = 0;
	- TW_UINT32 init_connect_result = 0;
	- TW_INT32 error = TW_OSL_ESUCCESS;
	-
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Wait for the controller to become ready. */
	- if ((error = tw_cli_poll_status(ctlr,
	- TWA_STATUS_MICROCONTROLLER_READY,
	- TW_CLI_REQUEST_TIMEOUT_PERIOD))) {
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1009, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Microcontroller not ready",
	- "error = %d", error);
	- return(error);
	- }
	- /* Drain the response queue. */
	- if ((error = tw_cli_drain_response_queue(ctlr))) {
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x100A, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Can't drain response queue",
	- "error = %d", error);
	- return(error);
	- }
	- /* Establish a logical connection with the controller. */
	- if ((error = tw_cli_init_connection(ctlr,
	- (TW_UINT16)(ctlr->max_simult_reqs),
	- TWA_EXTENDED_INIT_CONNECT, TWA_CURRENT_FW_SRL,
	- (TW_UINT16)(ctlr->arch_id),
	- TWA_CURRENT_FW_BRANCH(ctlr->arch_id),
	- TWA_CURRENT_FW_BUILD(ctlr->arch_id),
	- &fw_on_ctlr_srl, &fw_on_ctlr_arch_id,
	- &fw_on_ctlr_branch, &fw_on_ctlr_build,
	- &init_connect_result))) {
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x100B, 0x2, TW_CL_SEVERITY_WARNING_STRING,
	- "Can't initialize connection in current mode",
	- "error = %d", error);
	- return(error);
	- }
	- {
	- /* See if we can at least work with the firmware on the
	- * controller in the current mode.
	- */
	- if (init_connect_result & TWA_CTLR_FW_COMPATIBLE) {
	- /* Yes, we can. Make note of the operating mode. */
	- if (init_connect_result & TWA_CTLR_FW_SAME_OR_NEWER) {
	- ctlr->working_srl = TWA_CURRENT_FW_SRL;
	- ctlr->working_branch =
	- TWA_CURRENT_FW_BRANCH(ctlr->arch_id);
	- ctlr->working_build =
	- TWA_CURRENT_FW_BUILD(ctlr->arch_id);
	- } else {
	- ctlr->working_srl = fw_on_ctlr_srl;
	- ctlr->working_branch = fw_on_ctlr_branch;
	- ctlr->working_build = fw_on_ctlr_build;
	- }
	- } else {
	- /*
	- * No, we can't. See if we can at least work with
	- * it in the base mode.
	- */
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1010, 0x2, TW_CL_SEVERITY_WARNING_STRING,
	- "Driver/Firmware mismatch. "
	- "Negotiating for base level...",
	- " ");
	- if ((error = tw_cli_init_connection(ctlr,
	- (TW_UINT16)(ctlr->max_simult_reqs),
	- TWA_EXTENDED_INIT_CONNECT,
	- TWA_BASE_FW_SRL,
	- (TW_UINT16)(ctlr->arch_id),
	- TWA_BASE_FW_BRANCH, TWA_BASE_FW_BUILD,
	- &fw_on_ctlr_srl, &fw_on_ctlr_arch_id,
	- &fw_on_ctlr_branch, &fw_on_ctlr_build,
	- &init_connect_result))) {
	- tw_cl_create_event(ctlr->ctlr_handle,
	- TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1011, 0x1,
	- TW_CL_SEVERITY_ERROR_STRING,
	- "Can't initialize connection in "
	- "base mode",
	- " ");
	- return(error);
	- }
	- if (!(init_connect_result & TWA_CTLR_FW_COMPATIBLE)) {
	- /*
	- * The firmware on the controller is not even
	- * compatible with our base mode. We cannot
	- * work with it. Bail...
	- */
	- return(1);
	- }
	- /*
	- * We can work with this firmware, but only in
	- * base mode.
	- */
	- ctlr->working_srl = TWA_BASE_FW_SRL;
	- ctlr->working_branch = TWA_BASE_FW_BRANCH;
	- ctlr->working_build = TWA_BASE_FW_BUILD;
	- ctlr->operating_mode = TWA_BASE_MODE;
	- }
	- ctlr->fw_on_ctlr_srl = fw_on_ctlr_srl;
	- ctlr->fw_on_ctlr_branch = fw_on_ctlr_branch;
	- ctlr->fw_on_ctlr_build = fw_on_ctlr_build;
	- }
	-
	- /* Drain the AEN queue */
	- if ((error = tw_cli_drain_aen_queue(ctlr)))
	- /*
	- * We will just print that we couldn't drain the AEN queue.
	- * There's no need to bail out.
	- */
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1014, 0x2, TW_CL_SEVERITY_WARNING_STRING,
	- "Can't drain AEN queue",
	- "error = %d", error);
	-
	- /* Enable interrupts. */
	- tw_cli_enable_interrupts(ctlr);
	-
	- return(TW_OSL_ESUCCESS);
	-}
	-
	-/*
	- * Function name: tw_cl_shutdown_ctlr
	- * Description: Closes logical connection with the controller.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * flags -- more info passed by the OS Layer
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cl_shutdown_ctlr(struct tw_cl_ctlr_handle *ctlr_handle, TW_UINT32 flags)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	- TW_INT32 error;
	-
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(), "entered");
	- /*
	- * Mark the controller as inactive, disable any further interrupts,
	- * and notify the controller that we are going down.
	- */
	- ctlr->active = TW_CL_FALSE;
	-
	- tw_cli_disable_interrupts(ctlr);
	-
	- /* Let the controller know that we are going down. */
	- if ((error = tw_cli_init_connection(ctlr, TWA_SHUTDOWN_MESSAGE_CREDITS,
	- 0, 0, 0, 0, 0, TW_CL_NULL, TW_CL_NULL, TW_CL_NULL,
	- TW_CL_NULL, TW_CL_NULL)))
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1015, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Can't close connection with controller",
	- "error = %d", error);
	-
	- if (flags & TW_CL_STOP_CTLR_ONLY)
	- goto ret;
	-
	- /* Destroy all locks used by CL. */
	- tw_osl_destroy_lock(ctlr_handle, ctlr->gen_lock);
	- tw_osl_destroy_lock(ctlr_handle, ctlr->io_lock);
	-
	-ret:
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_init_connection
	- * Description: Sends init_connection cmd to firmware
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * message_credits -- max # of requests that we might send
	- * down simultaneously. This will be
	- * typically set to 256 at init-time or
	- * after a reset, and to 1 at shutdown-time
	- * set_features -- indicates if we intend to use 64-bit
	- * sg, also indicates if we want to do a
	- * basic or an extended init_connection;
	- *
	- * Note: The following input/output parameters are valid, only in case of an
	- * extended init_connection:
	- *
	- * current_fw_srl -- srl of fw we are bundled
	- * with, if any; 0 otherwise
	- * current_fw_arch_id -- arch_id of fw we are bundled
	- * with, if any; 0 otherwise
	- * current_fw_branch -- branch # of fw we are bundled
	- * with, if any; 0 otherwise
	- * current_fw_build -- build # of fw we are bundled
	- * with, if any; 0 otherwise
	- * Output: fw_on_ctlr_srl -- srl of fw on ctlr
	- * fw_on_ctlr_arch_id -- arch_id of fw on ctlr
	- * fw_on_ctlr_branch -- branch # of fw on ctlr
	- * fw_on_ctlr_build -- build # of fw on ctlr
	- * init_connect_result -- result bitmap of fw response
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_init_connection(struct tw_cli_ctlr_context *ctlr,
	- TW_UINT16 message_credits, TW_UINT32 set_features,
	- TW_UINT16 current_fw_srl, TW_UINT16 current_fw_arch_id,
	- TW_UINT16 current_fw_branch, TW_UINT16 current_fw_build,
	- TW_UINT16 fw_on_ctlr_srl, TW_UINT16 fw_on_ctlr_arch_id,
	- TW_UINT16 fw_on_ctlr_branch, TW_UINT16 fw_on_ctlr_build,
	- TW_UINT32 *init_connect_result)
	-{
	- struct tw_cli_req_context *req;
	- struct tw_cl_command_init_connect *init_connect;
	- TW_INT32 error = TW_OSL_EBUSY;
	-
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Get a request packet. */
	- if ((req = tw_cli_get_request(ctlr
	- )) == TW_CL_NULL)
	- goto out;
	-
	- req->flags \|= TW_CLI_REQ_FLAGS_INTERNAL;
	-
	- /* Build the cmd pkt. */
	- init_connect = &(req->cmd_pkt->command.cmd_pkt_7k.init_connect);
	-
	- req->cmd_pkt->cmd_hdr.header_desc.size_header = 128;
	-
	- init_connect->res1__opcode =
	- BUILD_RES__OPCODE(0, TWA_FW_CMD_INIT_CONNECTION);
	- init_connect->request_id =
	- (TW_UINT8)(TW_CL_SWAP16(req->request_id));
	- init_connect->message_credits = TW_CL_SWAP16(message_credits);
	- init_connect->features = TW_CL_SWAP32(set_features);
	- if (ctlr->flags & TW_CL_64BIT_ADDRESSES)
	- init_connect->features \|= TW_CL_SWAP32(TWA_64BIT_SG_ADDRESSES);
	- if (set_features & TWA_EXTENDED_INIT_CONNECT) {
	- /*
	- * Fill in the extra fields needed for an extended
	- * init_connect.
	- */
	- init_connect->size = 6;
	- init_connect->fw_srl = TW_CL_SWAP16(current_fw_srl);
	- init_connect->fw_arch_id = TW_CL_SWAP16(current_fw_arch_id);
	- init_connect->fw_branch = TW_CL_SWAP16(current_fw_branch);
	- init_connect->fw_build = TW_CL_SWAP16(current_fw_build);
	- } else
	- init_connect->size = 3;
	-
	- /* Submit the command, and wait for it to complete. */
	- error = tw_cli_submit_and_poll_request(req,
	- TW_CLI_REQUEST_TIMEOUT_PERIOD);
	- if (error)
	- goto out;
	- if ((error = init_connect->status)) {
	-#if 0
	- tw_cli_create_ctlr_event(ctlr,
	- TW_CL_MESSAGE_SOURCE_CONTROLLER_ERROR,
	- &(req->cmd_pkt->cmd_hdr));
	-#endif // 0
	- goto out;
	- }
	- if (set_features & TWA_EXTENDED_INIT_CONNECT) {
	- *fw_on_ctlr_srl = TW_CL_SWAP16(init_connect->fw_srl);
	- *fw_on_ctlr_arch_id = TW_CL_SWAP16(init_connect->fw_arch_id);
	- *fw_on_ctlr_branch = TW_CL_SWAP16(init_connect->fw_branch);
	- *fw_on_ctlr_build = TW_CL_SWAP16(init_connect->fw_build);
	- *init_connect_result = TW_CL_SWAP32(init_connect->result);
	- }
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- return(error);
	-
	-out:
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1016, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "init_connection failed",
	- "error = %d", error);
	- if (req)
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- return(error);
	-}
	Index: sys/dev/twa/tw_cl_intr.c
	===================================================================
	--- sys/dev/twa/tw_cl_intr.c
	+++ /dev/null
	@@ -1,724 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- * Modifications by: Manjunath Ranganathaiah
	- */
	-
	-/*
	- * Common Layer interrupt handling functions.
	- */
	-
	-#include "tw_osl_share.h"
	-#include "tw_cl_share.h"
	-#include "tw_cl_fwif.h"
	-#include "tw_cl_ioctl.h"
	-#include "tw_cl.h"
	-#include "tw_cl_externs.h"
	-#include "tw_osl_ioctl.h"
	-
	-/*
	- * Function name: twa_interrupt
	- * Description: Interrupt handler. Determines the kind of interrupt,
	- * and returns TW_CL_TRUE if it recognizes the interrupt.
	- *
	- * Input: ctlr_handle -- controller handle
	- * Output: None
	- * Return value: TW_CL_TRUE -- interrupt recognized
	- * TW_CL_FALSE-- interrupt not recognized
	- */
	-TW_INT32
	-tw_cl_interrupt(struct tw_cl_ctlr_handle *ctlr_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	- TW_UINT32 status_reg;
	- TW_INT32 rc = TW_CL_FALSE;
	-
	- tw_cli_dbg_printf(10, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* If we don't have controller context, bail */
	- if (ctlr == NULL)
	- goto out;
	-
	- /*
	- * Bail If we get an interrupt while resetting, or shutting down.
	- */
	- if (ctlr->reset_in_progress \|\| !(ctlr->active))
	- goto out;
	-
	- /* Read the status register to determine the type of interrupt. */
	- status_reg = TW_CLI_READ_STATUS_REGISTER(ctlr_handle);
	- if (tw_cli_check_ctlr_state(ctlr, status_reg))
	- goto out;
	-
	- /* Clear the interrupt. */
	- if (status_reg & TWA_STATUS_HOST_INTERRUPT) {
	- tw_cli_dbg_printf(6, ctlr_handle, tw_osl_cur_func(),
	- "Host interrupt");
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle,
	- TWA_CONTROL_CLEAR_HOST_INTERRUPT);
	- }
	- if (status_reg & TWA_STATUS_ATTENTION_INTERRUPT) {
	- tw_cli_dbg_printf(6, ctlr_handle, tw_osl_cur_func(),
	- "Attention interrupt");
	- rc \|= TW_CL_TRUE; /* request for a deferred isr call */
	- tw_cli_process_attn_intr(ctlr);
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle,
	- TWA_CONTROL_CLEAR_ATTENTION_INTERRUPT);
	- }
	- if (status_reg & TWA_STATUS_COMMAND_INTERRUPT) {
	- tw_cli_dbg_printf(6, ctlr_handle, tw_osl_cur_func(),
	- "Command interrupt");
	- rc \|= TW_CL_TRUE; /* request for a deferred isr call */
	- tw_cli_process_cmd_intr(ctlr);
	- if ((TW_CL_Q_FIRST_ITEM(&(ctlr->req_q_head[TW_CLI_PENDING_Q]))) == TW_CL_NULL)
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle,
	- TWA_CONTROL_MASK_COMMAND_INTERRUPT);
	- }
	- if (status_reg & TWA_STATUS_RESPONSE_INTERRUPT) {
	- tw_cli_dbg_printf(10, ctlr_handle, tw_osl_cur_func(),
	- "Response interrupt");
	- rc \|= TW_CL_TRUE; /* request for a deferred isr call */
	- tw_cli_process_resp_intr(ctlr);
	- }
	-out:
	- return(rc);
	-}
	-
	-/*
	- * Function name: tw_cli_process_host_intr
	- * Description: This function gets called if we triggered an interrupt.
	- * We don't use it as of now.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_process_host_intr(struct tw_cli_ctlr_context *ctlr)
	-{
	- tw_cli_dbg_printf(6, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-}
	-
	-/*
	- * Function name: tw_cli_process_attn_intr
	- * Description: This function gets called if the fw posted an AEN
	- * (Asynchronous Event Notification). It fetches
	- * all the AEN's that the fw might have posted.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_process_attn_intr(struct tw_cli_ctlr_context *ctlr)
	-{
	- TW_INT32 error;
	-
	- tw_cli_dbg_printf(6, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- if ((error = tw_cli_get_aen(ctlr))) {
	- /*
	- * If the driver is already in the process of retrieveing AEN's,
	- * we will be returned TW_OSL_EBUSY. In this case,
	- * tw_cli_param_callback or tw_cli_aen_callback will eventually
	- * retrieve the AEN this attention interrupt is for. So, we
	- * don't need to print the failure.
	- */
	- if (error != TW_OSL_EBUSY)
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1200, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Failed to fetch AEN",
	- "error = %d", error);
	- }
	-}
	-
	-/*
	- * Function name: tw_cli_process_cmd_intr
	- * Description: This function gets called if we hit a queue full
	- * condition earlier, and the fw is now ready for
	- * new cmds. Submits any pending requests.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_process_cmd_intr(struct tw_cli_ctlr_context *ctlr)
	-{
	- tw_cli_dbg_printf(6, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Start any requests that might be in the pending queue. */
	- tw_cli_submit_pending_queue(ctlr);
	-
	- /*
	- * If tw_cli_submit_pending_queue was unsuccessful due to a "cmd queue
	- * full" condition, cmd_intr will already have been unmasked by
	- * tw_cli_submit_cmd. We don't need to do it again... simply return.
	- */
	-}
	-
	-/*
	- * Function name: tw_cli_process_resp_intr
	- * Description: Looks for cmd completions from fw; queues cmds completed
	- * by fw into complete queue.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: 0 -- no ctlr error
	- * non-zero-- ctlr error
	- */
	-TW_INT32
	-tw_cli_process_resp_intr(struct tw_cli_ctlr_context *ctlr)
	-{
	- TW_UINT32 resp;
	- struct tw_cli_req_context *req;
	- TW_INT32 error;
	- TW_UINT32 status_reg;
	-
	- tw_cli_dbg_printf(10, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- for (;;) {
	- status_reg = TW_CLI_READ_STATUS_REGISTER(ctlr->ctlr_handle);
	- if ((error = tw_cli_check_ctlr_state(ctlr, status_reg)))
	- break;
	- if (status_reg & TWA_STATUS_RESPONSE_QUEUE_EMPTY) {
	- tw_cli_dbg_printf(7, ctlr->ctlr_handle,
	- tw_osl_cur_func(), "Response queue empty");
	- break;
	- }
	-
	- /* Response queue is not empty. */
	- resp = TW_CLI_READ_RESPONSE_QUEUE(ctlr->ctlr_handle);
	- {
	- req = &(ctlr->req_ctxt_buf[GET_RESP_ID(resp)]);
	- }
	-
	- if (req->state != TW_CLI_REQ_STATE_BUSY) {
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1201, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Unposted command completed!!",
	- "request = %p, status = %d",
	- req, req->state);
	-#ifdef TW_OSL_DEBUG
	- tw_cl_print_ctlr_stats(ctlr->ctlr_handle);
	-#endif /* TW_OSL_DEBUG */
	- continue;
	- }
	-
	- /*
	- * Remove the request from the busy queue, mark it as complete,
	- * and enqueue it in the complete queue.
	- */
	- tw_cli_req_q_remove_item(req, TW_CLI_BUSY_Q);
	- req->state = TW_CLI_REQ_STATE_COMPLETE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_COMPLETE_Q);
	- }
	-
	- /* Complete this, and other requests in the complete queue. */
	- tw_cli_process_complete_queue(ctlr);
	-
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_submit_pending_queue
	- * Description: Kick starts any requests in the pending queue.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: 0 -- all pending requests submitted successfully
	- * non-zero-- otherwise
	- */
	-TW_INT32
	-tw_cli_submit_pending_queue(struct tw_cli_ctlr_context *ctlr)
	-{
	- struct tw_cli_req_context *req;
	- TW_INT32 error = TW_OSL_ESUCCESS;
	-
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /*
	- * Pull requests off the pending queue, and submit them.
	- */
	- while ((req = tw_cli_req_q_remove_head(ctlr, TW_CLI_PENDING_Q)) !=
	- TW_CL_NULL) {
	- if ((error = tw_cli_submit_cmd(req))) {
	- if (error == TW_OSL_EBUSY) {
	- tw_cli_dbg_printf(2, ctlr->ctlr_handle,
	- tw_osl_cur_func(),
	- "Requeueing pending request");
	- req->state = TW_CLI_REQ_STATE_PENDING;
	- /*
	- * Queue the request at the head of the pending
	- * queue, and break away, so we don't try to
	- * submit any more requests.
	- */
	- tw_cli_req_q_insert_head(req, TW_CLI_PENDING_Q);
	- break;
	- } else {
	- tw_cl_create_event(ctlr->ctlr_handle,
	- TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1202, 0x1,
	- TW_CL_SEVERITY_ERROR_STRING,
	- "Could not start request "
	- "in pending queue",
	- "request = %p, opcode = 0x%x, "
	- "error = %d", req,
	- GET_OPCODE(req->cmd_pkt->
	- command.cmd_pkt_9k.res__opcode),
	- error);
	- /*
	- * Set the appropriate error and call the CL
	- * internal callback if there's one. If the
	- * request originator is polling for completion,
	- * he should be checking req->error to
	- * determine that the request did not go
	- * through. The request originators are
	- * responsible for the clean-up.
	- */
	- req->error_code = error;
	- req->state = TW_CLI_REQ_STATE_COMPLETE;
	- if (req->tw_cli_callback)
	- req->tw_cli_callback(req);
	- error = TW_OSL_ESUCCESS;
	- }
	- }
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_process_complete_queue
	- * Description: Calls the CL internal callback routine, if any, for
	- * each request in the complete queue.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_process_complete_queue(struct tw_cli_ctlr_context *ctlr)
	-{
	- struct tw_cli_req_context *req;
	-
	- tw_cli_dbg_printf(10, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /*
	- * Pull commands off the completed list, dispatch them appropriately.
	- */
	- while ((req = tw_cli_req_q_remove_head(ctlr, TW_CLI_COMPLETE_Q)) !=
	- TW_CL_NULL) {
	- /* Call the CL internal callback, if there's one. */
	- if (req->tw_cli_callback)
	- req->tw_cli_callback(req);
	- }
	-}
	-
	-/*
	- * Function name: tw_cli_complete_io
	- * Description: CL internal callback for SCSI/fw passthru requests.
	- *
	- * Input: req -- ptr to CL internal request context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_complete_io(struct tw_cli_req_context *req)
	-{
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	- struct tw_cl_req_packet *req_pkt =
	- (struct tw_cl_req_packet *)(req->orig_req);
	-
	- tw_cli_dbg_printf(8, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- req_pkt->status = TW_CL_ERR_REQ_SUCCESS;
	- if (req->error_code) {
	- req_pkt->status = TW_CL_ERR_REQ_UNABLE_TO_SUBMIT_COMMAND;
	- goto out;
	- }
	-
	- if (req->state != TW_CLI_REQ_STATE_COMPLETE) {
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1203, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "I/O completion on incomplete command!!",
	- "request = %p, status = %d",
	- req, req->state);
	-#ifdef TW_OSL_DEBUG
	- tw_cl_print_ctlr_stats(ctlr->ctlr_handle);
	-#endif /* TW_OSL_DEBUG */
	- return;
	- }
	-
	- if (req->flags & TW_CLI_REQ_FLAGS_PASSTHRU) {
	- /* Copy the command packet back into OSL's space. */
	- tw_osl_memcpy(req_pkt->gen_req_pkt.pt_req.cmd_pkt, req->cmd_pkt,
	- sizeof(struct tw_cl_command_packet));
	- } else
	- tw_cli_scsi_complete(req);
	-
	-out:
	- req_pkt->tw_osl_callback(req->req_handle);
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	-}
	-
	-/*
	- * Function name: tw_cli_scsi_complete
	- * Description: Completion routine for SCSI requests.
	- *
	- * Input: req -- ptr to CL internal request context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_scsi_complete(struct tw_cli_req_context *req)
	-{
	- struct tw_cl_req_packet *req_pkt =
	- (struct tw_cl_req_packet *)(req->orig_req);
	- struct tw_cl_scsi_req_packet *scsi_req =
	- &(req_pkt->gen_req_pkt.scsi_req);
	- struct tw_cl_command_9k *cmd =
	- &(req->cmd_pkt->command.cmd_pkt_9k);
	- struct tw_cl_command_header *cmd_hdr;
	- TW_UINT16 error;
	- TW_UINT8 *cdb;
	-
	- tw_cli_dbg_printf(8, req->ctlr->ctlr_handle, tw_osl_cur_func(),
	- "entered");
	-
	- scsi_req->scsi_status = cmd->status;
	- if (! cmd->status)
	- return;
	-
	- tw_cli_dbg_printf(1, req->ctlr->ctlr_handle, tw_osl_cur_func(),
	- "req_id = 0x%x, status = 0x%x",
	- GET_REQ_ID(cmd->lun_l4__req_id), cmd->status);
	-
	- cmd_hdr = &(req->cmd_pkt->cmd_hdr);
	- error = cmd_hdr->status_block.error;
	- if ((error == TWA_ERROR_LOGICAL_UNIT_NOT_SUPPORTED) \|\|
	- (error == TWA_ERROR_UNIT_OFFLINE)) {
	- if (GET_LUN_L4(cmd->lun_l4__req_id))
	- req_pkt->status \|= TW_CL_ERR_REQ_INVALID_LUN;
	- else
	- req_pkt->status \|= TW_CL_ERR_REQ_INVALID_TARGET;
	- } else {
	- tw_cli_dbg_printf(2, req->ctlr->ctlr_handle,
	- tw_osl_cur_func(),
	- "cmd = %x %x %x %x %x %x %x",
	- GET_OPCODE(cmd->res__opcode),
	- GET_SGL_OFF(cmd->res__opcode),
	- cmd->unit,
	- cmd->lun_l4__req_id,
	- cmd->status,
	- cmd->sgl_offset,
	- cmd->lun_h4__sgl_entries);
	-
	- cdb = (TW_UINT8 *)(cmd->cdb);
	- tw_cli_dbg_printf(2, req->ctlr->ctlr_handle,
	- tw_osl_cur_func(),
	- "cdb = %x %x %x %x %x %x %x %x "
	- "%x %x %x %x %x %x %x %x",
	- cdb[0], cdb[1], cdb[2], cdb[3],
	- cdb[4], cdb[5], cdb[6], cdb[7],
	- cdb[8], cdb[9], cdb[10], cdb[11],
	- cdb[12], cdb[13], cdb[14], cdb[15]);
	-
	-#if 0
	- /*
	- * Print the error. Firmware doesn't yet support
	- * the 'Mode Sense' cmd. Don't print if the cmd
	- * is 'Mode Sense', and the error is 'Invalid field
	- * in CDB'.
	- */
	- if (! ((cdb[0] == 0x1A) && (error == 0x10D)))
	- tw_cli_create_ctlr_event(req->ctlr,
	- TW_CL_MESSAGE_SOURCE_CONTROLLER_ERROR,
	- cmd_hdr);
	-#endif // 0
	- }
	-
	- if (scsi_req->sense_data) {
	- tw_osl_memcpy(scsi_req->sense_data, cmd_hdr->sense_data,
	- TWA_SENSE_DATA_LENGTH);
	- scsi_req->sense_len = TWA_SENSE_DATA_LENGTH;
	- req_pkt->status \|= TW_CL_ERR_REQ_AUTO_SENSE_VALID;
	- }
	- req_pkt->status \|= TW_CL_ERR_REQ_SCSI_ERROR;
	-}
	-
	-/*
	- * Function name: tw_cli_param_callback
	- * Description: Callback for get/set_param requests.
	- *
	- * Input: req -- ptr to completed request pkt
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_param_callback(struct tw_cli_req_context *req)
	-{
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	- union tw_cl_command_7k *cmd =
	- &(req->cmd_pkt->command.cmd_pkt_7k);
	- TW_INT32 error;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /*
	- * If the request was never submitted to the controller, the function
	- * that sets req->error is responsible for calling tw_cl_create_event.
	- */
	- if (! req->error_code)
	- if (cmd->param.status) {
	-#if 0
	- tw_cli_create_ctlr_event(ctlr,
	- TW_CL_MESSAGE_SOURCE_CONTROLLER_ERROR,
	- &(req->cmd_pkt->cmd_hdr));
	-#endif // 0
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1204, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "get/set_param failed",
	- "status = %d", cmd->param.status);
	- }
	-
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	-
	- if ((ctlr->get_more_aens) && (!(ctlr->reset_in_progress))) {
	- ctlr->get_more_aens = TW_CL_FALSE;
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "Fetching more AEN's");
	- if ((error = tw_cli_get_aen(ctlr)))
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1205, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Failed to fetch all AEN's from param_callback",
	- "error = %d", error);
	- }
	-}
	-
	-/*
	- * Function name: tw_cli_aen_callback
	- * Description: Callback for requests to fetch AEN's.
	- *
	- * Input: req -- ptr to completed request pkt
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_aen_callback(struct tw_cli_req_context *req)
	-{
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	- struct tw_cl_command_header *cmd_hdr;
	- struct tw_cl_command_9k *cmd =
	- &(req->cmd_pkt->command.cmd_pkt_9k);
	- TW_UINT16 aen_code = TWA_AEN_QUEUE_EMPTY;
	- TW_INT32 error;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "req_id = 0x%x, req error = %d, status = 0x%x",
	- GET_REQ_ID(cmd->lun_l4__req_id), req->error_code, cmd->status);
	-
	- /*
	- * If the request was never submitted to the controller, the function
	- * that sets error is responsible for calling tw_cl_create_event.
	- */
	- if (!(error = req->error_code))
	- if ((error = cmd->status)) {
	- cmd_hdr = (struct tw_cl_command_header *)
	- (&(req->cmd_pkt->cmd_hdr));
	-#if 0
	- tw_cli_create_ctlr_event(ctlr,
	- TW_CL_MESSAGE_SOURCE_CONTROLLER_ERROR,
	- cmd_hdr);
	-#endif // 0
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1206, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Request Sense failed",
	- "opcode = 0x%x, status = %d",
	- GET_OPCODE(cmd->res__opcode), cmd->status);
	- }
	-
	- if (error) {
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- return;
	- }
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "Request Sense command succeeded");
	-
	- aen_code = tw_cli_manage_aen(ctlr, req);
	-
	- if (aen_code != TWA_AEN_SYNC_TIME_WITH_HOST) {
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- if (aen_code != TWA_AEN_QUEUE_EMPTY)
	- if ((error = tw_cli_get_aen(ctlr)))
	- tw_cl_create_event(ctlr->ctlr_handle,
	- TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1207, 0x1,
	- TW_CL_SEVERITY_ERROR_STRING,
	- "Failed to fetch all AEN's",
	- "error = %d", error);
	- }
	-}
	-
	-/*
	- * Function name: tw_cli_manage_aen
	- * Description: Handles AEN's.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * req -- ptr to CL internal request context
	- * Output: None
	- * Return value: None
	- */
	-TW_UINT16
	-tw_cli_manage_aen(struct tw_cli_ctlr_context *ctlr,
	- struct tw_cli_req_context *req)
	-{
	- struct tw_cl_command_header *cmd_hdr;
	- TW_UINT16 aen_code;
	- TW_TIME local_time;
	- TW_TIME sync_time;
	- TW_UINT32 error;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- cmd_hdr = (struct tw_cl_command_header *)(req->data);
	- aen_code = cmd_hdr->status_block.error;
	-
	- switch (aen_code) {
	- case TWA_AEN_SYNC_TIME_WITH_HOST:
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "Received AEN_SYNC_TIME");
	- /*
	- * Free the internal req pkt right here, since
	- * tw_cli_set_param will need it.
	- */
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	-
	- /*
	- * We will use a callback in tw_cli_set_param only when
	- * interrupts are enabled and we can expect our callback
	- * to get called. Setting the get_more_aens
	- * flag will make the callback continue to try to retrieve
	- * more AEN's.
	- */
	- if (ctlr->interrupts_enabled)
	- ctlr->get_more_aens = TW_CL_TRUE;
	- /* Calculate time (in seconds) since last Sunday 12.00 AM. */
	- local_time = tw_osl_get_local_time();
	- sync_time = (local_time - (3 * 86400)) % 604800;
	- if ((error = tw_cli_set_param(ctlr, TWA_PARAM_TIME_TABLE,
	- TWA_PARAM_TIME_SCHED_TIME, 4,
	- &sync_time,
	- (ctlr->interrupts_enabled)
	- ? tw_cli_param_callback : TW_CL_NULL)))
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1208, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Unable to sync time with ctlr",
	- "error = %d", error);
	-
	- break;
	-
	- case TWA_AEN_QUEUE_EMPTY:
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "AEN queue empty");
	- break;
	-
	- default:
	- /* Queue the event. */
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "Queueing AEN");
	- tw_cli_create_ctlr_event(ctlr,
	- TW_CL_MESSAGE_SOURCE_CONTROLLER_EVENT,
	- cmd_hdr);
	- break;
	- } /* switch */
	- return(aen_code);
	-}
	-
	-/*
	- * Function name: tw_cli_enable_interrupts
	- * Description: Enables interrupts on the controller
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_enable_interrupts(struct tw_cli_ctlr_context *ctlr)
	-{
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- ctlr->interrupts_enabled = TW_CL_TRUE;
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr->ctlr_handle,
	- TWA_CONTROL_CLEAR_ATTENTION_INTERRUPT \|
	- TWA_CONTROL_UNMASK_RESPONSE_INTERRUPT \|
	- TWA_CONTROL_ENABLE_INTERRUPTS);
	-}
	-
	-/*
	- * Function name: twa_setup
	- * Description: Disables interrupts on the controller
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_disable_interrupts(struct tw_cli_ctlr_context *ctlr)
	-{
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr->ctlr_handle,
	- TWA_CONTROL_DISABLE_INTERRUPTS);
	- ctlr->interrupts_enabled = TW_CL_FALSE;
	-}
	Index: sys/dev/twa/tw_cl_io.c
	===================================================================
	--- sys/dev/twa/tw_cl_io.c
	+++ /dev/null
	@@ -1,1407 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- * Modifications by: Manjunath Ranganathaiah
	- */
	-
	-/*
	- * Common Layer I/O functions.
	- */
	-
	-#include "tw_osl_share.h"
	-#include "tw_cl_share.h"
	-#include "tw_cl_fwif.h"
	-#include "tw_cl_ioctl.h"
	-#include "tw_cl.h"
	-#include "tw_cl_externs.h"
	-#include "tw_osl_ioctl.h"
	-
	-#include <cam/cam.h>
	-#include <cam/cam_ccb.h>
	-#include <cam/cam_xpt_sim.h>
	-
	-/*
	- * Function name: tw_cl_start_io
	- * Description: Interface to OS Layer for accepting SCSI requests.
	- *
	- * Input: ctlr_handle -- controller handle
	- * req_pkt -- OSL built request packet
	- * req_handle -- request handle
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cl_start_io(struct tw_cl_ctlr_handle *ctlr_handle,
	- struct tw_cl_req_packet req_pkt, struct tw_cl_req_handle req_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr;
	- struct tw_cli_req_context *req;
	- struct tw_cl_command_9k *cmd;
	- struct tw_cl_scsi_req_packet *scsi_req;
	- TW_INT32 error = TW_CL_ERR_REQ_SUCCESS;
	-
	- tw_cli_dbg_printf(10, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- ctlr = (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	-
	- /*
	- * If working with a firmware version that does not support multiple
	- * luns, and this request is directed at a non-zero lun, error it
	- * back right away.
	- */
	- if ((req_pkt->gen_req_pkt.scsi_req.lun) &&
	- (ctlr->working_srl < TWA_MULTI_LUN_FW_SRL)) {
	- req_pkt->status \|= (TW_CL_ERR_REQ_INVALID_LUN \|
	- TW_CL_ERR_REQ_SCSI_ERROR);
	- req_pkt->tw_osl_callback(req_handle);
	- return(TW_CL_ERR_REQ_SUCCESS);
	- }
	-
	- if ((req = tw_cli_get_request(ctlr
	- )) == TW_CL_NULL) {
	- tw_cli_dbg_printf(2, ctlr_handle, tw_osl_cur_func(),
	- "Out of request context packets: returning busy");
	- return(TW_OSL_EBUSY);
	- }
	-
	- req_handle->cl_req_ctxt = req;
	- req->req_handle = req_handle;
	- req->orig_req = req_pkt;
	- req->tw_cli_callback = tw_cli_complete_io;
	-
	- req->flags \|= TW_CLI_REQ_FLAGS_EXTERNAL;
	- req->flags \|= TW_CLI_REQ_FLAGS_9K;
	-
	- scsi_req = &(req_pkt->gen_req_pkt.scsi_req);
	-
	- /* Build the cmd pkt. */
	- cmd = &(req->cmd_pkt->command.cmd_pkt_9k);
	-
	- req->cmd_pkt->cmd_hdr.header_desc.size_header = 128;
	-
	- cmd->res__opcode = BUILD_RES__OPCODE(0, TWA_FW_CMD_EXECUTE_SCSI);
	- cmd->unit = (TW_UINT8)(scsi_req->unit);
	- cmd->lun_l4__req_id = TW_CL_SWAP16(
	- BUILD_LUN_L4__REQ_ID(scsi_req->lun, req->request_id));
	- cmd->status = 0;
	- cmd->sgl_offset = 16; /* offset from end of hdr = max cdb len */
	- tw_osl_memcpy(cmd->cdb, scsi_req->cdb, scsi_req->cdb_len);
	-
	- if (req_pkt->flags & TW_CL_REQ_CALLBACK_FOR_SGLIST) {
	- TW_UINT32 num_sgl_entries;
	-
	- req_pkt->tw_osl_sgl_callback(req_handle, cmd->sg_list,
	- &num_sgl_entries);
	- cmd->lun_h4__sgl_entries =
	- TW_CL_SWAP16(BUILD_LUN_H4__SGL_ENTRIES(scsi_req->lun,
	- num_sgl_entries));
	- } else {
	- cmd->lun_h4__sgl_entries =
	- TW_CL_SWAP16(BUILD_LUN_H4__SGL_ENTRIES(scsi_req->lun,
	- scsi_req->sgl_entries));
	- tw_cli_fill_sg_list(ctlr, scsi_req->sg_list,
	- cmd->sg_list, scsi_req->sgl_entries);
	- }
	-
	- if (((TW_CL_Q_FIRST_ITEM(&(ctlr->req_q_head[TW_CLI_PENDING_Q]))) != TW_CL_NULL) \|\|
	- (ctlr->reset_in_progress)) {
	- tw_cli_req_q_insert_tail(req, TW_CLI_PENDING_Q);
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle,
	- TWA_CONTROL_UNMASK_COMMAND_INTERRUPT);
	- } else if ((error = tw_cli_submit_cmd(req))) {
	- tw_cli_dbg_printf(2, ctlr_handle, tw_osl_cur_func(),
	- "Could not start request. request = %p, error = %d",
	- req, error);
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_submit_cmd
	- * Description: Submits a cmd to firmware.
	- *
	- * Input: req -- ptr to CL internal request context
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_submit_cmd(struct tw_cli_req_context *req)
	-{
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	- struct tw_cl_ctlr_handle *ctlr_handle = ctlr->ctlr_handle;
	- TW_UINT32 status_reg;
	- TW_INT32 error = 0;
	-
	- tw_cli_dbg_printf(10, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Serialize access to the controller cmd queue. */
	- tw_osl_get_lock(ctlr_handle, ctlr->io_lock);
	-
	- /* For 9650SE first write low 4 bytes */
	- if ((ctlr->device_id == TW_CL_DEVICE_ID_9K_E) \|\|
	- (ctlr->device_id == TW_CL_DEVICE_ID_9K_SA))
	- tw_osl_write_reg(ctlr_handle,
	- TWA_COMMAND_QUEUE_OFFSET_LOW,
	- (TW_UINT32)(req->cmd_pkt_phys + sizeof(struct tw_cl_command_header)), 4);
	-
	- status_reg = TW_CLI_READ_STATUS_REGISTER(ctlr_handle);
	- if (status_reg & TWA_STATUS_COMMAND_QUEUE_FULL) {
	- struct tw_cl_req_packet *req_pkt =
	- (struct tw_cl_req_packet *)(req->orig_req);
	-
	- tw_cli_dbg_printf(7, ctlr_handle, tw_osl_cur_func(),
	- "Cmd queue full");
	-
	- if ((req->flags & TW_CLI_REQ_FLAGS_INTERNAL)
	- \|\| ((req_pkt) &&
	- (req_pkt->flags & TW_CL_REQ_RETRY_ON_BUSY))
	- ) {
	- if (req->state != TW_CLI_REQ_STATE_PENDING) {
	- tw_cli_dbg_printf(2, ctlr_handle,
	- tw_osl_cur_func(),
	- "pending internal/ioctl request");
	- req->state = TW_CLI_REQ_STATE_PENDING;
	- tw_cli_req_q_insert_tail(req, TW_CLI_PENDING_Q);
	- /* Unmask command interrupt. */
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle,
	- TWA_CONTROL_UNMASK_COMMAND_INTERRUPT);
	- } else
	- error = TW_OSL_EBUSY;
	- } else {
	- error = TW_OSL_EBUSY;
	- }
	- } else {
	- tw_cli_dbg_printf(10, ctlr_handle, tw_osl_cur_func(),
	- "Submitting command");
	-
	- /* Insert command into busy queue */
	- req->state = TW_CLI_REQ_STATE_BUSY;
	- tw_cli_req_q_insert_tail(req, TW_CLI_BUSY_Q);
	-
	- if ((ctlr->device_id == TW_CL_DEVICE_ID_9K_E) \|\|
	- (ctlr->device_id == TW_CL_DEVICE_ID_9K_SA)) {
	- /* Now write the high 4 bytes */
	- tw_osl_write_reg(ctlr_handle,
	- TWA_COMMAND_QUEUE_OFFSET_HIGH,
	- (TW_UINT32)(((TW_UINT64)(req->cmd_pkt_phys + sizeof(struct tw_cl_command_header)))>>32), 4);
	- } else {
	- if (ctlr->flags & TW_CL_64BIT_ADDRESSES) {
	- /* First write the low 4 bytes, then the high 4. */
	- tw_osl_write_reg(ctlr_handle,
	- TWA_COMMAND_QUEUE_OFFSET_LOW,
	- (TW_UINT32)(req->cmd_pkt_phys + sizeof(struct tw_cl_command_header)), 4);
	- tw_osl_write_reg(ctlr_handle,
	- TWA_COMMAND_QUEUE_OFFSET_HIGH,
	- (TW_UINT32)(((TW_UINT64)(req->cmd_pkt_phys + sizeof(struct tw_cl_command_header)))>>32), 4);
	- } else
	- tw_osl_write_reg(ctlr_handle,
	- TWA_COMMAND_QUEUE_OFFSET,
	- (TW_UINT32)(req->cmd_pkt_phys + sizeof(struct tw_cl_command_header)), 4);
	- }
	- }
	-
	- tw_osl_free_lock(ctlr_handle, ctlr->io_lock);
	-
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cl_fw_passthru
	- * Description: Interface to OS Layer for accepting firmware
	- * passthru requests.
	- * Input: ctlr_handle -- controller handle
	- * req_pkt -- OSL built request packet
	- * req_handle -- request handle
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cl_fw_passthru(struct tw_cl_ctlr_handle *ctlr_handle,
	- struct tw_cl_req_packet req_pkt, struct tw_cl_req_handle req_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr;
	- struct tw_cli_req_context *req;
	- union tw_cl_command_7k *cmd_7k;
	- struct tw_cl_command_9k *cmd_9k;
	- struct tw_cl_passthru_req_packet *pt_req;
	- TW_UINT8 opcode;
	- TW_UINT8 sgl_offset;
	- TW_VOID *sgl = TW_CL_NULL;
	- TW_INT32 error = TW_CL_ERR_REQ_SUCCESS;
	-
	- tw_cli_dbg_printf(5, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- ctlr = (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	-
	- if ((req = tw_cli_get_request(ctlr
	- )) == TW_CL_NULL) {
	- tw_cli_dbg_printf(2, ctlr_handle, tw_osl_cur_func(),
	- "Out of request context packets: returning busy");
	- return(TW_OSL_EBUSY);
	- }
	-
	- req_handle->cl_req_ctxt = req;
	- req->req_handle = req_handle;
	- req->orig_req = req_pkt;
	- req->tw_cli_callback = tw_cli_complete_io;
	-
	- req->flags \|= TW_CLI_REQ_FLAGS_PASSTHRU;
	-
	- pt_req = &(req_pkt->gen_req_pkt.pt_req);
	-
	- tw_osl_memcpy(req->cmd_pkt, pt_req->cmd_pkt,
	- pt_req->cmd_pkt_length);
	- /* Build the cmd pkt. */
	- if ((opcode = GET_OPCODE(((TW_UINT8 *)
	- (pt_req->cmd_pkt))[sizeof(struct tw_cl_command_header)]))
	- == TWA_FW_CMD_EXECUTE_SCSI) {
	- TW_UINT16 lun_l4, lun_h4;
	-
	- tw_cli_dbg_printf(5, ctlr_handle, tw_osl_cur_func(),
	- "passthru: 9k cmd pkt");
	- req->flags \|= TW_CLI_REQ_FLAGS_9K;
	- cmd_9k = &(req->cmd_pkt->command.cmd_pkt_9k);
	- lun_l4 = GET_LUN_L4(cmd_9k->lun_l4__req_id);
	- lun_h4 = GET_LUN_H4(cmd_9k->lun_h4__sgl_entries);
	- cmd_9k->lun_l4__req_id = TW_CL_SWAP16(
	- BUILD_LUN_L4__REQ_ID(lun_l4, req->request_id));
	- if (pt_req->sgl_entries) {
	- cmd_9k->lun_h4__sgl_entries =
	- TW_CL_SWAP16(BUILD_LUN_H4__SGL_ENTRIES(lun_h4,
	- pt_req->sgl_entries));
	- sgl = (TW_VOID *)(cmd_9k->sg_list);
	- }
	- } else {
	- tw_cli_dbg_printf(5, ctlr_handle, tw_osl_cur_func(),
	- "passthru: 7k cmd pkt");
	- cmd_7k = &(req->cmd_pkt->command.cmd_pkt_7k);
	- cmd_7k->generic.request_id =
	- (TW_UINT8)(TW_CL_SWAP16(req->request_id));
	- if ((sgl_offset =
	- GET_SGL_OFF(cmd_7k->generic.sgl_off__opcode))) {
	- if (ctlr->device_id == TW_CL_DEVICE_ID_9K_SA)
	- sgl = (((TW_UINT32 *)cmd_7k) + cmd_7k->generic.size);
	- else
	- sgl = (((TW_UINT32 *)cmd_7k) + sgl_offset);
	- cmd_7k->generic.size += pt_req->sgl_entries *
	- ((ctlr->flags & TW_CL_64BIT_ADDRESSES) ? 3 : 2);
	- }
	- }
	-
	- if (sgl)
	- tw_cli_fill_sg_list(ctlr, pt_req->sg_list,
	- sgl, pt_req->sgl_entries);
	-
	- if (((TW_CL_Q_FIRST_ITEM(&(ctlr->req_q_head[TW_CLI_PENDING_Q]))) != TW_CL_NULL) \|\|
	- (ctlr->reset_in_progress)) {
	- tw_cli_req_q_insert_tail(req, TW_CLI_PENDING_Q);
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle,
	- TWA_CONTROL_UNMASK_COMMAND_INTERRUPT);
	- } else if ((error = tw_cli_submit_cmd(req))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1100, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Failed to start passthru command",
	- "error = %d", error);
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cl_ioctl
	- * Description: Handler of CL supported ioctl cmds.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * cmd -- ioctl cmd
	- * buf -- ptr to buffer in kernel memory, which is
	- * a copy of the input buffer in user-space
	- * Output: buf -- ptr to buffer in kernel memory, which will
	- * need to be copied to the output buffer in
	- * user-space
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cl_ioctl(struct tw_cl_ctlr_handle ctlr_handle, u_long cmd, TW_VOID buf)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	- struct tw_cl_ioctl_packet *user_buf =
	- (struct tw_cl_ioctl_packet *)buf;
	- struct tw_cl_event_packet event_buf;
	- TW_INT32 event_index;
	- TW_INT32 start_index;
	- TW_INT32 error = TW_OSL_ESUCCESS;
	-
	- tw_cli_dbg_printf(5, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Serialize access to the AEN queue and the ioctl lock. */
	- tw_osl_get_lock(ctlr_handle, ctlr->gen_lock);
	-
	- switch (cmd) {
	- case TW_CL_IOCTL_GET_FIRST_EVENT:
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Get First Event");
	-
	- if (ctlr->aen_q_wrapped) {
	- if (ctlr->aen_q_overflow) {
	- /*
	- * The aen queue has wrapped, even before some
	- * events have been retrieved. Let the caller
	- * know that he missed out on some AEN's.
	- */
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_AEN_OVERFLOW;
	- ctlr->aen_q_overflow = TW_CL_FALSE;
	- } else
	- user_buf->driver_pkt.status = 0;
	- event_index = ctlr->aen_head;
	- } else {
	- if (ctlr->aen_head == ctlr->aen_tail) {
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_AEN_NO_EVENTS;
	- break;
	- }
	- user_buf->driver_pkt.status = 0;
	- event_index = ctlr->aen_tail; /* = 0 */
	- }
	- tw_osl_memcpy(user_buf->data_buf,
	- &(ctlr->aen_queue[event_index]),
	- sizeof(struct tw_cl_event_packet));
	-
	- ctlr->aen_queue[event_index].retrieved = TW_CL_AEN_RETRIEVED;
	-
	- break;
	-
	- case TW_CL_IOCTL_GET_LAST_EVENT:
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Get Last Event");
	-
	- if (ctlr->aen_q_wrapped) {
	- if (ctlr->aen_q_overflow) {
	- /*
	- * The aen queue has wrapped, even before some
	- * events have been retrieved. Let the caller
	- * know that he missed out on some AEN's.
	- */
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_AEN_OVERFLOW;
	- ctlr->aen_q_overflow = TW_CL_FALSE;
	- } else
	- user_buf->driver_pkt.status = 0;
	- } else {
	- if (ctlr->aen_head == ctlr->aen_tail) {
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_AEN_NO_EVENTS;
	- break;
	- }
	- user_buf->driver_pkt.status = 0;
	- }
	- event_index = (ctlr->aen_head - 1 + ctlr->max_aens_supported) %
	- ctlr->max_aens_supported;
	-
	- tw_osl_memcpy(user_buf->data_buf,
	- &(ctlr->aen_queue[event_index]),
	- sizeof(struct tw_cl_event_packet));
	-
	- ctlr->aen_queue[event_index].retrieved = TW_CL_AEN_RETRIEVED;
	-
	- break;
	-
	- case TW_CL_IOCTL_GET_NEXT_EVENT:
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Get Next Event");
	-
	- user_buf->driver_pkt.status = 0;
	- if (ctlr->aen_q_wrapped) {
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Get Next Event: wrapped");
	- if (ctlr->aen_q_overflow) {
	- /*
	- * The aen queue has wrapped, even before some
	- * events have been retrieved. Let the caller
	- * know that he missed out on some AEN's.
	- */
	- tw_cli_dbg_printf(2, ctlr_handle,
	- tw_osl_cur_func(),
	- "Get Next Event: overflow");
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_AEN_OVERFLOW;
	- ctlr->aen_q_overflow = TW_CL_FALSE;
	- }
	- start_index = ctlr->aen_head;
	- } else {
	- if (ctlr->aen_head == ctlr->aen_tail) {
	- tw_cli_dbg_printf(3, ctlr_handle,
	- tw_osl_cur_func(),
	- "Get Next Event: empty queue");
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_AEN_NO_EVENTS;
	- break;
	- }
	- start_index = ctlr->aen_tail; /* = 0 */
	- }
	- tw_osl_memcpy(&event_buf, user_buf->data_buf,
	- sizeof(struct tw_cl_event_packet));
	-
	- event_index = (start_index + event_buf.sequence_id -
	- ctlr->aen_queue[start_index].sequence_id + 1) %
	- ctlr->max_aens_supported;
	-
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Get Next Event: si = %x, ei = %x, ebsi = %x, "
	- "sisi = %x, eisi = %x",
	- start_index, event_index, event_buf.sequence_id,
	- ctlr->aen_queue[start_index].sequence_id,
	- ctlr->aen_queue[event_index].sequence_id);
	-
	- if (! (ctlr->aen_queue[event_index].sequence_id >
	- event_buf.sequence_id)) {
	- /*
	- * We don't have any event matching the criterion. So,
	- * we have to report TW_CL_ERROR_NO_EVENTS. If we also
	- * encountered an overflow condition above, we cannot
	- * report both conditions during this call. We choose
	- * to report NO_EVENTS this time, and an overflow the
	- * next time we are called.
	- */
	- if (user_buf->driver_pkt.status ==
	- TW_CL_ERROR_AEN_OVERFLOW) {
	- /*
	- * Make a note so we report the overflow
	- * next time.
	- */
	- ctlr->aen_q_overflow = TW_CL_TRUE;
	- }
	- user_buf->driver_pkt.status = TW_CL_ERROR_AEN_NO_EVENTS;
	- break;
	- }
	- /* Copy the event -- even if there has been an overflow. */
	- tw_osl_memcpy(user_buf->data_buf,
	- &(ctlr->aen_queue[event_index]),
	- sizeof(struct tw_cl_event_packet));
	-
	- ctlr->aen_queue[event_index].retrieved = TW_CL_AEN_RETRIEVED;
	-
	- break;
	-
	- case TW_CL_IOCTL_GET_PREVIOUS_EVENT:
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Get Previous Event");
	-
	- user_buf->driver_pkt.status = 0;
	- if (ctlr->aen_q_wrapped) {
	- if (ctlr->aen_q_overflow) {
	- /*
	- * The aen queue has wrapped, even before some
	- * events have been retrieved. Let the caller
	- * know that he missed out on some AEN's.
	- */
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_AEN_OVERFLOW;
	- ctlr->aen_q_overflow = TW_CL_FALSE;
	- }
	- start_index = ctlr->aen_head;
	- } else {
	- if (ctlr->aen_head == ctlr->aen_tail) {
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_AEN_NO_EVENTS;
	- break;
	- }
	- start_index = ctlr->aen_tail; /* = 0 */
	- }
	- tw_osl_memcpy(&event_buf, user_buf->data_buf,
	- sizeof(struct tw_cl_event_packet));
	-
	- event_index = (start_index + event_buf.sequence_id -
	- ctlr->aen_queue[start_index].sequence_id - 1) %
	- ctlr->max_aens_supported;
	-
	- if (! (ctlr->aen_queue[event_index].sequence_id <
	- event_buf.sequence_id)) {
	- /*
	- * We don't have any event matching the criterion. So,
	- * we have to report TW_CL_ERROR_NO_EVENTS. If we also
	- * encountered an overflow condition above, we cannot
	- * report both conditions during this call. We choose
	- * to report NO_EVENTS this time, and an overflow the
	- * next time we are called.
	- */
	- if (user_buf->driver_pkt.status ==
	- TW_CL_ERROR_AEN_OVERFLOW) {
	- /*
	- * Make a note so we report the overflow
	- * next time.
	- */
	- ctlr->aen_q_overflow = TW_CL_TRUE;
	- }
	- user_buf->driver_pkt.status = TW_CL_ERROR_AEN_NO_EVENTS;
	- break;
	- }
	- /* Copy the event -- even if there has been an overflow. */
	- tw_osl_memcpy(user_buf->data_buf,
	- &(ctlr->aen_queue[event_index]),
	- sizeof(struct tw_cl_event_packet));
	-
	- ctlr->aen_queue[event_index].retrieved = TW_CL_AEN_RETRIEVED;
	-
	- break;
	-
	- case TW_CL_IOCTL_GET_LOCK:
	- {
	- struct tw_cl_lock_packet lock_pkt;
	- TW_TIME cur_time;
	-
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Get ioctl lock");
	-
	- cur_time = tw_osl_get_local_time();
	- tw_osl_memcpy(&lock_pkt, user_buf->data_buf,
	- sizeof(struct tw_cl_lock_packet));
	-
	- if ((ctlr->ioctl_lock.lock == TW_CLI_LOCK_FREE) \|\|
	- (lock_pkt.force_flag) \|\|
	- (cur_time >= ctlr->ioctl_lock.timeout)) {
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "GET_LOCK: Getting lock!");
	- ctlr->ioctl_lock.lock = TW_CLI_LOCK_HELD;
	- ctlr->ioctl_lock.timeout =
	- cur_time + (lock_pkt.timeout_msec / 1000);
	- lock_pkt.time_remaining_msec = lock_pkt.timeout_msec;
	- user_buf->driver_pkt.status = 0;
	- } else {
	- tw_cli_dbg_printf(2, ctlr_handle, tw_osl_cur_func(),
	- "GET_LOCK: Lock already held!");
	- lock_pkt.time_remaining_msec = (TW_UINT32)(
	- (ctlr->ioctl_lock.timeout - cur_time) * 1000);
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_IOCTL_LOCK_ALREADY_HELD;
	- }
	- tw_osl_memcpy(user_buf->data_buf, &lock_pkt,
	- sizeof(struct tw_cl_lock_packet));
	- break;
	- }
	-
	- case TW_CL_IOCTL_RELEASE_LOCK:
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Release ioctl lock");
	-
	- if (ctlr->ioctl_lock.lock == TW_CLI_LOCK_FREE) {
	- tw_cli_dbg_printf(2, ctlr_handle, tw_osl_cur_func(),
	- "twa_ioctl: RELEASE_LOCK: Lock not held!");
	- user_buf->driver_pkt.status =
	- TW_CL_ERROR_IOCTL_LOCK_NOT_HELD;
	- } else {
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "RELEASE_LOCK: Releasing lock!");
	- ctlr->ioctl_lock.lock = TW_CLI_LOCK_FREE;
	- user_buf->driver_pkt.status = 0;
	- }
	- break;
	-
	- case TW_CL_IOCTL_GET_COMPATIBILITY_INFO:
	- {
	- struct tw_cl_compatibility_packet comp_pkt;
	-
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Get compatibility info");
	-
	- tw_osl_memcpy(comp_pkt.driver_version,
	- TW_OSL_DRIVER_VERSION_STRING,
	- sizeof(TW_OSL_DRIVER_VERSION_STRING));
	- comp_pkt.working_srl = ctlr->working_srl;
	- comp_pkt.working_branch = ctlr->working_branch;
	- comp_pkt.working_build = ctlr->working_build;
	- comp_pkt.driver_srl_high = TWA_CURRENT_FW_SRL;
	- comp_pkt.driver_branch_high =
	- TWA_CURRENT_FW_BRANCH(ctlr->arch_id);
	- comp_pkt.driver_build_high =
	- TWA_CURRENT_FW_BUILD(ctlr->arch_id);
	- comp_pkt.driver_srl_low = TWA_BASE_FW_SRL;
	- comp_pkt.driver_branch_low = TWA_BASE_FW_BRANCH;
	- comp_pkt.driver_build_low = TWA_BASE_FW_BUILD;
	- comp_pkt.fw_on_ctlr_srl = ctlr->fw_on_ctlr_srl;
	- comp_pkt.fw_on_ctlr_branch = ctlr->fw_on_ctlr_branch;
	- comp_pkt.fw_on_ctlr_build = ctlr->fw_on_ctlr_build;
	- user_buf->driver_pkt.status = 0;
	-
	- /* Copy compatibility information to user space. */
	- tw_osl_memcpy(user_buf->data_buf, &comp_pkt,
	- (sizeof(struct tw_cl_compatibility_packet) <
	- user_buf->driver_pkt.buffer_length) ?
	- sizeof(struct tw_cl_compatibility_packet) :
	- user_buf->driver_pkt.buffer_length);
	- break;
	- }
	-
	- default:
	- /* Unknown opcode. */
	- tw_cli_dbg_printf(3, ctlr_handle, tw_osl_cur_func(),
	- "Unknown ioctl cmd 0x%x", cmd);
	- error = TW_OSL_ENOTTY;
	- }
	-
	- tw_osl_free_lock(ctlr_handle, ctlr->gen_lock);
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_get_param
	- * Description: Get a firmware parameter.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * table_id -- parameter table #
	- * param_id -- index of the parameter in the table
	- * param_size -- size of the parameter in bytes
	- * callback -- ptr to function, if any, to be called
	- * back on completion; TW_CL_NULL if no callback.
	- * Output: param_data -- param value
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_get_param(struct tw_cli_ctlr_context *ctlr, TW_INT32 table_id,
	- TW_INT32 param_id, TW_VOID *param_data, TW_INT32 param_size,
	- TW_VOID (* callback)(struct tw_cli_req_context *req))
	-{
	- struct tw_cli_req_context *req;
	- union tw_cl_command_7k *cmd;
	- struct tw_cl_param_9k *param = TW_CL_NULL;
	- TW_INT32 error = TW_OSL_EBUSY;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Get a request packet. */
	- if ((req = tw_cli_get_request(ctlr
	- )) == TW_CL_NULL)
	- goto out;
	-
	- /* Make sure this is the only CL internal request at this time. */
	- if (ctlr->internal_req_busy) {
	- error = TW_OSL_EBUSY;
	- goto out;
	- }
	- ctlr->internal_req_busy = TW_CL_TRUE;
	- req->data = ctlr->internal_req_data;
	- req->data_phys = ctlr->internal_req_data_phys;
	- req->length = TW_CLI_SECTOR_SIZE;
	- req->flags \|= TW_CLI_REQ_FLAGS_INTERNAL;
	-
	- /* Initialize memory to read data into. */
	- param = (struct tw_cl_param_9k *)(req->data);
	- tw_osl_memzero(param, sizeof(struct tw_cl_param_9k) - 1 + param_size);
	-
	- /* Build the cmd pkt. */
	- cmd = &(req->cmd_pkt->command.cmd_pkt_7k);
	-
	- req->cmd_pkt->cmd_hdr.header_desc.size_header = 128;
	-
	- cmd->param.sgl_off__opcode =
	- BUILD_SGL_OFF__OPCODE(2, TWA_FW_CMD_GET_PARAM);
	- cmd->param.request_id = (TW_UINT8)(TW_CL_SWAP16(req->request_id));
	- cmd->param.host_id__unit = BUILD_HOST_ID__UNIT(0, 0);
	- cmd->param.param_count = TW_CL_SWAP16(1);
	-
	- if (ctlr->flags & TW_CL_64BIT_ADDRESSES) {
	- ((struct tw_cl_sg_desc64 *)(cmd->param.sgl))[0].address =
	- TW_CL_SWAP64(req->data_phys);
	- ((struct tw_cl_sg_desc64 *)(cmd->param.sgl))[0].length =
	- TW_CL_SWAP32(req->length);
	- cmd->param.size = 2 + 3;
	- } else {
	- ((struct tw_cl_sg_desc32 *)(cmd->param.sgl))[0].address =
	- TW_CL_SWAP32(req->data_phys);
	- ((struct tw_cl_sg_desc32 *)(cmd->param.sgl))[0].length =
	- TW_CL_SWAP32(req->length);
	- cmd->param.size = 2 + 2;
	- }
	-
	- /* Specify which parameter we need. */
	- param->table_id = TW_CL_SWAP16(table_id \| TWA_9K_PARAM_DESCRIPTOR);
	- param->parameter_id = (TW_UINT8)(param_id);
	- param->parameter_size_bytes = TW_CL_SWAP16(param_size);
	-
	- /* Submit the command. */
	- if (callback == TW_CL_NULL) {
	- /* There's no call back; wait till the command completes. */
	- error = tw_cli_submit_and_poll_request(req,
	- TW_CLI_REQUEST_TIMEOUT_PERIOD);
	- if (error)
	- goto out;
	- if ((error = cmd->param.status)) {
	-#if 0
	- tw_cli_create_ctlr_event(ctlr,
	- TW_CL_MESSAGE_SOURCE_CONTROLLER_ERROR,
	- &(req->cmd_pkt->cmd_hdr));
	-#endif // 0
	- goto out;
	- }
	- tw_osl_memcpy(param_data, param->data, param_size);
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- } else {
	- /* There's a call back. Simply submit the command. */
	- req->tw_cli_callback = callback;
	- if ((error = tw_cli_submit_cmd(req)))
	- goto out;
	- }
	- return(0);
	-
	-out:
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1101, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "get_param failed",
	- "error = %d", error);
	- if (param)
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- if (req)
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- return(1);
	-}
	-
	-/*
	- * Function name: tw_cli_set_param
	- * Description: Set a firmware parameter.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * table_id -- parameter table #
	- * param_id -- index of the parameter in the table
	- * param_size -- size of the parameter in bytes
	- * callback -- ptr to function, if any, to be called
	- * back on completion; TW_CL_NULL if no callback.
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_set_param(struct tw_cli_ctlr_context *ctlr, TW_INT32 table_id,
	- TW_INT32 param_id, TW_INT32 param_size, TW_VOID *data,
	- TW_VOID (* callback)(struct tw_cli_req_context *req))
	-{
	- struct tw_cli_req_context *req;
	- union tw_cl_command_7k *cmd;
	- struct tw_cl_param_9k *param = TW_CL_NULL;
	- TW_INT32 error = TW_OSL_EBUSY;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Get a request packet. */
	- if ((req = tw_cli_get_request(ctlr
	- )) == TW_CL_NULL)
	- goto out;
	-
	- /* Make sure this is the only CL internal request at this time. */
	- if (ctlr->internal_req_busy) {
	- error = TW_OSL_EBUSY;
	- goto out;
	- }
	- ctlr->internal_req_busy = TW_CL_TRUE;
	- req->data = ctlr->internal_req_data;
	- req->data_phys = ctlr->internal_req_data_phys;
	- req->length = TW_CLI_SECTOR_SIZE;
	- req->flags \|= TW_CLI_REQ_FLAGS_INTERNAL;
	-
	- /* Initialize memory to send data using. */
	- param = (struct tw_cl_param_9k *)(req->data);
	- tw_osl_memzero(param, sizeof(struct tw_cl_param_9k) - 1 + param_size);
	-
	- /* Build the cmd pkt. */
	- cmd = &(req->cmd_pkt->command.cmd_pkt_7k);
	-
	- req->cmd_pkt->cmd_hdr.header_desc.size_header = 128;
	-
	- cmd->param.sgl_off__opcode =
	- BUILD_SGL_OFF__OPCODE(2, TWA_FW_CMD_SET_PARAM);
	- cmd->param.request_id = (TW_UINT8)(TW_CL_SWAP16(req->request_id));
	- cmd->param.host_id__unit = BUILD_HOST_ID__UNIT(0, 0);
	- cmd->param.param_count = TW_CL_SWAP16(1);
	-
	- if (ctlr->flags & TW_CL_64BIT_ADDRESSES) {
	- ((struct tw_cl_sg_desc64 *)(cmd->param.sgl))[0].address =
	- TW_CL_SWAP64(req->data_phys);
	- ((struct tw_cl_sg_desc64 *)(cmd->param.sgl))[0].length =
	- TW_CL_SWAP32(req->length);
	- cmd->param.size = 2 + 3;
	- } else {
	- ((struct tw_cl_sg_desc32 *)(cmd->param.sgl))[0].address =
	- TW_CL_SWAP32(req->data_phys);
	- ((struct tw_cl_sg_desc32 *)(cmd->param.sgl))[0].length =
	- TW_CL_SWAP32(req->length);
	- cmd->param.size = 2 + 2;
	- }
	-
	- /* Specify which parameter we want to set. */
	- param->table_id = TW_CL_SWAP16(table_id \| TWA_9K_PARAM_DESCRIPTOR);
	- param->parameter_id = (TW_UINT8)(param_id);
	- param->parameter_size_bytes = TW_CL_SWAP16(param_size);
	- tw_osl_memcpy(param->data, data, param_size);
	-
	- /* Submit the command. */
	- if (callback == TW_CL_NULL) {
	- /* There's no call back; wait till the command completes. */
	- error = tw_cli_submit_and_poll_request(req,
	- TW_CLI_REQUEST_TIMEOUT_PERIOD);
	- if (error)
	- goto out;
	- if ((error = cmd->param.status)) {
	-#if 0
	- tw_cli_create_ctlr_event(ctlr,
	- TW_CL_MESSAGE_SOURCE_CONTROLLER_ERROR,
	- &(req->cmd_pkt->cmd_hdr));
	-#endif // 0
	- goto out;
	- }
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- } else {
	- /* There's a call back. Simply submit the command. */
	- req->tw_cli_callback = callback;
	- if ((error = tw_cli_submit_cmd(req)))
	- goto out;
	- }
	- return(error);
	-
	-out:
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1102, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "set_param failed",
	- "error = %d", error);
	- if (param)
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- if (req)
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_submit_and_poll_request
	- * Description: Sends down a firmware cmd, and waits for the completion
	- * in a tight loop.
	- *
	- * Input: req -- ptr to request pkt
	- * timeout -- max # of seconds to wait before giving up
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_submit_and_poll_request(struct tw_cli_req_context *req,
	- TW_UINT32 timeout)
	-{
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	- TW_TIME end_time;
	- TW_INT32 error;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /*
	- * If the cmd queue is full, tw_cli_submit_cmd will queue this
	- * request in the pending queue, since this is an internal request.
	- */
	- if ((error = tw_cli_submit_cmd(req))) {
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1103, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Failed to start internal request",
	- "error = %d", error);
	- return(error);
	- }
	-
	- /*
	- * Poll for the response until the command gets completed, or there's
	- * a timeout.
	- */
	- end_time = tw_osl_get_local_time() + timeout;
	- do {
	- if ((error = req->error_code))
	- /*
	- * This will take care of completion due to a reset,
	- * or a failure in tw_cli_submit_pending_queue.
	- * The caller should do the clean-up.
	- */
	- return(error);
	-
	- /* See if the command completed. */
	- tw_cli_process_resp_intr(ctlr);
	-
	- if ((req->state != TW_CLI_REQ_STATE_BUSY) &&
	- (req->state != TW_CLI_REQ_STATE_PENDING))
	- return(req->state != TW_CLI_REQ_STATE_COMPLETE);
	- } while (tw_osl_get_local_time() <= end_time);
	-
	- /* Time out! */
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1104, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Internal request timed out",
	- "request = %p", req);
	-
	- /*
	- * We will reset the controller only if the request has already been
	- * submitted, so as to not lose the request packet. If a busy request
	- * timed out, the reset will take care of freeing resources. If a
	- * pending request timed out, we will free resources for that request,
	- * right here, thereby avoiding a reset. So, the caller is expected
	- * to NOT cleanup when TW_OSL_ETIMEDOUT is returned.
	- */
	-
	- /*
	- * We have to make sure that this timed out request, if it were in the
	- * pending queue, doesn't get submitted while we are here, from
	- * tw_cli_submit_pending_queue. There could be a race in that case.
	- * Need to revisit.
	- */
	- if (req->state == TW_CLI_REQ_STATE_PENDING) {
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "Removing request from pending queue");
	- /*
	- * Request was never submitted. Clean up. Note that we did
	- * not do a reset. So, we have to remove the request ourselves
	- * from the pending queue (as against tw_cli_drain_pendinq_queue
	- * taking care of it).
	- */
	- tw_cli_req_q_remove_item(req, TW_CLI_PENDING_Q);
	- if ((TW_CL_Q_FIRST_ITEM(&(ctlr->req_q_head[TW_CLI_PENDING_Q]))) == TW_CL_NULL)
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr->ctlr_handle,
	- TWA_CONTROL_MASK_COMMAND_INTERRUPT);
	- if (req->data)
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- }
	-
	- return(TW_OSL_ETIMEDOUT);
	-}
	-
	-/*
	- * Function name: tw_cl_reset_ctlr
	- * Description: Soft resets and then initializes the controller;
	- * drains any incomplete requests.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * req_handle -- ptr to request handle
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cl_reset_ctlr(struct tw_cl_ctlr_handle *ctlr_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	- struct twa_softc *sc = ctlr_handle->osl_ctlr_ctxt;
	- struct tw_cli_req_context *req;
	- TW_INT32 reset_attempt = 1;
	- TW_INT32 error = 0;
	-
	- tw_cli_dbg_printf(2, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- ctlr->reset_in_progress = TW_CL_TRUE;
	- twa_teardown_intr(sc);
	-
	- /*
	- * Error back all requests in the complete, busy, and pending queues.
	- * If any request is already on its way to getting submitted, it's in
	- * none of these queues and so, will not be completed. That request
	- * will continue its course and get submitted to the controller after
	- * the reset is done (and io_lock is released).
	- */
	- tw_cli_drain_complete_queue(ctlr);
	- tw_cli_drain_busy_queue(ctlr);
	- tw_cli_drain_pending_queue(ctlr);
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- ctlr->get_more_aens = TW_CL_FALSE;
	-
	- /* Soft reset the controller. */
	- while (reset_attempt <= TW_CLI_MAX_RESET_ATTEMPTS) {
	- if ((error = tw_cli_soft_reset(ctlr))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1105, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Controller reset failed",
	- "error = %d; attempt %d", error, reset_attempt++);
	- reset_attempt++;
	- continue;
	- }
	-
	- /* Re-establish logical connection with the controller. */
	- if ((error = tw_cli_init_connection(ctlr,
	- (TW_UINT16)(ctlr->max_simult_reqs),
	- 0, 0, 0, 0, 0, TW_CL_NULL, TW_CL_NULL, TW_CL_NULL,
	- TW_CL_NULL, TW_CL_NULL))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1106, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Can't initialize connection after reset",
	- "error = %d", error);
	- reset_attempt++;
	- continue;
	- }
	-
	-#ifdef TW_OSL_DEBUG
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1107, 0x3, TW_CL_SEVERITY_INFO_STRING,
	- "Controller reset done!", " ");
	-#endif /* TW_OSL_DEBUG */
	- break;
	- } /* End of while */
	-
	- /* Move commands from the reset queue to the pending queue. */
	- while ((req = tw_cli_req_q_remove_head(ctlr, TW_CLI_RESET_Q)) != TW_CL_NULL) {
	- tw_osl_timeout(req->req_handle);
	- tw_cli_req_q_insert_tail(req, TW_CLI_PENDING_Q);
	- }
	-
	- twa_setup_intr(sc);
	- tw_cli_enable_interrupts(ctlr);
	- if ((TW_CL_Q_FIRST_ITEM(&(ctlr->req_q_head[TW_CLI_PENDING_Q]))) != TW_CL_NULL)
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle,
	- TWA_CONTROL_UNMASK_COMMAND_INTERRUPT);
	- ctlr->reset_in_progress = TW_CL_FALSE;
	- ctlr->reset_needed = TW_CL_FALSE;
	-
	- /* Request for a bus re-scan. */
	- tw_osl_scan_bus(ctlr_handle);
	-
	- return(error);
	-}
	-
	-TW_VOID
	-tw_cl_set_reset_needed(struct tw_cl_ctlr_handle *ctlr_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	-
	- ctlr->reset_needed = TW_CL_TRUE;
	-}
	-
	-TW_INT32
	-tw_cl_is_reset_needed(struct tw_cl_ctlr_handle *ctlr_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	-
	- return(ctlr->reset_needed);
	-}
	-
	-TW_INT32
	-tw_cl_is_active(struct tw_cl_ctlr_handle *ctlr_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)
	- (ctlr_handle->cl_ctlr_ctxt);
	-
	- return(ctlr->active);
	-}
	-
	-/*
	- * Function name: tw_cli_soft_reset
	- * Description: Does the actual soft reset.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_soft_reset(struct tw_cli_ctlr_context *ctlr)
	-{
	- struct tw_cl_ctlr_handle *ctlr_handle = ctlr->ctlr_handle;
	- int found;
	- int loop_count;
	- TW_UINT32 error;
	-
	- tw_cli_dbg_printf(1, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1108, 0x3, TW_CL_SEVERITY_INFO_STRING,
	- "Resetting controller...",
	- " ");
	-
	- /* Don't let any new commands get submitted to the controller. */
	- tw_osl_get_lock(ctlr_handle, ctlr->io_lock);
	-
	- TW_CLI_SOFT_RESET(ctlr_handle);
	-
	- if ((ctlr->device_id == TW_CL_DEVICE_ID_9K_X) \|\|
	- (ctlr->device_id == TW_CL_DEVICE_ID_9K_E) \|\|
	- (ctlr->device_id == TW_CL_DEVICE_ID_9K_SA)) {
	- /*
	- * There's a hardware bug in the G133 ASIC, which can lead to
	- * PCI parity errors and hangs, if the host accesses any
	- * registers when the firmware is resetting the hardware, as
	- * part of a hard/soft reset. The window of time when the
	- * problem can occur is about 10 ms. Here, we will handshake
	- * with the firmware to find out when the firmware is pulling
	- * down the hardware reset pin, and wait for about 500 ms to
	- * make sure we don't access any hardware registers (for
	- * polling) during that window.
	- */
	- ctlr->reset_phase1_in_progress = TW_CL_TRUE;
	- loop_count = 0;
	- do {
	- found = (tw_cli_find_response(ctlr, TWA_RESET_PHASE1_NOTIFICATION_RESPONSE) == TW_OSL_ESUCCESS);
	- tw_osl_delay(10);
	- loop_count++;
	- error = 0x7888;
	- } while (!found && (loop_count < 6000000)); /* Loop for no more than 60 seconds */
	-
	- if (!found) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1109, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Missed firmware handshake after soft-reset",
	- "error = %d", error);
	- tw_osl_free_lock(ctlr_handle, ctlr->io_lock);
	- return(error);
	- }
	-
	- tw_osl_delay(TWA_RESET_PHASE1_WAIT_TIME_MS * 1000);
	- ctlr->reset_phase1_in_progress = TW_CL_FALSE;
	- }
	-
	- if ((error = tw_cli_poll_status(ctlr,
	- TWA_STATUS_MICROCONTROLLER_READY \|
	- TWA_STATUS_ATTENTION_INTERRUPT,
	- TW_CLI_RESET_TIMEOUT_PERIOD))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1109, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Micro-ctlr not ready/No attn intr after reset",
	- "error = %d", error);
	- tw_osl_free_lock(ctlr_handle, ctlr->io_lock);
	- return(error);
	- }
	-
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr_handle,
	- TWA_CONTROL_CLEAR_ATTENTION_INTERRUPT);
	-
	- if ((error = tw_cli_drain_response_queue(ctlr))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x110A, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Can't drain response queue after reset",
	- "error = %d", error);
	- tw_osl_free_lock(ctlr_handle, ctlr->io_lock);
	- return(error);
	- }
	-
	- tw_osl_free_lock(ctlr_handle, ctlr->io_lock);
	-
	- if ((error = tw_cli_drain_aen_queue(ctlr))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x110B, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Can't drain AEN queue after reset",
	- "error = %d", error);
	- return(error);
	- }
	-
	- if ((error = tw_cli_find_aen(ctlr, TWA_AEN_SOFT_RESET))) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x110C, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Reset not reported by controller",
	- "error = %d", error);
	- return(error);
	- }
	-
	- return(TW_OSL_ESUCCESS);
	-}
	-
	-/*
	- * Function name: tw_cli_send_scsi_cmd
	- * Description: Sends down a scsi cmd to fw.
	- *
	- * Input: req -- ptr to request pkt
	- * cmd -- opcode of scsi cmd to send
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_send_scsi_cmd(struct tw_cli_req_context *req, TW_INT32 cmd)
	-{
	- struct tw_cl_command_packet *cmdpkt;
	- struct tw_cl_command_9k *cmd9k;
	- struct tw_cli_ctlr_context *ctlr;
	- TW_INT32 error;
	-
	- ctlr = req->ctlr;
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Make sure this is the only CL internal request at this time. */
	- if (ctlr->internal_req_busy)
	- return(TW_OSL_EBUSY);
	- ctlr->internal_req_busy = TW_CL_TRUE;
	- req->data = ctlr->internal_req_data;
	- req->data_phys = ctlr->internal_req_data_phys;
	- tw_osl_memzero(req->data, TW_CLI_SECTOR_SIZE);
	- req->length = TW_CLI_SECTOR_SIZE;
	-
	- /* Build the cmd pkt. */
	- cmdpkt = req->cmd_pkt;
	-
	- cmdpkt->cmd_hdr.header_desc.size_header = 128;
	-
	- cmd9k = &(cmdpkt->command.cmd_pkt_9k);
	-
	- cmd9k->res__opcode =
	- BUILD_RES__OPCODE(0, TWA_FW_CMD_EXECUTE_SCSI);
	- cmd9k->unit = 0;
	- cmd9k->lun_l4__req_id = TW_CL_SWAP16(req->request_id);
	- cmd9k->status = 0;
	- cmd9k->sgl_offset = 16; /* offset from end of hdr = max cdb len */
	- cmd9k->lun_h4__sgl_entries = TW_CL_SWAP16(1);
	-
	- if (req->ctlr->flags & TW_CL_64BIT_ADDRESSES) {
	- ((struct tw_cl_sg_desc64 *)(cmd9k->sg_list))[0].address =
	- TW_CL_SWAP64(req->data_phys);
	- ((struct tw_cl_sg_desc64 *)(cmd9k->sg_list))[0].length =
	- TW_CL_SWAP32(req->length);
	- } else {
	- ((struct tw_cl_sg_desc32 *)(cmd9k->sg_list))[0].address =
	- TW_CL_SWAP32(req->data_phys);
	- ((struct tw_cl_sg_desc32 *)(cmd9k->sg_list))[0].length =
	- TW_CL_SWAP32(req->length);
	- }
	-
	- cmd9k->cdb[0] = (TW_UINT8)cmd;
	- cmd9k->cdb[4] = 128;
	-
	- if ((error = tw_cli_submit_cmd(req)))
	- if (error != TW_OSL_EBUSY) {
	- tw_cli_dbg_printf(1, ctlr->ctlr_handle,
	- tw_osl_cur_func(),
	- "Failed to start SCSI command",
	- "request = %p, error = %d", req, error);
	- return(TW_OSL_EIO);
	- }
	- return(TW_OSL_ESUCCESS);
	-}
	-
	-/*
	- * Function name: tw_cli_get_aen
	- * Description: Sends down a Request Sense cmd to fw to fetch an AEN.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_get_aen(struct tw_cli_ctlr_context *ctlr)
	-{
	- struct tw_cli_req_context *req;
	- TW_INT32 error;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- if ((req = tw_cli_get_request(ctlr
	- )) == TW_CL_NULL)
	- return(TW_OSL_EBUSY);
	-
	- req->flags \|= TW_CLI_REQ_FLAGS_INTERNAL;
	- req->flags \|= TW_CLI_REQ_FLAGS_9K;
	- req->tw_cli_callback = tw_cli_aen_callback;
	- if ((error = tw_cli_send_scsi_cmd(req, 0x03 /* REQUEST_SENSE */))) {
	- tw_cli_dbg_printf(1, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "Could not send SCSI command",
	- "request = %p, error = %d", req, error);
	- if (req->data)
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_fill_sg_list
	- * Description: Fills in the scatter/gather list.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * sgl_src -- ptr to fill the sg list from
	- * sgl_dest-- ptr to sg list
	- * nsegments--# of segments
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_fill_sg_list(struct tw_cli_ctlr_context ctlr, TW_VOID sgl_src,
	- TW_VOID *sgl_dest, TW_INT32 num_sgl_entries)
	-{
	- TW_INT32 i;
	-
	- tw_cli_dbg_printf(10, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- if (ctlr->flags & TW_CL_64BIT_ADDRESSES) {
	- struct tw_cl_sg_desc64 *sgl_s =
	- (struct tw_cl_sg_desc64 *)sgl_src;
	- struct tw_cl_sg_desc64 *sgl_d =
	- (struct tw_cl_sg_desc64 *)sgl_dest;
	-
	- tw_cli_dbg_printf(10, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "64 bit addresses");
	- for (i = 0; i < num_sgl_entries; i++) {
	- sgl_d[i].address = TW_CL_SWAP64(sgl_s->address);
	- sgl_d[i].length = TW_CL_SWAP32(sgl_s->length);
	- sgl_s++;
	- if (ctlr->flags & TW_CL_64BIT_SG_LENGTH)
	- sgl_s = (struct tw_cl_sg_desc64 *)
	- (((TW_INT8 *)(sgl_s)) + 4);
	- }
	- } else {
	- struct tw_cl_sg_desc32 *sgl_s =
	- (struct tw_cl_sg_desc32 *)sgl_src;
	- struct tw_cl_sg_desc32 *sgl_d =
	- (struct tw_cl_sg_desc32 *)sgl_dest;
	-
	- tw_cli_dbg_printf(10, ctlr->ctlr_handle, tw_osl_cur_func(),
	- "32 bit addresses");
	- for (i = 0; i < num_sgl_entries; i++) {
	- sgl_d[i].address = TW_CL_SWAP32(sgl_s[i].address);
	- sgl_d[i].length = TW_CL_SWAP32(sgl_s[i].length);
	- }
	- }
	-}
	Index: sys/dev/twa/tw_cl_ioctl.h
	===================================================================
	--- sys/dev/twa/tw_cl_ioctl.h
	+++ /dev/null
	@@ -1,104 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_CL_IOCTL_H
	-
	-#define TW_CL_IOCTL_H
	-
	-/*
	- * Macros and structures for Common Layer handled ioctls.
	- */
	-
	-#define TW_CL_AEN_NOT_RETRIEVED 0x1
	-#define TW_CL_AEN_RETRIEVED 0x2
	-
	-#define TW_CL_ERROR_AEN_NO_EVENTS 0x1003 /* No more events */
	-#define TW_CL_ERROR_AEN_OVERFLOW 0x1004 /* AEN overflow occurred */
	-
	-#define TW_CL_ERROR_IOCTL_LOCK_NOT_HELD 0x1001 /* Not locked */
	-#define TW_CL_ERROR_IOCTL_LOCK_ALREADY_HELD 0x1002 /* Already locked */
	-
	-#pragma pack(1)
	-
	-/* Structure used to handle GET/RELEASE LOCK ioctls. */
	-struct tw_cl_lock_packet {
	- TW_UINT32 timeout_msec;
	- TW_UINT32 time_remaining_msec;
	- TW_UINT32 force_flag;
	-};
	-
	-/* Structure used to handle GET COMPATIBILITY INFO ioctl. */
	-struct tw_cl_compatibility_packet {
	- TW_UINT8 driver_version[32];/* driver version */
	- TW_UINT16 working_srl; /* driver & firmware negotiated srl */
	- TW_UINT16 working_branch; /* branch # of the firmware that the
	- driver is compatible with */
	- TW_UINT16 working_build; /* build # of the firmware that the
	- driver is compatible with */
	- TW_UINT16 driver_srl_high;/* highest driver supported srl */
	- TW_UINT16 driver_branch_high;/* highest driver supported branch */
	- TW_UINT16 driver_build_high;/* highest driver supported build */
	- TW_UINT16 driver_srl_low;/* lowest driver supported srl */
	- TW_UINT16 driver_branch_low;/* lowest driver supported branch */
	- TW_UINT16 driver_build_low;/* lowest driver supported build */
	- TW_UINT16 fw_on_ctlr_srl; /* srl of running firmware */
	- TW_UINT16 fw_on_ctlr_branch;/* branch # of running firmware */
	- TW_UINT16 fw_on_ctlr_build;/* build # of running firmware */
	-};
	-
	-/* Driver understandable part of the ioctl packet built by the API. */
	-struct tw_cl_driver_packet {
	- TW_UINT32 control_code;
	- TW_UINT32 status;
	- TW_UINT32 unique_id;
	- TW_UINT32 sequence_id;
	- TW_UINT32 os_status;
	- TW_UINT32 buffer_length;
	-};
	-
	-/* ioctl packet built by the API. */
	-struct tw_cl_ioctl_packet {
	- struct tw_cl_driver_packet driver_pkt;
	- TW_INT8 padding[488];
	- struct tw_cl_command_packet cmd_pkt;
	- TW_INT8 data_buf[1];
	-};
	-
	-#pragma pack()
	-
	-#endif /* TW_CL_IOCTL_H */
	Index: sys/dev/twa/tw_cl_misc.c
	===================================================================
	--- sys/dev/twa/tw_cl_misc.c
	+++ /dev/null
	@@ -1,991 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- * Modifications by: Manjunath Ranganathaiah
	- */
	-
	-/*
	- * Common Layer miscellaneous functions.
	- */
	-
	-#include "tw_osl_share.h"
	-#include "tw_cl_share.h"
	-#include "tw_cl_fwif.h"
	-#include "tw_cl_ioctl.h"
	-#include "tw_cl.h"
	-#include "tw_cl_externs.h"
	-#include "tw_osl_ioctl.h"
	-
	-/* AEN severity table. */
	-TW_INT8 *tw_cli_severity_string_table[] = {
	- "None",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_SEVERITY_WARNING_STRING,
	- TW_CL_SEVERITY_INFO_STRING,
	- TW_CL_SEVERITY_DEBUG_STRING,
	- ""
	-};
	-
	-/*
	- * Function name: tw_cli_drain_complete_queue
	- * Description: This function gets called during a controller reset.
	- * It errors back to the OS Layer, all those requests that
	- * are in the complete queue, at the time of the reset.
	- * Any CL internal requests will be simply freed.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_drain_complete_queue(struct tw_cli_ctlr_context *ctlr)
	-{
	- struct tw_cli_req_context *req;
	- struct tw_cl_req_packet *req_pkt;
	-
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Walk the busy queue. */
	- while ((req = tw_cli_req_q_remove_head(ctlr, TW_CLI_COMPLETE_Q)) !=
	- TW_CL_NULL) {
	- if (req->flags & TW_CLI_REQ_FLAGS_INTERNAL) {
	- /*
	- * It's an internal request. Set the appropriate
	- * error and call the CL internal callback if there's
	- * one. If the request originator is polling for
	- * completion, he should be checking req->error to
	- * determine that the request did not go through.
	- * The request originators are responsible for the
	- * clean-up.
	- */
	- req->error_code = TW_CL_ERR_REQ_BUS_RESET;
	- if (req->tw_cli_callback)
	- req->tw_cli_callback(req);
	- } else if (req->flags & TW_CLI_REQ_FLAGS_PASSTHRU) {
	- /* It's a passthru request. Complete it. */
	- if ((req_pkt = req->orig_req) != TW_CL_NULL) {
	- req_pkt->status = TW_CL_ERR_REQ_BUS_RESET;
	-
	- if (req_pkt->tw_osl_callback)
	- req_pkt->tw_osl_callback(req->req_handle);
	- }
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- } else {
	- /* It's an external (SCSI) request. Add it to the reset queue. */
	- tw_osl_untimeout(req->req_handle);
	- tw_cli_req_q_insert_tail(req, TW_CLI_RESET_Q);
	- }
	- } /* End of while loop */
	-}
	-
	-/*
	- * Function name: tw_cli_drain_busy_queue
	- * Description: This function gets called during a controller reset.
	- * It errors back to the OS Layer, all those requests that
	- * were pending with the firmware, at the time of the
	- * reset.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_drain_busy_queue(struct tw_cli_ctlr_context *ctlr)
	-{
	- struct tw_cli_req_context *req;
	- struct tw_cl_req_packet *req_pkt;
	-
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Walk the busy queue. */
	- while ((req = tw_cli_req_q_remove_head(ctlr, TW_CLI_BUSY_Q)) !=
	- TW_CL_NULL) {
	- if (req->flags & TW_CLI_REQ_FLAGS_INTERNAL) {
	- /*
	- * It's an internal request. Set the appropriate
	- * error and call the CL internal callback if there's
	- * one. If the request originator is polling for
	- * completion, he should be checking req->error to
	- * determine that the request did not go through.
	- * The request originators are responsible for the
	- * clean-up.
	- */
	- req->error_code = TW_CL_ERR_REQ_BUS_RESET;
	- if (req->tw_cli_callback)
	- req->tw_cli_callback(req);
	- } else if (req->flags & TW_CLI_REQ_FLAGS_PASSTHRU) {
	- /* It's a passthru request. Complete it. */
	- if ((req_pkt = req->orig_req) != TW_CL_NULL) {
	- req_pkt->status = TW_CL_ERR_REQ_BUS_RESET;
	-
	- if (req_pkt->tw_osl_callback)
	- req_pkt->tw_osl_callback(req->req_handle);
	- }
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- } else {
	- /* It's an external (SCSI) request. Add it to the reset queue. */
	- tw_osl_untimeout(req->req_handle);
	- tw_cli_req_q_insert_tail(req, TW_CLI_RESET_Q);
	- }
	- } /* End of while loop */
	-}
	-
	-/*
	- * Function name: tw_cli_drain_pending_queue
	- * Description: This function gets called during a controller reset.
	- * It errors back to the OS Layer, all those requests that
	- * were in the pending queue, at the time of the reset.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-
	-TW_VOID
	-tw_cli_drain_pending_queue(struct tw_cli_ctlr_context *ctlr)
	-{
	- struct tw_cli_req_context *req;
	- struct tw_cl_req_packet *req_pkt;
	-
	- tw_cli_dbg_printf(3, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /*
	- * Pull requests off the pending queue, and complete them.
	- */
	- while ((req = tw_cli_req_q_remove_head(ctlr, TW_CLI_PENDING_Q)) !=
	- TW_CL_NULL) {
	- if (req->flags & TW_CLI_REQ_FLAGS_INTERNAL) {
	- /*
	- * It's an internal request. Set the appropriate
	- * error and call the CL internal callback if there's
	- * one. If the request originator is polling for
	- * completion, he should be checking req->error to
	- * determine that the request did not go through.
	- * The request originators are responsible for the
	- * clean-up.
	- */
	- req->error_code = TW_CL_ERR_REQ_BUS_RESET;
	- if (req->tw_cli_callback)
	- req->tw_cli_callback(req);
	- } else if (req->flags & TW_CLI_REQ_FLAGS_PASSTHRU) {
	- /* It's a passthru request. Complete it. */
	- if ((req_pkt = req->orig_req) != TW_CL_NULL) {
	- req_pkt->status = TW_CL_ERR_REQ_BUS_RESET;
	-
	- if (req_pkt->tw_osl_callback)
	- req_pkt->tw_osl_callback(req->req_handle);
	- }
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- } else {
	- /* It's an external (SCSI) request. Add it to the reset queue. */
	- tw_osl_untimeout(req->req_handle);
	- tw_cli_req_q_insert_tail(req, TW_CLI_RESET_Q);
	- }
	- } /* End of while loop */
	-}
	-
	-/*
	- * Function name: tw_cli_drain_response_queue
	- * Description: Drain the controller response queue.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_drain_response_queue(struct tw_cli_ctlr_context *ctlr)
	-{
	- TW_UINT32 resp;
	- TW_UINT32 status_reg;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- for (;;) {
	- status_reg = TW_CLI_READ_STATUS_REGISTER(ctlr->ctlr_handle);
	-
	- if (status_reg & TWA_STATUS_RESPONSE_QUEUE_EMPTY)
	- return(TW_OSL_ESUCCESS); /* no more response queue entries */
	-
	- resp = TW_CLI_READ_RESPONSE_QUEUE(ctlr->ctlr_handle);
	- }
	-}
	-
	-/*
	- * Function name: tw_cli_find_response
	- * Description: Find a particular response in the ctlr response queue.
	- *
	- * Input: ctlr -- ptr to per ctlr structure
	- * req_id -- request id of the response to look for
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_find_response(struct tw_cli_ctlr_context *ctlr, TW_INT32 req_id)
	-{
	- TW_UINT32 resp;
	- TW_INT32 resp_id;
	- TW_UINT32 status_reg;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- for (;;) {
	- status_reg = TW_CLI_READ_STATUS_REGISTER(ctlr->ctlr_handle);
	-
	- if (status_reg & TWA_STATUS_RESPONSE_QUEUE_EMPTY)
	- return(TW_OSL_ENOTTY); /* no more response queue entries */
	-
	- if (ctlr->device_id == TW_CL_DEVICE_ID_9K) {
	- resp = TW_CLI_READ_RESPONSE_QUEUE(ctlr->ctlr_handle);
	- resp_id = GET_RESP_ID(resp);
	- } else {
	- resp = TW_CLI_READ_LARGE_RESPONSE_QUEUE(
	- ctlr->ctlr_handle);
	- resp_id = GET_LARGE_RESP_ID(resp);
	- }
	- if (resp_id == req_id)
	- return(TW_OSL_ESUCCESS); /* found the req_id */
	- }
	-}
	-
	-/*
	- * Function name: tw_cli_drain_aen_queue
	- * Description: Fetches all un-retrieved AEN's posted by fw.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_drain_aen_queue(struct tw_cli_ctlr_context *ctlr)
	-{
	- struct tw_cli_req_context *req;
	- struct tw_cl_command_header *cmd_hdr;
	- TW_TIME end_time;
	- TW_UINT16 aen_code;
	- TW_INT32 error;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- for (;;) {
	- if ((req = tw_cli_get_request(ctlr
	- )) == TW_CL_NULL) {
	- error = TW_OSL_EBUSY;
	- break;
	- }
	-
	- req->flags \|= TW_CLI_REQ_FLAGS_INTERNAL;
	- req->tw_cli_callback = TW_CL_NULL;
	- if ((error = tw_cli_send_scsi_cmd(req,
	- 0x03 /* REQUEST_SENSE */))) {
	- tw_cli_dbg_printf(1, ctlr->ctlr_handle,
	- tw_osl_cur_func(),
	- "Cannot send command to fetch aen");
	- break;
	- }
	-
	- end_time = tw_osl_get_local_time() +
	- TW_CLI_REQUEST_TIMEOUT_PERIOD;
	- do {
	- if ((error = req->error_code))
	- /*
	- * This will take care of completion due to
	- * a reset, or a failure in
	- * tw_cli_submit_pending_queue.
	- */
	- goto out;
	-
	- tw_cli_process_resp_intr(req->ctlr);
	-
	- if ((req->state != TW_CLI_REQ_STATE_BUSY) &&
	- (req->state != TW_CLI_REQ_STATE_PENDING))
	- break;
	- } while (tw_osl_get_local_time() <= end_time);
	-
	- if (req->state != TW_CLI_REQ_STATE_COMPLETE) {
	- error = TW_OSL_ETIMEDOUT;
	- break;
	- }
	-
	- if ((error = req->cmd_pkt->command.cmd_pkt_9k.status)) {
	- cmd_hdr = &req->cmd_pkt->cmd_hdr;
	-#if 0
	- tw_cli_create_ctlr_event(ctlr,
	- TW_CL_MESSAGE_SOURCE_CONTROLLER_ERROR,
	- cmd_hdr);
	-#endif // 0
	- break;
	- }
	-
	- aen_code = tw_cli_manage_aen(ctlr, req);
	- if (aen_code == TWA_AEN_QUEUE_EMPTY)
	- break;
	- if (aen_code == TWA_AEN_SYNC_TIME_WITH_HOST)
	- continue;
	-
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- }
	-
	-out:
	- if (req) {
	- if (req->data)
	- ctlr->internal_req_busy = TW_CL_FALSE;
	- tw_cli_req_q_insert_tail(req, TW_CLI_FREE_Q);
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_find_aen
	- * Description: Reports whether a given AEN ever occurred.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * aen_code-- AEN to look for
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_find_aen(struct tw_cli_ctlr_context *ctlr, TW_UINT16 aen_code)
	-{
	- TW_UINT32 last_index;
	- TW_INT32 i;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- if (ctlr->aen_q_wrapped)
	- last_index = ctlr->aen_head;
	- else
	- last_index = 0;
	-
	- i = ctlr->aen_head;
	- do {
	- i = (i + ctlr->max_aens_supported - 1) %
	- ctlr->max_aens_supported;
	- if (ctlr->aen_queue[i].aen_code == aen_code)
	- return(TW_OSL_ESUCCESS);
	- } while (i != last_index);
	-
	- return(TW_OSL_EGENFAILURE);
	-}
	-
	-/*
	- * Function name: tw_cli_poll_status
	- * Description: Poll for a given status to show up in the firmware
	- * status register.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * status -- status to look for
	- * timeout -- max # of seconds to wait before giving up
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_cli_poll_status(struct tw_cli_ctlr_context *ctlr, TW_UINT32 status,
	- TW_UINT32 timeout)
	-{
	- TW_TIME end_time;
	- TW_UINT32 status_reg;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- end_time = tw_osl_get_local_time() + timeout;
	- do {
	- status_reg = TW_CLI_READ_STATUS_REGISTER(ctlr->ctlr_handle);
	- if ((status_reg & status) == status)
	- /* got the required bit(s) */
	- return(TW_OSL_ESUCCESS);
	-
	- tw_osl_delay(1000);
	- } while (tw_osl_get_local_time() <= end_time);
	-
	- return(TW_OSL_ETIMEDOUT);
	-}
	-
	-/*
	- * Function name: tw_cl_create_event
	- * Description: Creates and queues ctlr/CL/OSL AEN's to be
	- * supplied to user-space tools on request.
	- * Also notifies OS Layer.
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * queue_event-- TW_CL_TRUE --> queue event;
	- * TW_CL_FALSE--> don't queue event
	- * (simply notify OSL)
	- * event_src -- source of event
	- * event_code -- AEN/error code
	- * severity -- severity of event
	- * severity_str--Text description of severity
	- * event_desc -- standard string related to the event/error
	- * event_specific_desc -- format string for additional
	- * info about the event
	- * ... -- additional arguments conforming to the format
	- * specified by event_specific_desc
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cl_create_event(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_UINT8 queue_event, TW_UINT8 event_src, TW_UINT16 event_code,
	- TW_UINT8 severity, TW_UINT8 severity_str, TW_UINT8 event_desc,
	- TW_UINT8 *event_specific_desc, ...)
	-{
	- struct tw_cli_ctlr_context *ctlr = ctlr_handle->cl_ctlr_ctxt;
	- struct tw_cl_event_packet event_pkt;
	- struct tw_cl_event_packet *event;
	- TW_UINT32 aen_head;
	- va_list ap;
	-
	- tw_cli_dbg_printf(8, ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- if ((ctlr) && (queue_event)) {
	- /* Protect access to ctlr->aen_head. */
	- tw_osl_get_lock(ctlr_handle, ctlr->gen_lock);
	-
	- aen_head = ctlr->aen_head;
	- ctlr->aen_head = (aen_head + 1) % ctlr->max_aens_supported;
	-
	- /* Queue the event. */
	- event = &(ctlr->aen_queue[aen_head]);
	- tw_osl_memzero(event->parameter_data,
	- sizeof(event->parameter_data));
	-
	- if (event->retrieved == TW_CL_AEN_NOT_RETRIEVED)
	- ctlr->aen_q_overflow = TW_CL_TRUE;
	- event->sequence_id = ++(ctlr->aen_cur_seq_id);
	- if ((aen_head + 1) == ctlr->max_aens_supported) {
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle,
	- tw_osl_cur_func(), "AEN queue wrapped");
	- ctlr->aen_q_wrapped = TW_CL_TRUE;
	- }
	-
	- /* Free access to ctlr->aen_head. */
	- tw_osl_free_lock(ctlr_handle, ctlr->gen_lock);
	- } else {
	- event = &event_pkt;
	- tw_osl_memzero(event, sizeof(struct tw_cl_event_packet));
	- }
	-
	- event->event_src = event_src;
	- event->time_stamp_sec = (TW_UINT32)tw_osl_get_local_time();
	- event->aen_code = event_code;
	- event->severity = severity;
	- tw_osl_strcpy(event->severity_str, severity_str);
	- event->retrieved = TW_CL_AEN_NOT_RETRIEVED;
	-
	- va_start(ap, event_specific_desc);
	- tw_osl_vsprintf(event->parameter_data, event_specific_desc, ap);
	- va_end(ap);
	-
	- event->parameter_len =
	- (TW_UINT8)(tw_osl_strlen(event->parameter_data));
	- tw_osl_strcpy(event->parameter_data + event->parameter_len + 1,
	- event_desc);
	- event->parameter_len += (1 + tw_osl_strlen(event_desc));
	-
	- tw_cli_dbg_printf(4, ctlr_handle, tw_osl_cur_func(),
	- "event = %x %x %x %x %x %x %x\n %s",
	- event->sequence_id,
	- event->time_stamp_sec,
	- event->aen_code,
	- event->severity,
	- event->retrieved,
	- event->repeat_count,
	- event->parameter_len,
	- event->parameter_data);
	-
	- tw_osl_notify_event(ctlr_handle, event);
	-}
	-
	-/*
	- * Function name: tw_cli_get_request
	- * Description: Gets a request pkt from the free queue.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * req_pkt -- ptr to OSL built req_pkt, if there's one
	- * Output: None
	- * Return value: ptr to request pkt -- success
	- * TW_CL_NULL -- failure
	- */
	-struct tw_cli_req_context *
	-tw_cli_get_request(struct tw_cli_ctlr_context *ctlr
	- )
	-{
	- struct tw_cli_req_context *req;
	-
	- tw_cli_dbg_printf(4, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- {
	- /* Get a free request packet. */
	- req = tw_cli_req_q_remove_head(ctlr, TW_CLI_FREE_Q);
	- }
	-
	- /* Initialize some fields to their defaults. */
	- if (req) {
	- req->req_handle = TW_CL_NULL;
	- req->data = TW_CL_NULL;
	- req->length = 0;
	- req->data_phys = 0;
	- req->state = TW_CLI_REQ_STATE_INIT; /* req being initialized */
	- req->flags = 0;
	- req->error_code = 0;
	- req->orig_req = TW_CL_NULL;
	- req->tw_cli_callback = TW_CL_NULL;
	-
	- /*
	- * Look at the status field in the command packet to see how
	- * it completed the last time it was used, and zero out only
	- * the portions that might have changed. Note that we don't
	- * care to zero out the sglist.
	- */
	- if (req->cmd_pkt->command.cmd_pkt_9k.status)
	- tw_osl_memzero(req->cmd_pkt,
	- sizeof(struct tw_cl_command_header) +
	- 28 /* max bytes before sglist */);
	- else
	- tw_osl_memzero(&(req->cmd_pkt->command),
	- 28 /* max bytes before sglist */);
	- }
	- return(req);
	-}
	-
	-/*
	- * Function name: tw_cli_dbg_printf
	- * Description: Calls OSL print function if dbg_level is appropriate
	- *
	- * Input: dbg_level -- Determines whether or not to print
	- * ctlr_handle -- controller handle
	- * cur_func -- text name of calling function
	- * fmt -- format string for the arguments to follow
	- * ... -- variable number of arguments, to be printed
	- * based on the fmt string
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_dbg_printf(TW_UINT8 dbg_level,
	- struct tw_cl_ctlr_handle ctlr_handle, const TW_INT8 cur_func,
	- TW_INT8 *fmt, ...)
	-{
	-#ifdef TW_OSL_DEBUG
	- TW_INT8 print_str[256];
	- va_list ap;
	-
	- tw_osl_memzero(print_str, 256);
	- if (dbg_level <= TW_OSL_DEBUG_LEVEL_FOR_CL) {
	- tw_osl_sprintf(print_str, "%s: ", cur_func);
	-
	- va_start(ap, fmt);
	- tw_osl_vsprintf(print_str + tw_osl_strlen(print_str), fmt, ap);
	- va_end(ap);
	-
	- tw_osl_strcpy(print_str + tw_osl_strlen(print_str), "\n");
	- tw_osl_dbg_printf(ctlr_handle, "%s", print_str);
	- }
	-#endif /* TW_OSL_DEBUG */
	-}
	-
	-/*
	- * Function name: tw_cli_notify_ctlr_info
	- * Description: Notify OSL of controller info (fw/BIOS versions, etc.).
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cli_notify_ctlr_info(struct tw_cli_ctlr_context *ctlr)
	-{
	- TW_INT8 fw_ver[16];
	- TW_INT8 bios_ver[16];
	- TW_INT8 ctlr_model[16];
	- TW_INT32 error[3];
	- TW_UINT8 num_ports = 0;
	-
	- tw_cli_dbg_printf(5, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Get the port count. */
	- error[0] = tw_cli_get_param(ctlr, TWA_PARAM_CONTROLLER_TABLE,
	- TWA_PARAM_CONTROLLER_PORT_COUNT, &num_ports,
	- 1, TW_CL_NULL);
	-
	- /* Get the firmware and BIOS versions. */
	- error[0] = tw_cli_get_param(ctlr, TWA_PARAM_VERSION_TABLE,
	- TWA_PARAM_VERSION_FW, fw_ver, 16, TW_CL_NULL);
	- error[1] = tw_cli_get_param(ctlr, TWA_PARAM_VERSION_TABLE,
	- TWA_PARAM_VERSION_BIOS, bios_ver, 16, TW_CL_NULL);
	- error[2] = tw_cli_get_param(ctlr, TWA_PARAM_VERSION_TABLE,
	- TWA_PARAM_CTLR_MODEL, ctlr_model, 16, TW_CL_NULL);
	-
	- tw_cl_create_event(ctlr->ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR,
	- 0x1300, 0x3, TW_CL_SEVERITY_INFO_STRING,
	- "Controller details:",
	- "Model %.16s, %d ports, Firmware %.16s, BIOS %.16s",
	- error[2]?(TW_INT8 *)TW_CL_NULL:ctlr_model,
	- num_ports,
	- error[0]?(TW_INT8 *)TW_CL_NULL:fw_ver,
	- error[1]?(TW_INT8 *)TW_CL_NULL:bios_ver);
	-}
	-
	-/*
	- * Function name: tw_cli_check_ctlr_state
	- * Description: Makes sure that the fw status register reports a
	- * proper status.
	- *
	- * Input: ctlr -- ptr to CL internal ctlr context
	- * status_reg-- value in the status register
	- * Output: None
	- * Return value: 0 -- no errors
	- * non-zero-- errors
	- */
	-TW_INT32
	-tw_cli_check_ctlr_state(struct tw_cli_ctlr_context *ctlr, TW_UINT32 status_reg)
	-{
	- struct tw_cl_ctlr_handle *ctlr_handle = ctlr->ctlr_handle;
	- TW_INT32 error = TW_OSL_ESUCCESS;
	-
	- tw_cli_dbg_printf(8, ctlr->ctlr_handle, tw_osl_cur_func(), "entered");
	-
	- /* Check if the 'micro-controller ready' bit is not set. */
	- if (!(status_reg & TWA_STATUS_MICROCONTROLLER_READY)) {
	- TW_INT8 desc[200];
	-
	- tw_osl_memzero(desc, 200);
	- if (!(ctlr->reset_phase1_in_progress)) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1301, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Missing expected status bit(s)",
	- "status reg = 0x%x; Missing bits: %s",
	- status_reg,
	- tw_cli_describe_bits(
	- TWA_STATUS_MICROCONTROLLER_READY,
	- desc));
	- error = TW_OSL_EGENFAILURE;
	- }
	- }
	-
	- /* Check if any error bits are set. */
	- if ((status_reg & TWA_STATUS_UNEXPECTED_BITS) != 0) {
	- TW_INT8 desc[200];
	-
	- tw_osl_memzero(desc, 200);
	-
	- /* Skip queue error msgs during 9650SE/9690SA reset */
	- if (((ctlr->device_id != TW_CL_DEVICE_ID_9K_E) &&
	- (ctlr->device_id != TW_CL_DEVICE_ID_9K_SA)) \|\|
	- (!(ctlr->reset_in_progress)) \|\|
	- ((status_reg & TWA_STATUS_QUEUE_ERROR_INTERRUPT) == 0))
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1302, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Unexpected status bit(s)",
	- "status reg = 0x%x Unexpected bits: %s",
	- status_reg & TWA_STATUS_UNEXPECTED_BITS,
	- tw_cli_describe_bits(status_reg &
	- TWA_STATUS_UNEXPECTED_BITS, desc));
	-
	- if (status_reg & TWA_STATUS_PCI_PARITY_ERROR_INTERRUPT) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1303, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "PCI parity error: clearing... "
	- "Re-seat/move/replace card",
	- "status reg = 0x%x %s",
	- status_reg,
	- tw_cli_describe_bits(status_reg, desc));
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr->ctlr_handle,
	- TWA_CONTROL_CLEAR_PARITY_ERROR);
	-
	-#ifdef TW_OSL_PCI_CONFIG_ACCESSIBLE
	- tw_osl_write_pci_config(ctlr->ctlr_handle,
	- TW_CLI_PCI_CONFIG_STATUS_OFFSET,
	- TWA_PCI_CONFIG_CLEAR_PARITY_ERROR, 2);
	-#endif /* TW_OSL_PCI_CONFIG_ACCESSIBLE */
	-
	- }
	-
	- if (status_reg & TWA_STATUS_PCI_ABORT_INTERRUPT) {
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1304, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "PCI abort: clearing... ",
	- "status reg = 0x%x %s",
	- status_reg,
	- tw_cli_describe_bits(status_reg, desc));
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr->ctlr_handle,
	- TWA_CONTROL_CLEAR_PCI_ABORT);
	-
	-#ifdef TW_OSL_PCI_CONFIG_ACCESSIBLE
	- tw_osl_write_pci_config(ctlr->ctlr_handle,
	- TW_CLI_PCI_CONFIG_STATUS_OFFSET,
	- TWA_PCI_CONFIG_CLEAR_PCI_ABORT, 2);
	-#endif /* TW_OSL_PCI_CONFIG_ACCESSIBLE */
	- }
	-
	- if (status_reg & TWA_STATUS_QUEUE_ERROR_INTERRUPT) {
	- /* Skip queue error msgs during 9650SE/9690SA reset */
	- if (((ctlr->device_id != TW_CL_DEVICE_ID_9K_E) &&
	- (ctlr->device_id != TW_CL_DEVICE_ID_9K_SA)) \|\|
	- (!(ctlr->reset_in_progress)))
	- tw_cl_create_event(ctlr_handle, TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT,
	- 0x1305, 0x1, TW_CL_SEVERITY_ERROR_STRING,
	- "Controller queue error: clearing... ",
	- "status reg = 0x%x %s",
	- status_reg,
	- tw_cli_describe_bits(status_reg, desc));
	- TW_CLI_WRITE_CONTROL_REGISTER(ctlr->ctlr_handle,
	- TWA_CONTROL_CLEAR_QUEUE_ERROR);
	- }
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_cli_describe_bits
	- * Description: Given the value of the status register, returns a
	- * string describing the meaning of each set bit.
	- *
	- * Input: reg -- status register value
	- * Output: Pointer to a string describing each set bit
	- * Return value: Pointer to the string describing each set bit
	- */
	-TW_INT8 *
	-tw_cli_describe_bits(TW_UINT32 reg, TW_INT8 *str)
	-{
	- tw_osl_strcpy(str, "[");
	-
	- if (reg & TWA_STATUS_COMMAND_QUEUE_EMPTY)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "CMD_Q_EMPTY,");
	- if (reg & TWA_STATUS_MICROCONTROLLER_READY)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "MC_RDY,");
	- if (reg & TWA_STATUS_RESPONSE_QUEUE_EMPTY)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "RESP_Q_EMPTY,");
	- if (reg & TWA_STATUS_COMMAND_QUEUE_FULL)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "CMD_Q_FULL,");
	- if (reg & TWA_STATUS_RESPONSE_INTERRUPT)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "RESP_INTR,");
	- if (reg & TWA_STATUS_COMMAND_INTERRUPT)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "CMD_INTR,");
	- if (reg & TWA_STATUS_ATTENTION_INTERRUPT)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "ATTN_INTR,");
	- if (reg & TWA_STATUS_HOST_INTERRUPT)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "HOST_INTR,");
	- if (reg & TWA_STATUS_PCI_ABORT_INTERRUPT)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "PCI_ABRT,");
	- if (reg & TWA_STATUS_QUEUE_ERROR_INTERRUPT)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "Q_ERR,");
	- if (reg & TWA_STATUS_PCI_PARITY_ERROR_INTERRUPT)
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "PCI_PERR");
	-
	- tw_osl_strcpy(&str[tw_osl_strlen(str)], "]");
	- return(str);
	-}
	-
	-#ifdef TW_OSL_DEBUG
	-
	-/*
	- * Function name: tw_cl_print_ctlr_stats
	- * Description: Prints the current status of the controller.
	- *
	- * Input: ctlr_handle-- controller handle
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cl_print_ctlr_stats(struct tw_cl_ctlr_handle *ctlr_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	- TW_UINT32 status_reg;
	- TW_INT8 desc[200];
	-
	- tw_cli_dbg_printf(7, ctlr->ctlr_handle, "", "entered");
	-
	- /* Print current controller details. */
	- tw_cli_dbg_printf(0, ctlr_handle, "", "cl_ctlr_ctxt = %p", ctlr);
	-
	- tw_osl_memzero(desc, 200);
	- status_reg = TW_CLI_READ_STATUS_REGISTER(ctlr_handle);
	- tw_cli_dbg_printf(0, ctlr_handle, "", "status reg = 0x%x %s",
	- status_reg, tw_cli_describe_bits(status_reg, desc));
	-
	- tw_cli_dbg_printf(0, ctlr_handle, "", "CLq type current max");
	- tw_cli_dbg_printf(0, ctlr_handle, "", "free %04d %04d",
	- ctlr->q_stats[TW_CLI_FREE_Q].cur_len,
	- ctlr->q_stats[TW_CLI_FREE_Q].max_len);
	- tw_cli_dbg_printf(0, ctlr_handle, "", "busy %04d %04d",
	- ctlr->q_stats[TW_CLI_BUSY_Q].cur_len,
	- ctlr->q_stats[TW_CLI_BUSY_Q].max_len);
	- tw_cli_dbg_printf(0, ctlr_handle, "", "pending %04d %04d",
	- ctlr->q_stats[TW_CLI_PENDING_Q].cur_len,
	- ctlr->q_stats[TW_CLI_PENDING_Q].max_len);
	- tw_cli_dbg_printf(0, ctlr_handle, "", "complete %04d %04d",
	- ctlr->q_stats[TW_CLI_COMPLETE_Q].cur_len,
	- ctlr->q_stats[TW_CLI_COMPLETE_Q].max_len);
	- tw_cli_dbg_printf(0, ctlr_handle, "", "AEN queue head %d tail %d",
	- ctlr->aen_head, ctlr->aen_tail);
	-}
	-
	-/*
	- * Function name: tw_cl_reset_stats
	- * Description: Resets CL maintained statistics for the controller.
	- *
	- * Input: ctlr_handle-- controller handle
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cl_reset_stats(struct tw_cl_ctlr_handle *ctlr_handle)
	-{
	- struct tw_cli_ctlr_context *ctlr =
	- (struct tw_cli_ctlr_context *)(ctlr_handle->cl_ctlr_ctxt);
	-
	- tw_cli_dbg_printf(7, ctlr_handle, tw_osl_cur_func(), "entered");
	- ctlr->q_stats[TW_CLI_FREE_Q].max_len = 0;
	- ctlr->q_stats[TW_CLI_BUSY_Q].max_len = 0;
	- ctlr->q_stats[TW_CLI_PENDING_Q].max_len = 0;
	- ctlr->q_stats[TW_CLI_COMPLETE_Q].max_len = 0;
	-}
	-
	-/*
	- * Function name: tw_cli_print_req_info
	- * Description: Prints CL internal details of a given request.
	- *
	- * Input: req -- ptr to CL internal request context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_cl_print_req_info(struct tw_cl_req_handle *req_handle)
	-{
	- struct tw_cli_req_context *req = req_handle->cl_req_ctxt;
	- struct tw_cli_ctlr_context *ctlr = req->ctlr;
	- struct tw_cl_ctlr_handle *ctlr_handle = ctlr->ctlr_handle;
	- struct tw_cl_command_packet *cmd_pkt = req->cmd_pkt;
	- struct tw_cl_command_9k *cmd9k;
	- union tw_cl_command_7k *cmd7k;
	- TW_UINT8 *cdb;
	- TW_VOID *sgl;
	- TW_UINT32 sgl_entries;
	- TW_UINT32 i;
	-
	- tw_cli_dbg_printf(0, ctlr_handle, tw_osl_cur_func(),
	- "CL details for request:");
	- tw_cli_dbg_printf(0, ctlr_handle, tw_osl_cur_func(),
	- "req_handle = %p, ctlr = %p,\n"
	- "cmd_pkt = %p, cmd_pkt_phys = 0x%llx,\n"
	- "data = %p, length = 0x%x, data_phys = 0x%llx,\n"
	- "state = 0x%x, flags = 0x%x, error = 0x%x,\n"
	- "orig_req = %p, callback = %p, req_id = 0x%x,\n"
	- "next_req = %p, prev_req = %p",
	- req_handle, ctlr,
	- cmd_pkt, req->cmd_pkt_phys,
	- req->data, req->length, req->data_phys,
	- req->state, req->flags, req->error_code,
	- req->orig_req, req->tw_cli_callback, req->request_id,
	- req->link.next, req->link.prev);
	-
	- if (req->flags & TW_CLI_REQ_FLAGS_9K) {
	- cmd9k = &(cmd_pkt->command.cmd_pkt_9k);
	- sgl = cmd9k->sg_list;
	- sgl_entries = TW_CL_SWAP16(
	- GET_SGL_ENTRIES(cmd9k->lun_h4__sgl_entries));
	- tw_cli_dbg_printf(0, ctlr_handle, tw_osl_cur_func(),
	- "9K cmd: opcode = 0x%x, unit = 0x%x, req_id = 0x%x,\n"
	- "status = 0x%x, sgl_offset = 0x%x, sgl_entries = 0x%x",
	- GET_OPCODE(cmd9k->res__opcode),
	- cmd9k->unit,
	- TW_CL_SWAP16(GET_REQ_ID(cmd9k->lun_l4__req_id)),
	- cmd9k->status,
	- cmd9k->sgl_offset,
	- sgl_entries);
	-
	- cdb = (TW_UINT8 *)(cmd9k->cdb);
	- tw_cli_dbg_printf(0, ctlr_handle, tw_osl_cur_func(),
	- "CDB: %x %x %x %x %x %x %x %x"
	- "%x %x %x %x %x %x %x %x",
	- cdb[0], cdb[1], cdb[2], cdb[3],
	- cdb[4], cdb[5], cdb[6], cdb[7],
	- cdb[8], cdb[9], cdb[10], cdb[11],
	- cdb[12], cdb[13], cdb[14], cdb[15]);
	- } else {
	- cmd7k = &(cmd_pkt->command.cmd_pkt_7k);
	- sgl = cmd7k->param.sgl;
	- sgl_entries = (cmd7k->generic.size -
	- GET_SGL_OFF(cmd7k->generic.sgl_off__opcode)) /
	- ((ctlr->flags & TW_CL_64BIT_ADDRESSES) ? 3 : 2);
	- tw_cli_dbg_printf(0, ctlr_handle, tw_osl_cur_func(),
	- "7K cmd: opcode = 0x%x, sgl_offset = 0x%x,\n"
	- "size = 0x%x, req_id = 0x%x, unit = 0x%x,\n"
	- "status = 0x%x, flags = 0x%x, count = 0x%x",
	- GET_OPCODE(cmd7k->generic.sgl_off__opcode),
	- GET_SGL_OFF(cmd7k->generic.sgl_off__opcode),
	- cmd7k->generic.size,
	- TW_CL_SWAP16(cmd7k->generic.request_id),
	- GET_UNIT(cmd7k->generic.host_id__unit),
	- cmd7k->generic.status,
	- cmd7k->generic.flags,
	- TW_CL_SWAP16(cmd7k->generic.count));
	- }
	-
	- tw_cli_dbg_printf(0, ctlr_handle, tw_osl_cur_func(), "SG entries:");
	-
	- if (ctlr->flags & TW_CL_64BIT_ADDRESSES) {
	- struct tw_cl_sg_desc64 sgl64 = (struct tw_cl_sg_desc64 )sgl;
	-
	- for (i = 0; i < sgl_entries; i++) {
	- tw_cli_dbg_printf(0, ctlr_handle, tw_osl_cur_func(),
	- "0x%llx 0x%x",
	- sgl64[i].address, sgl64[i].length);
	- }
	- } else {
	- struct tw_cl_sg_desc32 sgl32 = (struct tw_cl_sg_desc32 )sgl;
	-
	- for (i = 0; i < sgl_entries; i++) {
	- tw_cli_dbg_printf(0, ctlr_handle, tw_osl_cur_func(),
	- "0x%x 0x%x",
	- sgl32[i].address, sgl32[i].length);
	- }
	- }
	-}
	-
	-#endif /* TW_OSL_DEBUG */
	Index: sys/dev/twa/tw_cl_share.h
	===================================================================
	--- sys/dev/twa/tw_cl_share.h
	+++ /dev/null
	@@ -1,535 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- * Modifications by: Manjunath Ranganathaiah
	- */
	-
	-#ifndef TW_CL_SHARE_H
	-
	-#define TW_CL_SHARE_H
	-
	-/*
	- * Macros, structures and functions shared between OSL and CL,
	- * and defined by CL.
	- */
	-
	-#define TW_CL_NULL ((TW_VOID *)0)
	-#define TW_CL_TRUE 1
	-#define TW_CL_FALSE 0
	-
	-#define TW_CL_VENDOR_ID 0x13C1 /* 3ware vendor id */
	-#define TW_CL_DEVICE_ID_9K 0x1002 /* 9000 PCI series device id */
	-#define TW_CL_DEVICE_ID_9K_X 0x1003 /* 9000 PCI-X series device id */
	-#define TW_CL_DEVICE_ID_9K_E 0x1004 /* 9000 PCIe series device id */
	-#define TW_CL_DEVICE_ID_9K_SA 0x1005 /* 9000 PCIe SAS series device id */
	-
	-#define TW_CL_BAR_TYPE_IO 1 /* I/O base address */
	-#define TW_CL_BAR_TYPE_MEM 2 /* memory base address */
	-#define TW_CL_BAR_TYPE_SBUF 3 /* SBUF base address */
	-
	-#ifdef TW_OSL_ENCLOSURE_SUPPORT
	-#define TW_CL_MAX_NUM_UNITS 65 /* max # of units we support
	- -- enclosure target id is 64 */
	-#else /* TW_OSL_ENCLOSURE_SUPPORT */
	-#define TW_CL_MAX_NUM_UNITS 32 /* max # of units we support */
	-#endif /* TW_OSL_ENCLOSURE_SUPPORT */
	-
	-#define TW_CL_MAX_NUM_LUNS 255 /* max # of LUN's we support */
	-#define TW_CL_MAX_IO_SIZE 0x20000 /* 128K */
	-
	-/*
	- * Though we can support 256 simultaneous requests, we advertise as capable
	- * of supporting only 255, since we want to keep one CL internal request
	- * context packet always available for internal requests.
	- */
	-#define TW_CL_MAX_SIMULTANEOUS_REQUESTS 256 /* max simult reqs supported */
	-
	-#define TW_CL_MAX_32BIT_SG_ELEMENTS 109 /* max 32-bit sg elements */
	-#define TW_CL_MAX_64BIT_SG_ELEMENTS 72 /* max 64-bit sg elements */
	-
	-/* Possible values of ctlr->flags */
	-#define TW_CL_64BIT_ADDRESSES (1<<0) /* 64 bit cmdpkt & SG addresses */
	-#define TW_CL_64BIT_SG_LENGTH (1<<1) /* 64 bit SG length */
	-#define TW_CL_START_CTLR_ONLY (1<<2) /* Start ctlr only */
	-#define TW_CL_STOP_CTLR_ONLY (1<<3) /* Stop ctlr only */
	-#define TW_CL_DEFERRED_INTR_USED (1<<5) /* OS Layer uses deferred intr */
	-
	-/* Possible error values from the Common Layer. */
	-#define TW_CL_ERR_REQ_SUCCESS 0
	-#define TW_CL_ERR_REQ_GENERAL_FAILURE (1<<0)
	-#define TW_CL_ERR_REQ_INVALID_TARGET (1<<1)
	-#define TW_CL_ERR_REQ_INVALID_LUN (1<<2)
	-#define TW_CL_ERR_REQ_SCSI_ERROR (1<<3)
	-#define TW_CL_ERR_REQ_AUTO_SENSE_VALID (1<<4)
	-#define TW_CL_ERR_REQ_BUS_RESET (1<<5)
	-#define TW_CL_ERR_REQ_UNABLE_TO_SUBMIT_COMMAND (1<<6)
	-
	-/* Possible values of req_pkt->flags */
	-#define TW_CL_REQ_RETRY_ON_BUSY (1<<0)
	-#define TW_CL_REQ_CALLBACK_FOR_SGLIST (1<<1)
	-
	-#define TW_CL_MESSAGE_SOURCE_CONTROLLER_ERROR 3
	-#define TW_CL_MESSAGE_SOURCE_CONTROLLER_EVENT 4
	-#define TW_CL_MESSAGE_SOURCE_COMMON_LAYER_ERROR 21
	-#define TW_CL_MESSAGE_SOURCE_COMMON_LAYER_EVENT 22
	-#define TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER 5
	-#define TW_CL_MESSAGE_SOURCE_FREEBSD_OS 8
	-#define TW_CL_MESSAGE_SOURCE_WINDOWS_DRIVER 7
	-#define TW_CL_MESSAGE_SOURCE_WINDOWS_OS 10
	-
	-#define TW_CL_SEVERITY_ERROR 0x1
	-#define TW_CL_SEVERITY_WARNING 0x2
	-#define TW_CL_SEVERITY_INFO 0x3
	-#define TW_CL_SEVERITY_DEBUG 0x4
	-
	-#define TW_CL_SEVERITY_ERROR_STRING "ERROR"
	-#define TW_CL_SEVERITY_WARNING_STRING "WARNING"
	-#define TW_CL_SEVERITY_INFO_STRING "INFO"
	-#define TW_CL_SEVERITY_DEBUG_STRING "DEBUG"
	-
	-/*
	- * Structure, a pointer to which is used as the controller handle in
	- * communications between the OS Layer and the Common Layer.
	- */
	-struct tw_cl_ctlr_handle {
	- TW_VOID osl_ctlr_ctxt; / OSL's ctlr context */
	- TW_VOID cl_ctlr_ctxt; / CL's ctlr context */
	-};
	-
	-/*
	- * Structure, a pointer to which is used as the request handle in
	- * communications between the OS Layer and the Common Layer.
	- */
	-struct tw_cl_req_handle {
	- TW_VOID osl_req_ctxt; / OSL's request context */
	- TW_VOID cl_req_ctxt; / CL's request context */
	- TW_UINT8 is_io; /* Only freeze/release simq for IOs */
	-};
	-
	-/* Structure used to describe SCSI requests to CL. */
	-struct tw_cl_scsi_req_packet {
	- TW_UINT32 unit; /* unit # to send cmd to */
	- TW_UINT32 lun; /* LUN to send cmd to */
	- TW_UINT8 cdb; / ptr to SCSI cdb */
	- TW_UINT32 cdb_len; /* # of valid cdb bytes */
	- TW_UINT32 sense_len; /* # of bytes of valid sense info */
	- TW_UINT8 sense_data; / ptr to sense data, if any */
	- TW_UINT32 scsi_status; /* SCSI status returned by fw */
	- TW_UINT32 sgl_entries; /* # of SG descriptors */
	- TW_UINT8 sg_list; / ptr to SG list */
	-};
	-
	-/* Structure used to describe pass through command packets to CL. */
	-struct tw_cl_passthru_req_packet {
	- TW_UINT8 cmd_pkt; / ptr to passthru cmd pkt */
	- TW_UINT32 cmd_pkt_length; /* size of cmd pkt */
	- TW_UINT32 sgl_entries; /* # of SG descriptors */
	- TW_UINT8 sg_list; / ptr to SG list */
	-};
	-
	-/* Request packet submitted to the Common Layer, by the OS Layer. */
	-struct tw_cl_req_packet {
	- TW_UINT32 cmd; /* Common Layer cmd */
	- TW_UINT32 flags; /* flags describing request */
	- TW_UINT32 status; /* Common Layer returned status */
	- TW_VOID (tw_osl_callback)(struct tw_cl_req_handle req_handle);
	- /* OSL routine to be called by CL on req completion */
	- TW_VOID (*tw_osl_sgl_callback)(
	- struct tw_cl_req_handle req_handle, TW_VOID sg_list,
	- TW_UINT32 *num_sgl_entries);
	- /* OSL callback to get SG list. */
	-
	- union {
	- struct tw_cl_scsi_req_packet scsi_req; /* SCSI req */
	- struct tw_cl_passthru_req_packet pt_req;/Passthru req/
	- } gen_req_pkt;
	-};
	-
	-#pragma pack(1)
	-/*
	- * Packet that describes an AEN/error generated by the controller,
	- * Common Layer, or even the OS Layer.
	- */
	-struct tw_cl_event_packet {
	- TW_UINT32 sequence_id;
	- TW_UINT32 time_stamp_sec;
	- TW_UINT16 aen_code;
	- TW_UINT8 severity;
	- TW_UINT8 retrieved;
	- TW_UINT8 repeat_count;
	- TW_UINT8 parameter_len;
	- TW_UINT8 parameter_data[98];
	- TW_UINT32 event_src;
	- TW_UINT8 severity_str[20];
	-};
	-#pragma pack()
	-
	-/* Structure to link 2 adjacent elements in a list. */
	-struct tw_cl_link {
	- struct tw_cl_link *next;
	- struct tw_cl_link *prev;
	-};
	-
	-#pragma pack(1)
	-/* Scatter/Gather list entry with 32 bit addresses. */
	-struct tw_cl_sg_desc32 {
	- TW_UINT32 address;
	- TW_UINT32 length;
	-};
	-
	-/* Scatter/Gather list entry with 64 bit addresses. */
	-struct tw_cl_sg_desc64 {
	- TW_UINT64 address;
	- TW_UINT32 length;
	-};
	-
	-#pragma pack()
	-
	-/* Byte swap functions. Valid only if running on big endian platforms. */
	-#ifdef TW_OSL_BIG_ENDIAN
	-
	-#define TW_CL_SWAP16_WITH_CAST(x) \
	- ((x << 8) \| (x >> 8))
	-
	-#define TW_CL_SWAP32_WITH_CAST(x) \
	- ((x << 24) \| ((x << 8) & (0xFF0000)) \| \
	- ((x >> 8) & (0xFF00)) \| (x >> 24))
	-
	-#define TW_CL_SWAP64_WITH_CAST(x) \
	- ((((TW_UINT64)(TW_CL_SWAP32(((TW_UINT32 *)(&(x)))[1]))) << 32) \|\
	- ((TW_UINT32)(TW_CL_SWAP32(((TW_UINT32 *)(&(x)))[0]))))
	-
	-#else /* TW_OSL_BIG_ENDIAN */
	-
	-#define TW_CL_SWAP16_WITH_CAST(x) x
	-#define TW_CL_SWAP32_WITH_CAST(x) x
	-#define TW_CL_SWAP64_WITH_CAST(x) x
	-
	-#endif /* TW_OSL_BIG_ENDIAN */
	-
	-#define TW_CL_SWAP16(x) TW_CL_SWAP16_WITH_CAST((TW_UINT16)(x))
	-#define TW_CL_SWAP32(x) TW_CL_SWAP32_WITH_CAST((TW_UINT32)(x))
	-#define TW_CL_SWAP64(x) TW_CL_SWAP64_WITH_CAST((TW_UINT64)(x))
	-
	-/* Queue manipulation functions. */
	-
	-/* Initialize a queue. */
	-#define TW_CL_Q_INIT(head) do { \
	- (head)->prev = (head)->next = head; \
	-} while (0)
	-
	-/* Insert an item at the head of the queue. */
	-#define TW_CL_Q_INSERT_HEAD(head, item) do { \
	- (item)->next = (head)->next; \
	- (item)->prev = head; \
	- (head)->next->prev = item; \
	- (head)->next = item; \
	-} while (0)
	-
	-/* Insert an item at the tail of the queue. */
	-#define TW_CL_Q_INSERT_TAIL(head, item) do { \
	- (item)->next = head; \
	- (item)->prev = (head)->prev; \
	- (head)->prev->next = item; \
	- (head)->prev = item; \
	-} while (0)
	-
	-/* Remove an item from the head of the queue. */
	-#define TW_CL_Q_REMOVE_ITEM(head, item) do { \
	- (item)->prev->next = (item)->next; \
	- (item)->next->prev = (item)->prev; \
	-} while (0)
	-
	-/* Retrieve the item at the head of the queue. */
	-#define TW_CL_Q_FIRST_ITEM(head) \
	- (((head)->next != head) ? ((head)->next) : TW_CL_NULL)
	-
	-/* Retrieve the item at the tail of the queue. */
	-#define TW_CL_Q_LAST_ITEM(head) \
	- (((head)->prev != head) ? ((head)->prev) : TW_CL_NULL)
	-
	-/* Retrieve the item next to a given item in the queue. */
	-#define TW_CL_Q_NEXT_ITEM(head, item) \
	- (((item)->next != head) ? ((item)->next) : TW_CL_NULL)
	-
	-/* Retrieve the item previous to a given item in the queue. */
	-#define TW_CL_Q_PREV_ITEM(head, item) \
	- (((item)->prev != head) ? ((item)->prev) : TW_CL_NULL)
	-
	-/* Determine the offset of a field from the head of the structure it is in. */
	-#define TW_CL_STRUCT_OFFSET(struct_type, field) \
	- (TW_INT8 )(&((struct_type )0)->field)
	-
	-/*
	- * Determine the address of the head of a structure, given the address of a
	- * field within it.
	- */
	-#define TW_CL_STRUCT_HEAD(addr, struct_type, field) \
	- (struct_type )((TW_INT8 )addr - \
	- TW_CL_STRUCT_OFFSET(struct_type, field))
	-
	-#ifndef TW_BUILDING_API
	-
	-#include "tw_osl_inline.h"
	-
	-/*
	- * The following are extern declarations of OS Layer defined functions called
	- * by the Common Layer. If any function has been defined as a macro in
	- * tw_osl_share.h, we will not make the extern declaration here.
	- */
	-
	-#ifndef tw_osl_breakpoint
	-/* Allows setting breakpoints in the CL code for debugging purposes. */
	-extern TW_VOID tw_osl_breakpoint(TW_VOID);
	-#endif
	-
	-#ifndef tw_osl_timeout
	-/* Start OS timeout() routine after controller reset sequence */
	-extern TW_VOID tw_osl_timeout(struct tw_cl_req_handle *req_handle);
	-#endif
	-
	-#ifndef tw_osl_untimeout
	-/* Stop OS timeout() routine during controller reset sequence */
	-extern TW_VOID tw_osl_untimeout(struct tw_cl_req_handle *req_handle);
	-#endif
	-
	-#ifndef tw_osl_cur_func
	-/* Text name of current function. */
	-extern TW_INT8 *tw_osl_cur_func(TW_VOID);
	-#endif
	-
	-#ifdef TW_OSL_DEBUG
	-#ifndef tw_osl_dbg_printf
	-/* Print to syslog/event log/debug console, as applicable. */
	-extern TW_INT32 tw_osl_dbg_printf(struct tw_cl_ctlr_handle *ctlr_handle,
	- const TW_INT8 *fmt, ...);
	-#endif
	-#endif /* TW_OSL_DEBUG */
	-
	-#ifndef tw_osl_delay
	-/* Cause a delay of usecs micro-seconds. */
	-extern TW_VOID tw_osl_delay(TW_INT32 usecs);
	-#endif
	-
	-#ifndef tw_osl_destroy_lock
	-/* Create/initialize a lock for CL's use. */
	-extern TW_VOID tw_osl_destroy_lock(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_LOCK_HANDLE *lock);
	-#endif
	-
	-#ifndef tw_osl_free_lock
	-/* Free a previously held lock. */
	-extern TW_VOID tw_osl_free_lock(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_LOCK_HANDLE *lock);
	-#endif
	-
	-#ifndef tw_osl_get_local_time
	-/* Get local time. */
	-extern TW_TIME tw_osl_get_local_time(TW_VOID);
	-#endif
	-
	-#ifndef tw_osl_get_lock
	-/* Acquire a lock. */
	-extern TW_VOID tw_osl_get_lock(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_LOCK_HANDLE *lock);
	-#endif
	-
	-#ifndef tw_osl_init_lock
	-/* Create/initialize a lock for CL's use. */
	-extern TW_VOID tw_osl_init_lock(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_INT8 lock_name, TW_LOCK_HANDLE lock);
	-#endif
	-
	-#ifndef tw_osl_memcpy
	-/* Copy 'size' bytes from 'src' to 'dest'. */
	-extern TW_VOID tw_osl_memcpy(TW_VOID src, TW_VOID dest, TW_INT32 size);
	-#endif
	-
	-#ifndef tw_osl_memzero
	-/* Zero 'size' bytes starting at 'addr'. */
	-extern TW_VOID tw_osl_memzero(TW_VOID *addr, TW_INT32 size);
	-#endif
	-
	-#ifndef tw_osl_notify_event
	-/* Notify OSL of a controller/CL (or even OSL) event. */
	-extern TW_VOID tw_osl_notify_event(struct tw_cl_ctlr_handle *ctlr_handle,
	- struct tw_cl_event_packet *event);
	-#endif
	-
	-#ifdef TW_OSL_PCI_CONFIG_ACCESSIBLE
	-#ifndef tw_osl_read_pci_config
	-/* Read 'size' bytes from 'offset' in the PCI config space. */
	-extern TW_UINT32 tw_osl_read_pci_config(
	- struct tw_cl_ctlr_handle *ctlr_handle, TW_INT32 offset, TW_INT32 size);
	-#endif
	-#endif /* TW_OSL_PCI_CONFIG_ACCESSIBLE */
	-
	-#ifndef tw_osl_read_reg
	-/* Read 'size' bytes at 'offset' from base address of this controller. */
	-extern TW_UINT32 tw_osl_read_reg(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_INT32 offset, TW_INT32 size);
	-#endif
	-
	-#ifndef tw_osl_scan_bus
	-/* Request OSL for a bus scan. */
	-extern TW_VOID tw_osl_scan_bus(struct tw_cl_ctlr_handle *ctlr_handle);
	-#endif
	-
	-#ifdef TW_OSL_CAN_SLEEP
	-#ifndef tw_osl_sleep
	-/* Sleep for 'timeout' ms or until woken up (by tw_osl_wakeup). */
	-extern TW_INT32 tw_osl_sleep(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_SLEEP_HANDLE *sleep_handle, TW_INT32 timeout);
	-#endif
	-#endif /* TW_OSL_CAN_SLEEP */
	-
	-#ifndef tw_osl_sprintf
	-/* Standard sprintf. */
	-extern TW_INT32 tw_osl_sprintf(TW_INT8 dest, const TW_INT8 fmt, ...);
	-#endif
	-
	-#ifndef tw_osl_strcpy
	-/* Copy string 'src' to 'dest'. */
	-extern TW_INT8 tw_osl_strcpy(TW_INT8 dest, TW_INT8 *src);
	-#endif
	-
	-#ifndef tw_osl_strlen
	-/* Return length of string pointed at by 'str'. */
	-extern TW_INT32 tw_osl_strlen(TW_VOID *str);
	-#endif
	-
	-#ifndef tw_osl_vsprintf
	-/* Standard vsprintf. */
	-extern TW_INT32 tw_osl_vsprintf(TW_INT8 dest, const TW_INT8 fmt, va_list ap);
	-#endif
	-
	-#ifdef TW_OSL_CAN_SLEEP
	-#ifndef tw_osl_wakeup
	-/* Wake up a thread sleeping by a call to tw_osl_sleep. */
	-extern TW_VOID tw_osl_wakeup(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_SLEEP_HANDLE *sleep_handle);
	-#endif
	-#endif /* TW_OSL_CAN_SLEEP */
	-
	-#ifdef TW_OSL_PCI_CONFIG_ACCESSIBLE
	-#ifndef tw_osl_write_pci_config
	-/* Write 'value' of 'size' bytes at 'offset' in the PCI config space. */
	-extern TW_VOID tw_osl_write_pci_config(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_INT32 offset, TW_INT32 value, TW_INT32 size);
	-#endif
	-#endif /* TW_OSL_PCI_CONFIG_ACCESSIBLE */
	-
	-#ifndef tw_osl_write_reg
	-/*
	- * Write 'value' of 'size' (max 4) bytes at 'offset' from base address of
	- * this controller.
	- */
	-extern TW_VOID tw_osl_write_reg(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_INT32 offset, TW_INT32 value, TW_INT32 size);
	-#endif
	-
	-/* Functions in the Common Layer */
	-
	-/* Creates and queues AEN's. Also notifies OS Layer. */
	-extern TW_VOID tw_cl_create_event(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_UINT8 queue_event, TW_UINT8 event_src, TW_UINT16 event_code,
	- TW_UINT8 severity, TW_UINT8 severity_str, TW_UINT8 event_desc,
	- TW_UINT8 *event_specific_desc, ...);
	-
	-/* Indicates whether a ctlr is supported by CL. */
	-extern TW_INT32 tw_cl_ctlr_supported(TW_INT32 vendor_id, TW_INT32 device_id);
	-
	-/* Submit a firmware cmd packet. */
	-extern TW_INT32 tw_cl_fw_passthru(struct tw_cl_ctlr_handle *ctlr_handle,
	- struct tw_cl_req_packet req_pkt, struct tw_cl_req_handle req_handle);
	-
	-/* Find out how much memory CL needs. */
	-extern TW_INT32 tw_cl_get_mem_requirements(
	- struct tw_cl_ctlr_handle *ctlr_handle, TW_UINT32 flags,
	- TW_INT32 device_id, TW_INT32 max_simult_reqs, TW_INT32 max_aens,
	- TW_UINT32 alignment, TW_UINT32 sg_size_factor,
	- TW_UINT32 non_dma_mem_size, TW_UINT32 dma_mem_size
	- );
	-
	-/* Return PCI BAR info. */
	-extern TW_INT32 tw_cl_get_pci_bar_info(TW_INT32 device_id, TW_INT32 bar_type,
	- TW_INT32 bar_num, TW_INT32 bar0_offset, TW_INT32 *bar_size);
	-
	-/* Initialize Common Layer for a given controller. */
	-extern TW_INT32 tw_cl_init_ctlr(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_UINT32 flags, TW_INT32 device_id, TW_INT32 max_simult_reqs,
	- TW_INT32 max_aens, TW_VOID non_dma_mem, TW_VOID dma_mem,
	- TW_UINT64 dma_mem_phys
	- );
	-
	-extern TW_VOID tw_cl_set_reset_needed(struct tw_cl_ctlr_handle *ctlr_handle);
	-extern TW_INT32 tw_cl_is_reset_needed(struct tw_cl_ctlr_handle *ctlr_handle);
	-extern TW_INT32 tw_cl_is_active(struct tw_cl_ctlr_handle *ctlr_handle);
	-
	-/* CL's interrupt handler. */
	-extern TW_INT32 tw_cl_interrupt(struct tw_cl_ctlr_handle *ctlr_handle);
	-
	-/* CL's ioctl handler. */
	-extern TW_INT32 tw_cl_ioctl(struct tw_cl_ctlr_handle *ctlr_handle,
	- u_long cmd, TW_VOID *buf);
	-
	-#ifdef TW_OSL_DEBUG
	-/* Print CL's state/statistics for a controller. */
	-extern TW_VOID tw_cl_print_ctlr_stats(struct tw_cl_ctlr_handle *ctlr_handle);
	-
	-/* Prints CL internal details of a given request. */
	-extern TW_VOID tw_cl_print_req_info(struct tw_cl_req_handle *req_handle);
	-#endif /* TW_OSL_DEBUG */
	-
	-/* Soft reset controller. */
	-extern TW_INT32 tw_cl_reset_ctlr(struct tw_cl_ctlr_handle *ctlr_handle);
	-
	-#ifdef TW_OSL_DEBUG
	-/* Reset CL's statistics for a controller. */
	-extern TW_VOID tw_cl_reset_stats(struct tw_cl_ctlr_handle *ctlr_handle);
	-#endif /* TW_OSL_DEBUG */
	-
	-/* Stop a controller. */
	-extern TW_INT32 tw_cl_shutdown_ctlr(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_UINT32 flags);
	-
	-/* Submit a SCSI I/O request. */
	-extern TW_INT32 tw_cl_start_io(struct tw_cl_ctlr_handle *ctlr_handle,
	- struct tw_cl_req_packet req_pkt, struct tw_cl_req_handle req_handle);
	-
	-#endif /* TW_BUILDING_API */
	-
	-#endif /* TW_CL_SHARE_H */
	Index: sys/dev/twa/tw_osl.h
	===================================================================
	--- sys/dev/twa/tw_osl.h
	+++ /dev/null
	@@ -1,298 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- * Modifications by: Manjunath Ranganathaiah
	- */
	-
	-#ifndef TW_OSL_H
	-
	-#define TW_OSL_H
	-
	-/*
	- * OS Layer internal macros, structures and functions.
	- */
	-
	-#define TW_OSLI_DEVICE_NAME "3ware 9000 series Storage Controller"
	-
	-#define TW_OSLI_MALLOC_CLASS M_TWA
	-#define TW_OSLI_MAX_NUM_REQUESTS TW_CL_MAX_SIMULTANEOUS_REQUESTS
	-/* Reserve two command packets. One for ioctls and one for AENs */
	-#define TW_OSLI_MAX_NUM_IOS (TW_OSLI_MAX_NUM_REQUESTS - 2)
	-#define TW_OSLI_MAX_NUM_AENS 0x100
	-
	-#ifdef PAE
	-#define TW_OSLI_DMA_BOUNDARY (1u << 31)
	-#else
	-#define TW_OSLI_DMA_BOUNDARY ((bus_size_t)((uint64_t)1 << 32))
	-#endif
	-
	-/* Possible values of req->state. */
	-#define TW_OSLI_REQ_STATE_INIT 0x0 /* being initialized */
	-#define TW_OSLI_REQ_STATE_BUSY 0x1 /* submitted to CL */
	-#define TW_OSLI_REQ_STATE_PENDING 0x2 /* in pending queue */
	-#define TW_OSLI_REQ_STATE_COMPLETE 0x3 /* completed by CL */
	-
	-/* Possible values of req->flags. */
	-#define TW_OSLI_REQ_FLAGS_DATA_IN (1<<0) /* read request */
	-#define TW_OSLI_REQ_FLAGS_DATA_OUT (1<<1) /* write request */
	-#define TW_OSLI_REQ_FLAGS_DATA_COPY_NEEDED (1<<2)/* data in ccb is misaligned,
	- have to copy to/from private buffer */
	-#define TW_OSLI_REQ_FLAGS_MAPPED (1<<3) /* request has been mapped */
	-#define TW_OSLI_REQ_FLAGS_IN_PROGRESS (1<<4) /* bus_dmamap_load returned
	- EINPROGRESS */
	-#define TW_OSLI_REQ_FLAGS_PASSTHRU (1<<5) /* pass through request */
	-#define TW_OSLI_REQ_FLAGS_SLEEPING (1<<6) /* owner sleeping on this cmd */
	-#define TW_OSLI_REQ_FLAGS_FAILED (1<<7) /* bus_dmamap_load() failed */
	-#define TW_OSLI_REQ_FLAGS_CCB (1<<8) /* req is ccb. */
	-
	-#ifdef TW_OSL_DEBUG
	-struct tw_osli_q_stats {
	- TW_UINT32 cur_len; /* current # of items in q */
	- TW_UINT32 max_len; /* max value reached by q_length */
	-};
	-#endif /* TW_OSL_DEBUG */
	-
	-/* Queues of OSL internal request context packets. */
	-#define TW_OSLI_FREE_Q 0 /* free q */
	-#define TW_OSLI_BUSY_Q 1 /* q of reqs submitted to CL */
	-#define TW_OSLI_Q_COUNT 2 /* total number of queues */
	-
	-/* Driver's request packet. */
	-struct tw_osli_req_context {
	- struct tw_cl_req_handle req_handle;/* tag to track req b/w OSL & CL */
	- struct mtx ioctl_wake_timeout_lock_handle;/* non-spin lock used to detect ioctl timeout */
	- struct mtx ioctl_wake_timeout_lock;/ ptr to above lock */
	- struct twa_softc ctlr; / ptr to OSL's controller context */
	- TW_VOID data; / ptr to data being passed to CL */
	- TW_UINT32 length; /* length of buf being passed to CL */
	- TW_UINT64 deadline;/* request timeout (in absolute time) */
	-
	- /*
	- * ptr to, and length of data passed to us from above, in case a buffer
	- * copy was done due to non-compliance to alignment requirements
	- */
	- TW_VOID *real_data;
	- TW_UINT32 real_length;
	-
	- TW_UINT32 state; /* request state */
	- TW_UINT32 flags; /* request flags */
	-
	- /* error encountered before request submission to CL */
	- TW_UINT32 error_code;
	-
	- /* ptr to orig req for use during callback */
	- TW_VOID *orig_req;
	-
	- struct tw_cl_link link; /* to link this request in a list */
	- bus_dmamap_t dma_map;/* DMA map for data */
	- struct tw_cl_req_packet req_pkt;/* req pkt understood by CL */
	-};
	-
	-/* Per-controller structure. */
	-struct twa_softc {
	- struct tw_cl_ctlr_handle ctlr_handle;
	- struct tw_osli_req_context *req_ctx_buf;
	-
	- /* Controller state. */
	- TW_UINT8 open;
	- TW_UINT32 flags;
	-
	- TW_INT32 device_id;
	- TW_UINT32 alignment;
	- TW_UINT32 sg_size_factor;
	-
	- TW_VOID *non_dma_mem;
	- TW_VOID *dma_mem;
	- TW_UINT64 dma_mem_phys;
	-
	- /* Request queues and arrays. */
	- struct tw_cl_link req_q_head[TW_OSLI_Q_COUNT];
	-
	- struct task deferred_intr_callback;/* taskqueue function */
	- struct mtx io_lock_handle;/* general purpose lock */
	- struct mtx io_lock;/ ptr to general purpose lock */
	- struct mtx q_lock_handle; /* queue manipulation lock */
	- struct mtx q_lock;/ ptr to queue manipulation lock */
	- struct mtx sim_lock_handle;/* sim lock shared with cam */
	- struct mtx sim_lock;/ ptr to sim lock */
	-
	- struct callout watchdog_callout[2]; /* For command timeout */
	- TW_UINT32 watchdog_index;
	-
	-#ifdef TW_OSL_DEBUG
	- struct tw_osli_q_stats q_stats[TW_OSLI_Q_COUNT];/* queue statistics */
	-#endif /* TW_OSL_DEBUG */
	-
	- device_t bus_dev; /* bus device */
	- struct cdev ctrl_dev; / control device */
	- struct resource reg_res; / register interface window */
	- TW_INT32 reg_res_id; /* register resource id */
	- bus_space_handle_t bus_handle; /* bus space handle */
	- bus_space_tag_t bus_tag; /* bus space tag */
	- bus_dma_tag_t parent_tag; /* parent DMA tag */
	- bus_dma_tag_t cmd_tag; /* DMA tag for CL's DMA'able mem */
	- bus_dma_tag_t dma_tag; /* data buffer DMA tag */
	- bus_dma_tag_t ioctl_tag; /* ioctl data buffer DMA tag */
	- bus_dmamap_t cmd_map; /* DMA map for CL's DMA'able mem */
	- bus_dmamap_t ioctl_map; /* DMA map for ioctl data buffers */
	- struct resource irq_res; / interrupt resource */
	- TW_INT32 irq_res_id; /* register resource id */
	- TW_VOID intr_handle; / interrupt handle */
	-
	- struct sysctl_ctx_list sysctl_ctxt; /* sysctl context */
	- struct sysctl_oid sysctl_tree; / sysctl oid */
	-
	- struct cam_sim sim; / sim for this controller */
	- struct cam_path path; / peripheral, path, tgt, lun
	- associated with this controller */
	-};
	-
	-/*
	- * Queue primitives.
	- */
	-
	-#ifdef TW_OSL_DEBUG
	-
	-#define TW_OSLI_Q_INIT(sc, q_type) do { \
	- (sc)->q_stats[q_type].cur_len = 0; \
	- (sc)->q_stats[q_type].max_len = 0; \
	-} while(0)
	-
	-#define TW_OSLI_Q_INSERT(sc, q_type) do { \
	- struct tw_osli_q_stats *q_stats = &((sc)->q_stats[q_type]); \
	- \
	- if (++(q_stats->cur_len) > q_stats->max_len) \
	- q_stats->max_len = q_stats->cur_len; \
	-} while(0)
	-
	-#define TW_OSLI_Q_REMOVE(sc, q_type) \
	- (sc)->q_stats[q_type].cur_len--
	-
	-#else /* TW_OSL_DEBUG */
	-
	-#define TW_OSLI_Q_INIT(sc, q_index)
	-#define TW_OSLI_Q_INSERT(sc, q_index)
	-#define TW_OSLI_Q_REMOVE(sc, q_index)
	-
	-#endif /* TW_OSL_DEBUG */
	-
	-/* Initialize a queue of requests. */
	-static __inline TW_VOID
	-tw_osli_req_q_init(struct twa_softc *sc, TW_UINT8 q_type)
	-{
	- TW_CL_Q_INIT(&(sc->req_q_head[q_type]));
	- TW_OSLI_Q_INIT(sc, q_type);
	-}
	-
	-/* Insert the given request at the head of the given queue (q_type). */
	-static __inline TW_VOID
	-tw_osli_req_q_insert_head(struct tw_osli_req_context *req, TW_UINT8 q_type)
	-{
	- mtx_lock_spin(req->ctlr->q_lock);
	- TW_CL_Q_INSERT_HEAD(&(req->ctlr->req_q_head[q_type]), &(req->link));
	- TW_OSLI_Q_INSERT(req->ctlr, q_type);
	- mtx_unlock_spin(req->ctlr->q_lock);
	-}
	-
	-/* Insert the given request at the tail of the given queue (q_type). */
	-static __inline TW_VOID
	-tw_osli_req_q_insert_tail(struct tw_osli_req_context *req, TW_UINT8 q_type)
	-{
	- mtx_lock_spin(req->ctlr->q_lock);
	- TW_CL_Q_INSERT_TAIL(&(req->ctlr->req_q_head[q_type]), &(req->link));
	- TW_OSLI_Q_INSERT(req->ctlr, q_type);
	- mtx_unlock_spin(req->ctlr->q_lock);
	-}
	-
	-/* Remove and return the request at the head of the given queue (q_type). */
	-static __inline struct tw_osli_req_context *
	-tw_osli_req_q_remove_head(struct twa_softc *sc, TW_UINT8 q_type)
	-{
	- struct tw_osli_req_context *req = NULL;
	- struct tw_cl_link *link;
	-
	- mtx_lock_spin(sc->q_lock);
	- if ((link = TW_CL_Q_FIRST_ITEM(&(sc->req_q_head[q_type]))) !=
	- TW_CL_NULL) {
	- req = TW_CL_STRUCT_HEAD(link,
	- struct tw_osli_req_context, link);
	- TW_CL_Q_REMOVE_ITEM(&(sc->req_q_head[q_type]), &(req->link));
	- TW_OSLI_Q_REMOVE(sc, q_type);
	- }
	- mtx_unlock_spin(sc->q_lock);
	- return(req);
	-}
	-
	-/* Remove the given request from the given queue (q_type). */
	-static __inline TW_VOID
	-tw_osli_req_q_remove_item(struct tw_osli_req_context *req, TW_UINT8 q_type)
	-{
	- mtx_lock_spin(req->ctlr->q_lock);
	- TW_CL_Q_REMOVE_ITEM(&(req->ctlr->req_q_head[q_type]), &(req->link));
	- TW_OSLI_Q_REMOVE(req->ctlr, q_type);
	- mtx_unlock_spin(req->ctlr->q_lock);
	-}
	-
	-#ifdef TW_OSL_DEBUG
	-
	-extern TW_INT32 TW_DEBUG_LEVEL_FOR_OSL;
	-
	-#define tw_osli_dbg_dprintf(dbg_level, sc, fmt, args...) \
	- if (dbg_level <= TW_DEBUG_LEVEL_FOR_OSL) \
	- device_printf(sc->bus_dev, "%s: " fmt "\n", \
	- __func__, ##args)
	-
	-#define tw_osli_dbg_printf(dbg_level, fmt, args...) \
	- if (dbg_level <= TW_DEBUG_LEVEL_FOR_OSL) \
	- printf("%s: " fmt "\n", __func__, ##args)
	-
	-#else /* TW_OSL_DEBUG */
	-
	-#define tw_osli_dbg_dprintf(dbg_level, sc, fmt, args...)
	-#define tw_osli_dbg_printf(dbg_level, fmt, args...)
	-
	-#endif /* TW_OSL_DEBUG */
	-
	-/* For regular printing. */
	-#define twa_printf(sc, fmt, args...) \
	- device_printf(((struct twa_softc *)(sc))->bus_dev, fmt, ##args)
	-
	-/* For printing in the "consistent error reporting" format. */
	-#define tw_osli_printf(sc, err_specific_desc, args...) \
	- device_printf((sc)->bus_dev, \
	- "%s: (0x%02X: 0x%04X): %s: " err_specific_desc "\n", ##args)
	-
	-#endif /* TW_OSL_H */
	Index: sys/dev/twa/tw_osl_cam.c
	===================================================================
	--- sys/dev/twa/tw_osl_cam.c
	+++ /dev/null
	@@ -1,663 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- * Modifications by: Manjunath Ranganathaiah
	- */
	-
	-/*
	- * FreeBSD CAM related functions.
	- */
	-
	-#include <dev/twa/tw_osl_includes.h>
	-
	-#include <cam/cam.h>
	-#include <cam/cam_ccb.h>
	-#include <cam/cam_sim.h>
	-#include <cam/cam_xpt_sim.h>
	-#include <cam/cam_debug.h>
	-#include <cam/cam_periph.h>
	-
	-#include <cam/scsi/scsi_all.h>
	-#include <cam/scsi/scsi_message.h>
	-
	-static TW_VOID twa_action(struct cam_sim sim, union ccb ccb);
	-static TW_VOID twa_poll(struct cam_sim *sim);
	-
	-static TW_INT32 tw_osli_execute_scsi(struct tw_osli_req_context *req,
	- union ccb *ccb);
	-
	-/*
	- * Function name: tw_osli_cam_attach
	- * Description: Attaches the driver to CAM.
	- *
	- * Input: sc -- ptr to OSL internal ctlr context
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_osli_cam_attach(struct twa_softc *sc)
	-{
	- struct cam_devq *devq;
	-
	- tw_osli_dbg_dprintf(3, sc, "entered");
	-
	- /*
	- * Create the device queue for our SIM.
	- */
	- if ((devq = cam_simq_alloc(TW_OSLI_MAX_NUM_IOS)) == NULL) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2100,
	- "Failed to create SIM device queue",
	- ENOMEM);
	- return(ENOMEM);
	- }
	-
	- /*
	- * Create a SIM entry. Though we can support TW_OSLI_MAX_NUM_REQUESTS
	- * simultaneous requests, we claim to be able to handle only
	- * TW_OSLI_MAX_NUM_IOS (two less), so that we always have a request
	- * packet available to service ioctls and AENs.
	- */
	- tw_osli_dbg_dprintf(3, sc, "Calling cam_sim_alloc");
	- sc->sim = cam_sim_alloc(twa_action, twa_poll, "twa", sc,
	- device_get_unit(sc->bus_dev), sc->sim_lock,
	- TW_OSLI_MAX_NUM_IOS, 1, devq);
	- if (sc->sim == NULL) {
	- cam_simq_free(devq);
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2101,
	- "Failed to create a SIM entry",
	- ENOMEM);
	- return(ENOMEM);
	- }
	-
	- /*
	- * Register the bus.
	- */
	- tw_osli_dbg_dprintf(3, sc, "Calling xpt_bus_register");
	- mtx_lock(sc->sim_lock);
	- if (xpt_bus_register(sc->sim, sc->bus_dev, 0) != CAM_SUCCESS) {
	- cam_sim_free(sc->sim, TRUE);
	- sc->sim = NULL; /* so cam_detach will not try to free it */
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2102,
	- "Failed to register the bus",
	- ENXIO);
	- mtx_unlock(sc->sim_lock);
	- return(ENXIO);
	- }
	-
	- tw_osli_dbg_dprintf(3, sc, "Calling xpt_create_path");
	- if (xpt_create_path(&sc->path, NULL,
	- cam_sim_path(sc->sim),
	- CAM_TARGET_WILDCARD,
	- CAM_LUN_WILDCARD) != CAM_REQ_CMP) {
	- xpt_bus_deregister(cam_sim_path (sc->sim));
	- /* Passing TRUE to cam_sim_free will free the devq as well. */
	- cam_sim_free(sc->sim, TRUE);
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2103,
	- "Failed to create path",
	- ENXIO);
	- mtx_unlock(sc->sim_lock);
	- return(ENXIO);
	- }
	- mtx_unlock(sc->sim_lock);
	-
	- tw_osli_dbg_dprintf(3, sc, "exiting");
	- return(0);
	-}
	-
	-/*
	- * Function name: tw_osli_cam_detach
	- * Description: Detaches the driver from CAM.
	- *
	- * Input: sc -- ptr to OSL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osli_cam_detach(struct twa_softc *sc)
	-{
	- tw_osli_dbg_dprintf(3, sc, "entered");
	-
	- mtx_lock(sc->sim_lock);
	-
	- if (sc->path)
	- xpt_free_path(sc->path);
	- if (sc->sim) {
	- xpt_bus_deregister(cam_sim_path(sc->sim));
	- /* Passing TRUE to cam_sim_free will free the devq as well. */
	- cam_sim_free(sc->sim, TRUE);
	- }
	- /* It's ok have 1 hold count while destroying the mutex */
	- mtx_destroy(sc->sim_lock);
	-}
	-
	-/*
	- * Function name: tw_osli_execute_scsi
	- * Description: Build a fw cmd, based on a CAM style ccb, and
	- * send it down.
	- *
	- * Input: req -- ptr to OSL internal request context
	- * ccb -- ptr to CAM style ccb
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_osli_execute_scsi(struct tw_osli_req_context req, union ccb ccb)
	-{
	- struct twa_softc *sc = req->ctlr;
	- struct tw_cl_req_packet *req_pkt;
	- struct tw_cl_scsi_req_packet *scsi_req;
	- struct ccb_hdr *ccb_h = &(ccb->ccb_h);
	- struct ccb_scsiio *csio = &(ccb->csio);
	- TW_INT32 error;
	-
	- tw_osli_dbg_dprintf(10, sc, "SCSI I/O request 0x%x",
	- csio->cdb_io.cdb_bytes[0]);
	-
	- if (ccb_h->target_id >= TW_CL_MAX_NUM_UNITS) {
	- tw_osli_dbg_dprintf(3, sc, "Invalid target. PTL = %x %x %jx",
	- ccb_h->path_id, ccb_h->target_id,
	- (uintmax_t)ccb_h->target_lun);
	- ccb_h->status \|= CAM_TID_INVALID;
	- xpt_done(ccb);
	- return(1);
	- }
	- if (ccb_h->target_lun >= TW_CL_MAX_NUM_LUNS) {
	- tw_osli_dbg_dprintf(3, sc, "Invalid lun. PTL = %x %x %jx",
	- ccb_h->path_id, ccb_h->target_id,
	- (uintmax_t)ccb_h->target_lun);
	- ccb_h->status \|= CAM_LUN_INVALID;
	- xpt_done(ccb);
	- return(1);
	- }
	-
	- if(ccb_h->flags & CAM_CDB_PHYS) {
	- tw_osli_printf(sc, "",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2105,
	- "Physical CDB address!");
	- ccb_h->status = CAM_REQ_INVALID;
	- xpt_done(ccb);
	- return(1);
	- }
	-
	- /*
	- * We are going to work on this request. Mark it as enqueued (though
	- * we don't actually queue it...)
	- */
	- ccb_h->status \|= CAM_SIM_QUEUED;
	-
	- if((ccb_h->flags & CAM_DIR_MASK) != CAM_DIR_NONE) {
	- if(ccb_h->flags & CAM_DIR_IN)
	- req->flags \|= TW_OSLI_REQ_FLAGS_DATA_IN;
	- else
	- req->flags \|= TW_OSLI_REQ_FLAGS_DATA_OUT;
	- }
	-
	- /* Build the CL understood request packet for SCSI cmds. */
	- req_pkt = &req->req_pkt;
	- req_pkt->status = 0;
	- req_pkt->tw_osl_callback = tw_osl_complete_io;
	- scsi_req = &(req_pkt->gen_req_pkt.scsi_req);
	- scsi_req->unit = ccb_h->target_id;
	- scsi_req->lun = ccb_h->target_lun;
	- scsi_req->sense_len = 0;
	- scsi_req->sense_data = (TW_UINT8 *)(&csio->sense_data);
	- scsi_req->scsi_status = 0;
	- if(ccb_h->flags & CAM_CDB_POINTER)
	- scsi_req->cdb = csio->cdb_io.cdb_ptr;
	- else
	- scsi_req->cdb = csio->cdb_io.cdb_bytes;
	- scsi_req->cdb_len = csio->cdb_len;
	-
	- if (csio->dxfer_len > TW_CL_MAX_IO_SIZE) {
	- tw_osli_printf(sc, "size = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2106,
	- "I/O size too big",
	- csio->dxfer_len);
	- ccb_h->status = CAM_REQ_TOO_BIG;
	- ccb_h->status &= ~CAM_SIM_QUEUED;
	- xpt_done(ccb);
	- return(1);
	- }
	- if ((ccb_h->flags & CAM_DATA_MASK) == CAM_DATA_VADDR) {
	- if ((req->length = csio->dxfer_len) != 0) {
	- req->data = csio->data_ptr;
	- scsi_req->sgl_entries = 1;
	- }
	- } else
	- req->flags \|= TW_OSLI_REQ_FLAGS_CCB;
	- req->deadline = tw_osl_get_local_time() + (ccb_h->timeout / 1000);
	-
	- /*
	- * twa_map_load_data_callback will fill in the SGL,
	- * and submit the I/O.
	- */
	- error = tw_osli_map_request(req);
	- if ((error) && (req->flags & TW_OSLI_REQ_FLAGS_FAILED)) {
	- req->deadline = 0;
	- ccb_h->status = CAM_REQ_CMP_ERR;
	- ccb_h->status &= ~CAM_SIM_QUEUED;
	- xpt_done(ccb);
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: twa_action
	- * Description: Driver entry point for CAM's use.
	- *
	- * Input: sim -- sim corresponding to the ctlr
	- * ccb -- ptr to CAM request
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-twa_action(struct cam_sim sim, union ccb ccb)
	-{
	- struct twa_softc sc = (struct twa_softc )cam_sim_softc(sim);
	- struct ccb_hdr *ccb_h = &(ccb->ccb_h);
	-
	- switch (ccb_h->func_code) {
	- case XPT_SCSI_IO: /* SCSI I/O */
	- {
	- struct tw_osli_req_context *req;
	-
	- req = tw_osli_get_request(sc);
	- if (req == NULL) {
	- tw_osli_dbg_dprintf(2, sc, "Cannot get request pkt.");
	- /*
	- * Freeze the simq to maintain ccb ordering. The next
	- * ccb that gets completed will unfreeze the simq.
	- */
	- ccb_h->status &= ~CAM_SIM_QUEUED;
	- ccb_h->status \|= CAM_REQUEUE_REQ;
	- xpt_done(ccb);
	- break;
	- }
	-
	- if ((tw_cl_is_reset_needed(&(req->ctlr->ctlr_handle)))) {
	- ccb_h->status &= ~CAM_SIM_QUEUED;
	- ccb_h->status \|= CAM_REQUEUE_REQ;
	- xpt_done(ccb);
	- tw_osli_req_q_insert_tail(req, TW_OSLI_FREE_Q);
	- break;
	- }
	-
	- req->req_handle.osl_req_ctxt = req;
	- req->req_handle.is_io = TW_CL_TRUE;
	- req->orig_req = ccb;
	- if (tw_osli_execute_scsi(req, ccb))
	- tw_osli_req_q_insert_tail(req, TW_OSLI_FREE_Q);
	- break;
	- }
	-
	- case XPT_ABORT:
	- tw_osli_dbg_dprintf(2, sc, "Abort request.");
	- ccb_h->status = CAM_UA_ABORT;
	- xpt_done(ccb);
	- break;
	-
	- case XPT_RESET_BUS:
	- tw_cl_create_event(&(sc->ctlr_handle), TW_CL_FALSE,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2108, 0x3, TW_CL_SEVERITY_INFO_STRING,
	- "Received Reset Bus request from CAM",
	- " ");
	-
	- tw_cl_set_reset_needed(&(sc->ctlr_handle));
	- ccb_h->status = CAM_REQ_CMP;
	- xpt_done(ccb);
	- break;
	-
	- case XPT_SET_TRAN_SETTINGS:
	- tw_osli_dbg_dprintf(3, sc, "XPT_SET_TRAN_SETTINGS");
	-
	- /*
	- * This command is not supported, since it's very specific
	- * to SCSI, and we are doing ATA.
	- */
	- ccb_h->status = CAM_FUNC_NOTAVAIL;
	- xpt_done(ccb);
	- break;
	-
	- case XPT_GET_TRAN_SETTINGS:
	- {
	- struct ccb_trans_settings *cts = &ccb->cts;
	- struct ccb_trans_settings_scsi *scsi =
	- &cts->proto_specific.scsi;
	- struct ccb_trans_settings_spi *spi =
	- &cts->xport_specific.spi;
	-
	- cts->protocol = PROTO_SCSI;
	- cts->protocol_version = SCSI_REV_2;
	- cts->transport = XPORT_SPI;
	- cts->transport_version = 2;
	-
	- spi->valid = CTS_SPI_VALID_DISC;
	- spi->flags = CTS_SPI_FLAGS_DISC_ENB;
	- scsi->valid = CTS_SCSI_VALID_TQ;
	- scsi->flags = CTS_SCSI_FLAGS_TAG_ENB;
	- tw_osli_dbg_dprintf(3, sc, "XPT_GET_TRAN_SETTINGS");
	- ccb_h->status = CAM_REQ_CMP;
	- xpt_done(ccb);
	- break;
	- }
	-
	- case XPT_CALC_GEOMETRY:
	- tw_osli_dbg_dprintf(3, sc, "XPT_CALC_GEOMETRY");
	- cam_calc_geometry(&ccb->ccg, 1/* extended */);
	- xpt_done(ccb);
	- break;
	-
	- case XPT_PATH_INQ: /* Path inquiry -- get twa properties */
	- {
	- struct ccb_pathinq *path_inq = &ccb->cpi;
	-
	- tw_osli_dbg_dprintf(3, sc, "XPT_PATH_INQ request");
	-
	- path_inq->version_num = 1;
	- path_inq->hba_inquiry = 0;
	- path_inq->target_sprt = 0;
	- path_inq->hba_misc = 0;
	- path_inq->hba_eng_cnt = 0;
	- path_inq->max_target = TW_CL_MAX_NUM_UNITS;
	- path_inq->max_lun = TW_CL_MAX_NUM_LUNS - 1;
	- path_inq->unit_number = cam_sim_unit(sim);
	- path_inq->bus_id = cam_sim_bus(sim);
	- path_inq->initiator_id = TW_CL_MAX_NUM_UNITS;
	- path_inq->base_transfer_speed = 100000;
	- strlcpy(path_inq->sim_vid, "FreeBSD", SIM_IDLEN);
	- strlcpy(path_inq->hba_vid, "3ware", HBA_IDLEN);
	- strlcpy(path_inq->dev_name, cam_sim_name(sim), DEV_IDLEN);
	- path_inq->transport = XPORT_SPI;
	- path_inq->transport_version = 2;
	- path_inq->protocol = PROTO_SCSI;
	- path_inq->protocol_version = SCSI_REV_2;
	- path_inq->maxio = TW_CL_MAX_IO_SIZE;
	- ccb_h->status = CAM_REQ_CMP;
	- xpt_done(ccb);
	- break;
	- }
	-
	- default:
	- tw_osli_dbg_dprintf(3, sc, "func_code = %x", ccb_h->func_code);
	- ccb_h->status = CAM_REQ_INVALID;
	- xpt_done(ccb);
	- break;
	- }
	-}
	-
	-/*
	- * Function name: twa_poll
	- * Description: Driver entry point called when interrupts are not
	- * available.
	- *
	- * Input: sim -- sim corresponding to the controller
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-twa_poll(struct cam_sim *sim)
	-{
	- struct twa_softc sc = (struct twa_softc )(cam_sim_softc(sim));
	-
	- tw_osli_dbg_dprintf(3, sc, "entering; sc = %p", sc);
	- tw_cl_interrupt(&(sc->ctlr_handle));
	- tw_osli_dbg_dprintf(3, sc, "exiting; sc = %p", sc);
	-}
	-
	-/*
	- * Function name: tw_osli_request_bus_scan
	- * Description: Requests CAM for a scan of the bus.
	- *
	- * Input: sc -- ptr to per ctlr structure
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_osli_request_bus_scan(struct twa_softc *sc)
	-{
	- union ccb *ccb;
	-
	- tw_osli_dbg_dprintf(3, sc, "entering");
	-
	- /* If we get here before sc->sim is initialized, return an error. */
	- if (!(sc->sim))
	- return(ENXIO);
	- if ((ccb = xpt_alloc_ccb()) == NULL)
	- return(ENOMEM);
	- mtx_lock(sc->sim_lock);
	- if (xpt_create_path(&ccb->ccb_h.path, NULL, cam_sim_path(sc->sim),
	- CAM_TARGET_WILDCARD, CAM_LUN_WILDCARD) != CAM_REQ_CMP) {
	- xpt_free_ccb(ccb);
	- mtx_unlock(sc->sim_lock);
	- return(EIO);
	- }
	-
	- xpt_rescan(ccb);
	- mtx_unlock(sc->sim_lock);
	- return(0);
	-}
	-
	-/*
	- * Function name: tw_osli_disallow_new_requests
	- * Description: Calls the appropriate CAM function, so as to freeze
	- * the flow of new requests from CAM to this controller.
	- *
	- * Input: sc -- ptr to OSL internal ctlr context
	- * req_handle -- ptr to request handle sent by OSL.
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osli_disallow_new_requests(struct twa_softc *sc,
	- struct tw_cl_req_handle *req_handle)
	-{
	- /* Only freeze/release the simq for IOs */
	- if (req_handle->is_io) {
	- struct tw_osli_req_context *req = req_handle->osl_req_ctxt;
	- union ccb ccb = (union ccb )(req->orig_req);
	-
	- xpt_freeze_simq(sc->sim, 1);
	- ccb->ccb_h.status \|= CAM_RELEASE_SIMQ;
	- }
	-}
	-
	-/*
	- * Function name: tw_osl_timeout
	- * Description: Call to timeout().
	- *
	- * Input: req_handle -- ptr to request handle sent by OSL.
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osl_timeout(struct tw_cl_req_handle *req_handle)
	-{
	- struct tw_osli_req_context *req = req_handle->osl_req_ctxt;
	- union ccb ccb = (union ccb )(req->orig_req);
	- struct ccb_hdr *ccb_h = &(ccb->ccb_h);
	-
	- req->deadline = tw_osl_get_local_time() + (ccb_h->timeout / 1000);
	-}
	-
	-/*
	- * Function name: tw_osl_untimeout
	- * Description: Inverse of call to timeout().
	- *
	- * Input: req_handle -- ptr to request handle sent by OSL.
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osl_untimeout(struct tw_cl_req_handle *req_handle)
	-{
	- struct tw_osli_req_context *req = req_handle->osl_req_ctxt;
	-
	- req->deadline = 0;
	-}
	-
	-/*
	- * Function name: tw_osl_scan_bus
	- * Description: CL calls this function to request for a bus scan.
	- *
	- * Input: ctlr_handle -- ptr to controller handle
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osl_scan_bus(struct tw_cl_ctlr_handle *ctlr_handle)
	-{
	- struct twa_softc *sc = ctlr_handle->osl_ctlr_ctxt;
	- TW_INT32 error;
	-
	- if ((error = tw_osli_request_bus_scan(sc)))
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2109,
	- "Bus scan request to CAM failed",
	- error);
	-}
	-
	-/*
	- * Function name: tw_osl_complete_io
	- * Description: Called to complete CAM scsi requests.
	- *
	- * Input: req_handle -- ptr to request handle
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osl_complete_io(struct tw_cl_req_handle *req_handle)
	-{
	- struct tw_osli_req_context *req = req_handle->osl_req_ctxt;
	- struct tw_cl_req_packet *req_pkt =
	- (struct tw_cl_req_packet *)(&req->req_pkt);
	- struct tw_cl_scsi_req_packet *scsi_req;
	- struct twa_softc *sc = req->ctlr;
	- union ccb ccb = (union ccb )(req->orig_req);
	-
	- tw_osli_dbg_dprintf(10, sc, "entering");
	-
	- if (req->state != TW_OSLI_REQ_STATE_BUSY)
	- tw_osli_printf(sc, "request = %p, status = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x210A,
	- "Unposted command completed!!",
	- req, req->state);
	-
	- /*
	- * Remove request from the busy queue. Just mark it complete.
	- * There's no need to move it into the complete queue as we are
	- * going to be done with it right now.
	- */
	- req->state = TW_OSLI_REQ_STATE_COMPLETE;
	- tw_osli_req_q_remove_item(req, TW_OSLI_BUSY_Q);
	-
	- tw_osli_unmap_request(req);
	-
	- req->deadline = 0;
	- if (req->error_code) {
	- /* This request never got submitted to the firmware. */
	- if (req->error_code == EBUSY) {
	- /*
	- * Cmd queue is full, or the Common Layer is out of
	- * resources. The simq will already have been frozen.
	- * When this ccb gets completed will unfreeze the simq.
	- */
	- ccb->ccb_h.status \|= CAM_REQUEUE_REQ;
	- }
	- else if (req->error_code == EFBIG)
	- ccb->ccb_h.status = CAM_REQ_TOO_BIG;
	- else
	- ccb->ccb_h.status = CAM_REQ_CMP_ERR;
	- } else {
	- scsi_req = &(req_pkt->gen_req_pkt.scsi_req);
	- if (req_pkt->status == TW_CL_ERR_REQ_SUCCESS)
	- ccb->ccb_h.status = CAM_REQ_CMP;
	- else {
	- if (req_pkt->status & TW_CL_ERR_REQ_INVALID_TARGET)
	- ccb->ccb_h.status \|= CAM_SEL_TIMEOUT;
	- else if (req_pkt->status & TW_CL_ERR_REQ_INVALID_LUN)
	- ccb->ccb_h.status \|= CAM_DEV_NOT_THERE;
	- else if (req_pkt->status & TW_CL_ERR_REQ_SCSI_ERROR)
	- ccb->ccb_h.status \|= CAM_SCSI_STATUS_ERROR;
	- else if (req_pkt->status & TW_CL_ERR_REQ_BUS_RESET)
	- ccb->ccb_h.status \|= (CAM_REQUEUE_REQ \| CAM_SCSI_BUS_RESET);
	- /*
	- * If none of the above errors occurred, simply
	- * mark completion error.
	- */
	- if (ccb->ccb_h.status == 0)
	- ccb->ccb_h.status = CAM_REQ_CMP_ERR;
	-
	- if (req_pkt->status & TW_CL_ERR_REQ_AUTO_SENSE_VALID) {
	- ccb->csio.sense_len = scsi_req->sense_len;
	- ccb->ccb_h.status \|= CAM_AUTOSNS_VALID;
	- }
	- }
	-
	- ccb->csio.scsi_status = scsi_req->scsi_status;
	- }
	-
	- ccb->ccb_h.status &= ~CAM_SIM_QUEUED;
	- mtx_lock(sc->sim_lock);
	- xpt_done(ccb);
	- mtx_unlock(sc->sim_lock);
	- if (! req->error_code)
	- /* twa_action will free the request otherwise */
	- tw_osli_req_q_insert_tail(req, TW_OSLI_FREE_Q);
	-}
	Index: sys/dev/twa/tw_osl_externs.h
	===================================================================
	--- sys/dev/twa/tw_osl_externs.h
	+++ /dev/null
	@@ -1,86 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_OSL_EXTERNS_H
	-
	-#define TW_OSL_EXTERNS_H
	-
	-/*
	- * Data structures and functions global to the OS Layer.
	- */
	-
	-/* External data structures. */
	-
	-extern int mp_ncpus;
	-
	-/* Functions in tw_osl_freebsd.c */
	-
	-/* Build a firmware passthru cmd pkt, and submit it to CL. */
	-extern TW_INT32 tw_osli_fw_passthru(struct twa_softc sc, TW_INT8 buf);
	-
	-/* Get an OSL internal request context packet. */
	-extern struct tw_osli_req_context tw_osli_get_request(struct twa_softc sc);
	-
	-/* Map data to DMA'able memory. */
	-extern TW_INT32 tw_osli_map_request(struct tw_osli_req_context *req);
	-
	-/* Undo mapping. */
	-extern TW_VOID tw_osli_unmap_request(struct tw_osli_req_context *req);
	-
	-/* Functions in tw_osl_cam.c */
	-
	-/* Attach to CAM. */
	-extern TW_INT32 tw_osli_cam_attach(struct twa_softc *sc);
	-
	-/* Detach from CAM. */
	-extern TW_VOID tw_osli_cam_detach(struct twa_softc *sc);
	-
	-/* Request CAM for a bus scan. */
	-extern TW_INT32 tw_osli_request_bus_scan(struct twa_softc *sc);
	-
	-/* Freeze ccb flow from CAM. */
	-extern TW_VOID tw_osli_disallow_new_requests(struct twa_softc *sc,
	- struct tw_cl_req_handle *req_handle);
	-
	-/* OSL's completion routine for SCSI I/O's. */
	-extern TW_VOID tw_osl_complete_io(struct tw_cl_req_handle *req_handle);
	-
	-/* OSL's completion routine for passthru requests. */
	-extern TW_VOID tw_osl_complete_passthru(struct tw_cl_req_handle *req_handle);
	-
	-#endif /* TW_OSL_EXTERNS_H */
	Index: sys/dev/twa/tw_osl_freebsd.c
	===================================================================
	--- sys/dev/twa/tw_osl_freebsd.c
	+++ /dev/null
	@@ -1,1661 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * Copyright (c) 2000 Michael Smith
	- * Copyright (c) 2000 BSDi
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- */
	-
	-#include <sys/cdefs.h>
	-__FBSDID("$FreeBSD$");
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- * Modifications by: Manjunath Ranganathaiah
	- */
	-
	-/*
	- * FreeBSD specific functions not related to CAM, and other
	- * miscellaneous functions.
	- */
	-
	-#include <dev/twa/tw_osl_includes.h>
	-#include <dev/twa/tw_cl_fwif.h>
	-#include <dev/twa/tw_cl_ioctl.h>
	-#include <dev/twa/tw_osl_ioctl.h>
	-
	-#ifdef TW_OSL_DEBUG
	-TW_INT32 TW_DEBUG_LEVEL_FOR_OSL = TW_OSL_DEBUG;
	-TW_INT32 TW_OSL_DEBUG_LEVEL_FOR_CL = TW_OSL_DEBUG;
	-#endif /* TW_OSL_DEBUG */
	-
	-static MALLOC_DEFINE(TW_OSLI_MALLOC_CLASS, "twa_commands", "twa commands");
	-
	-static d_open_t twa_open;
	-static d_close_t twa_close;
	-static d_ioctl_t twa_ioctl;
	-
	-static struct cdevsw twa_cdevsw = {
	- .d_version = D_VERSION,
	- .d_open = twa_open,
	- .d_close = twa_close,
	- .d_ioctl = twa_ioctl,
	- .d_name = "twa",
	-};
	-
	-static devclass_t twa_devclass;
	-
	-/*
	- * Function name: twa_open
	- * Description: Called when the controller is opened.
	- * Simply marks the controller as open.
	- *
	- * Input: dev -- control device corresponding to the ctlr
	- * flags -- mode of open
	- * fmt -- device type (character/block etc.)
	- * proc -- current process
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-static TW_INT32
	-twa_open(struct cdev dev, TW_INT32 flags, TW_INT32 fmt, struct thread proc)
	-{
	- struct twa_softc sc = (struct twa_softc )(dev->si_drv1);
	-
	- tw_osli_dbg_dprintf(5, sc, "entered");
	- sc->open = TW_CL_TRUE;
	- return(0);
	-}
	-
	-/*
	- * Function name: twa_close
	- * Description: Called when the controller is closed.
	- * Simply marks the controller as not open.
	- *
	- * Input: dev -- control device corresponding to the ctlr
	- * flags -- mode of corresponding open
	- * fmt -- device type (character/block etc.)
	- * proc -- current process
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-static TW_INT32
	-twa_close(struct cdev dev, TW_INT32 flags, TW_INT32 fmt, struct thread proc)
	-{
	- struct twa_softc sc = (struct twa_softc )(dev->si_drv1);
	-
	- tw_osli_dbg_dprintf(5, sc, "entered");
	- sc->open = TW_CL_FALSE;
	- return(0);
	-}
	-
	-/*
	- * Function name: twa_ioctl
	- * Description: Called when an ioctl is posted to the controller.
	- * Handles any OS Layer specific cmds, passes the rest
	- * on to the Common Layer.
	- *
	- * Input: dev -- control device corresponding to the ctlr
	- * cmd -- ioctl cmd
	- * buf -- ptr to buffer in kernel memory, which is
	- * a copy of the input buffer in user-space
	- * flags -- mode of corresponding open
	- * proc -- current process
	- * Output: buf -- ptr to buffer in kernel memory, which will
	- * be copied to the output buffer in user-space
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-static TW_INT32
	-twa_ioctl(struct cdev dev, u_long cmd, caddr_t buf, TW_INT32 flags, struct thread proc)
	-{
	- struct twa_softc sc = (struct twa_softc )(dev->si_drv1);
	- TW_INT32 error;
	-
	- tw_osli_dbg_dprintf(5, sc, "entered");
	-
	- switch (cmd) {
	- case TW_OSL_IOCTL_FIRMWARE_PASS_THROUGH:
	- tw_osli_dbg_dprintf(6, sc, "ioctl: fw_passthru");
	- error = tw_osli_fw_passthru(sc, (TW_INT8 *)buf);
	- break;
	-
	- case TW_OSL_IOCTL_SCAN_BUS:
	- /* Request CAM for a bus scan. */
	- tw_osli_dbg_dprintf(6, sc, "ioctl: scan bus");
	- error = tw_osli_request_bus_scan(sc);
	- break;
	-
	- default:
	- tw_osli_dbg_dprintf(6, sc, "ioctl: 0x%lx", cmd);
	- error = tw_cl_ioctl(&sc->ctlr_handle, cmd, buf);
	- break;
	- }
	- return(error);
	-}
	-
	-static TW_INT32 twa_probe(device_t dev);
	-static TW_INT32 twa_attach(device_t dev);
	-static TW_INT32 twa_detach(device_t dev);
	-static TW_INT32 twa_shutdown(device_t dev);
	-static TW_VOID twa_busdma_lock(TW_VOID *lock_arg, bus_dma_lock_op_t op);
	-static TW_VOID twa_pci_intr(TW_VOID *arg);
	-static TW_VOID twa_watchdog(TW_VOID *arg);
	-int twa_setup_intr(struct twa_softc *sc);
	-int twa_teardown_intr(struct twa_softc *sc);
	-
	-static TW_INT32 tw_osli_alloc_mem(struct twa_softc *sc);
	-static TW_VOID tw_osli_free_resources(struct twa_softc *sc);
	-
	-static TW_VOID twa_map_load_data_callback(TW_VOID *arg,
	- bus_dma_segment_t *segs, TW_INT32 nsegments, TW_INT32 error);
	-static TW_VOID twa_map_load_callback(TW_VOID *arg,
	- bus_dma_segment_t *segs, TW_INT32 nsegments, TW_INT32 error);
	-
	-static device_method_t twa_methods[] = {
	- /* Device interface */
	- DEVMETHOD(device_probe, twa_probe),
	- DEVMETHOD(device_attach, twa_attach),
	- DEVMETHOD(device_detach, twa_detach),
	- DEVMETHOD(device_shutdown, twa_shutdown),
	-
	- DEVMETHOD_END
	-};
	-
	-static driver_t twa_pci_driver = {
	- "twa",
	- twa_methods,
	- sizeof(struct twa_softc)
	-};
	-
	-DRIVER_MODULE(twa, pci, twa_pci_driver, twa_devclass, 0, 0);
	-MODULE_DEPEND(twa, cam, 1, 1, 1);
	-MODULE_DEPEND(twa, pci, 1, 1, 1);
	-
	-/*
	- * Function name: twa_probe
	- * Description: Called at driver load time. Claims 9000 ctlrs.
	- *
	- * Input: dev -- bus device corresponding to the ctlr
	- * Output: None
	- * Return value: <= 0 -- success
	- * > 0 -- failure
	- */
	-static TW_INT32
	-twa_probe(device_t dev)
	-{
	- static TW_UINT8 first_ctlr = 1;
	-
	- tw_osli_dbg_printf(3, "entered");
	-
	- if (tw_cl_ctlr_supported(pci_get_vendor(dev), pci_get_device(dev))) {
	- device_set_desc(dev, TW_OSLI_DEVICE_NAME);
	- /* Print the driver version only once. */
	- if (first_ctlr) {
	- printf("3ware device driver for 9000 series storage "
	- "controllers, version: %s\n",
	- TW_OSL_DRIVER_VERSION_STRING);
	- first_ctlr = 0;
	- }
	- return(0);
	- }
	- return(ENXIO);
	-}
	-
	-int twa_setup_intr(struct twa_softc *sc)
	-{
	- int error = 0;
	-
	- if (!(sc->intr_handle) && (sc->irq_res)) {
	- error = bus_setup_intr(sc->bus_dev, sc->irq_res,
	- INTR_TYPE_CAM \| INTR_MPSAFE,
	- NULL, twa_pci_intr,
	- sc, &sc->intr_handle);
	- }
	- return( error );
	-}
	-
	-int twa_teardown_intr(struct twa_softc *sc)
	-{
	- int error = 0;
	-
	- if ((sc->intr_handle) && (sc->irq_res)) {
	- error = bus_teardown_intr(sc->bus_dev,
	- sc->irq_res, sc->intr_handle);
	- sc->intr_handle = NULL;
	- }
	- return( error );
	-}
	-
	-/*
	- * Function name: twa_attach
	- * Description: Allocates pci resources; updates sc; adds a node to the
	- * sysctl tree to expose the driver version; makes calls
	- * (to the Common Layer) to initialize ctlr, and to
	- * attach to CAM.
	- *
	- * Input: dev -- bus device corresponding to the ctlr
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-static TW_INT32
	-twa_attach(device_t dev)
	-{
	- struct twa_softc *sc = device_get_softc(dev);
	- TW_INT32 bar_num;
	- TW_INT32 bar0_offset;
	- TW_INT32 bar_size;
	- TW_INT32 error;
	-
	- tw_osli_dbg_dprintf(3, sc, "entered");
	-
	- sc->ctlr_handle.osl_ctlr_ctxt = sc;
	-
	- /* Initialize the softc structure. */
	- sc->bus_dev = dev;
	- sc->device_id = pci_get_device(dev);
	-
	- /* Initialize the mutexes right here. */
	- sc->io_lock = &(sc->io_lock_handle);
	- mtx_init(sc->io_lock, "tw_osl_io_lock", NULL, MTX_SPIN);
	- sc->q_lock = &(sc->q_lock_handle);
	- mtx_init(sc->q_lock, "tw_osl_q_lock", NULL, MTX_SPIN);
	- sc->sim_lock = &(sc->sim_lock_handle);
	- mtx_init(sc->sim_lock, "tw_osl_sim_lock", NULL, MTX_DEF \| MTX_RECURSE);
	-
	- sysctl_ctx_init(&sc->sysctl_ctxt);
	- sc->sysctl_tree = SYSCTL_ADD_NODE(&sc->sysctl_ctxt,
	- SYSCTL_STATIC_CHILDREN(_hw), OID_AUTO, device_get_nameunit(dev),
	- CTLFLAG_RD \| CTLFLAG_MPSAFE, 0, "");
	- if (sc->sysctl_tree == NULL) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2000,
	- "Cannot add sysctl tree node",
	- ENXIO);
	- return(ENXIO);
	- }
	- SYSCTL_ADD_STRING(&sc->sysctl_ctxt, SYSCTL_CHILDREN(sc->sysctl_tree),
	- OID_AUTO, "driver_version", CTLFLAG_RD,
	- TW_OSL_DRIVER_VERSION_STRING, 0, "TWA driver version");
	-
	- /* Force the busmaster enable bit on, in case the BIOS forgot. */
	- pci_enable_busmaster(dev);
	-
	- /* Allocate the PCI register window. */
	- if ((error = tw_cl_get_pci_bar_info(sc->device_id, TW_CL_BAR_TYPE_MEM,
	- &bar_num, &bar0_offset, &bar_size))) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x201F,
	- "Can't get PCI BAR info",
	- error);
	- tw_osli_free_resources(sc);
	- return(error);
	- }
	- sc->reg_res_id = PCIR_BARS + bar0_offset;
	- if ((sc->reg_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY,
	- &(sc->reg_res_id), RF_ACTIVE))
	- == NULL) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2002,
	- "Can't allocate register window",
	- ENXIO);
	- tw_osli_free_resources(sc);
	- return(ENXIO);
	- }
	- sc->bus_tag = rman_get_bustag(sc->reg_res);
	- sc->bus_handle = rman_get_bushandle(sc->reg_res);
	-
	- /* Allocate and register our interrupt. */
	- sc->irq_res_id = 0;
	- if ((sc->irq_res = bus_alloc_resource_any(sc->bus_dev, SYS_RES_IRQ,
	- &(sc->irq_res_id),
	- RF_SHAREABLE \| RF_ACTIVE)) == NULL) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2003,
	- "Can't allocate interrupt",
	- ENXIO);
	- tw_osli_free_resources(sc);
	- return(ENXIO);
	- }
	- if ((error = twa_setup_intr(sc))) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2004,
	- "Can't set up interrupt",
	- error);
	- tw_osli_free_resources(sc);
	- return(error);
	- }
	-
	- if ((error = tw_osli_alloc_mem(sc))) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2005,
	- "Memory allocation failure",
	- error);
	- tw_osli_free_resources(sc);
	- return(error);
	- }
	-
	- /* Initialize the Common Layer for this controller. */
	- if ((error = tw_cl_init_ctlr(&sc->ctlr_handle, sc->flags, sc->device_id,
	- TW_OSLI_MAX_NUM_REQUESTS, TW_OSLI_MAX_NUM_AENS,
	- sc->non_dma_mem, sc->dma_mem,
	- sc->dma_mem_phys
	- ))) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2006,
	- "Failed to initialize Common Layer/controller",
	- error);
	- tw_osli_free_resources(sc);
	- return(error);
	- }
	-
	- /* Create the control device. */
	- sc->ctrl_dev = make_dev(&twa_cdevsw, device_get_unit(sc->bus_dev),
	- UID_ROOT, GID_OPERATOR, S_IRUSR \| S_IWUSR,
	- "twa%d", device_get_unit(sc->bus_dev));
	- sc->ctrl_dev->si_drv1 = sc;
	-
	- if ((error = tw_osli_cam_attach(sc))) {
	- tw_osli_free_resources(sc);
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2007,
	- "Failed to initialize CAM",
	- error);
	- return(error);
	- }
	-
	- sc->watchdog_index = 0;
	- callout_init(&(sc->watchdog_callout[0]), 1);
	- callout_init(&(sc->watchdog_callout[1]), 1);
	- callout_reset(&(sc->watchdog_callout[0]), 5*hz, twa_watchdog, &sc->ctlr_handle);
	- gone_in_dev(dev, 13, "twa(4) removed");
	-
	- return(0);
	-}
	-
	-static TW_VOID
	-twa_watchdog(TW_VOID *arg)
	-{
	- struct tw_cl_ctlr_handle *ctlr_handle =
	- (struct tw_cl_ctlr_handle *)arg;
	- struct twa_softc *sc = ctlr_handle->osl_ctlr_ctxt;
	- int i;
	- int i_need_a_reset = 0;
	- int driver_is_active = 0;
	- int my_watchdog_was_pending = 1234;
	- TW_UINT64 current_time;
	- struct tw_osli_req_context *my_req;
	-
	-//==============================================================================
	- current_time = (TW_UINT64) (tw_osl_get_local_time());
	-
	- for (i = 0; i < TW_OSLI_MAX_NUM_REQUESTS; i++) {
	- my_req = &(sc->req_ctx_buf[i]);
	-
	- if ((my_req->state == TW_OSLI_REQ_STATE_BUSY) &&
	- (my_req->deadline) &&
	- (my_req->deadline < current_time)) {
	- tw_cl_set_reset_needed(ctlr_handle);
	-#ifdef TW_OSL_DEBUG
	- device_printf((sc)->bus_dev, "Request %d timed out! d = %llu, c = %llu\n", i, my_req->deadline, current_time);
	-#else /* TW_OSL_DEBUG */
	- device_printf((sc)->bus_dev, "Request %d timed out!\n", i);
	-#endif /* TW_OSL_DEBUG */
	- break;
	- }
	- }
	-//==============================================================================
	-
	- i_need_a_reset = tw_cl_is_reset_needed(ctlr_handle);
	-
	- i = (int) ((sc->watchdog_index++) & 1);
	-
	- driver_is_active = tw_cl_is_active(ctlr_handle);
	-
	- if (i_need_a_reset) {
	-#ifdef TW_OSL_DEBUG
	- device_printf((sc)->bus_dev, "Watchdog rescheduled in 70 seconds\n");
	-#endif /* TW_OSL_DEBUG */
	- my_watchdog_was_pending =
	- callout_reset(&(sc->watchdog_callout[i]), 70*hz, twa_watchdog, &sc->ctlr_handle);
	- tw_cl_reset_ctlr(ctlr_handle);
	-#ifdef TW_OSL_DEBUG
	- device_printf((sc)->bus_dev, "Watchdog reset completed!\n");
	-#endif /* TW_OSL_DEBUG */
	- } else if (driver_is_active) {
	- my_watchdog_was_pending =
	- callout_reset(&(sc->watchdog_callout[i]), 5*hz, twa_watchdog, &sc->ctlr_handle);
	- }
	-#ifdef TW_OSL_DEBUG
	- if (i_need_a_reset \|\| my_watchdog_was_pending)
	- device_printf((sc)->bus_dev, "i_need_a_reset = %d, "
	- "driver_is_active = %d, my_watchdog_was_pending = %d\n",
	- i_need_a_reset, driver_is_active, my_watchdog_was_pending);
	-#endif /* TW_OSL_DEBUG */
	-}
	-
	-/*
	- * Function name: tw_osli_alloc_mem
	- * Description: Allocates memory needed both by CL and OSL.
	- *
	- * Input: sc -- OSL internal controller context
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-static TW_INT32
	-tw_osli_alloc_mem(struct twa_softc *sc)
	-{
	- struct tw_osli_req_context *req;
	- TW_UINT32 max_sg_elements;
	- TW_UINT32 non_dma_mem_size;
	- TW_UINT32 dma_mem_size;
	- TW_INT32 error;
	- TW_INT32 i;
	-
	- tw_osli_dbg_dprintf(3, sc, "entered");
	-
	- sc->flags \|= (sizeof(bus_addr_t) == 8) ? TW_CL_64BIT_ADDRESSES : 0;
	- sc->flags \|= (sizeof(bus_size_t) == 8) ? TW_CL_64BIT_SG_LENGTH : 0;
	-
	- max_sg_elements = (sizeof(bus_addr_t) == 8) ?
	- TW_CL_MAX_64BIT_SG_ELEMENTS : TW_CL_MAX_32BIT_SG_ELEMENTS;
	-
	- if ((error = tw_cl_get_mem_requirements(&sc->ctlr_handle, sc->flags,
	- sc->device_id, TW_OSLI_MAX_NUM_REQUESTS, TW_OSLI_MAX_NUM_AENS,
	- &(sc->alignment), &(sc->sg_size_factor),
	- &non_dma_mem_size, &dma_mem_size
	- ))) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2008,
	- "Can't get Common Layer's memory requirements",
	- error);
	- return(error);
	- }
	-
	- if ((sc->non_dma_mem = malloc(non_dma_mem_size, TW_OSLI_MALLOC_CLASS,
	- M_WAITOK)) == NULL) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2009,
	- "Can't allocate non-dma memory",
	- ENOMEM);
	- return(ENOMEM);
	- }
	-
	- /* Create the parent dma tag. */
	- if (bus_dma_tag_create(bus_get_dma_tag(sc->bus_dev), /* parent */
	- sc->alignment, /* alignment */
	- TW_OSLI_DMA_BOUNDARY, /* boundary */
	- BUS_SPACE_MAXADDR, /* lowaddr */
	- BUS_SPACE_MAXADDR, /* highaddr */
	- NULL, NULL, /* filter, filterarg */
	- TW_CL_MAX_IO_SIZE, /* maxsize */
	- max_sg_elements, /* nsegments */
	- TW_CL_MAX_IO_SIZE, /* maxsegsize */
	- 0, /* flags */
	- NULL, /* lockfunc */
	- NULL, /* lockfuncarg */
	- &sc->parent_tag /* tag */)) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x200A,
	- "Can't allocate parent DMA tag",
	- ENOMEM);
	- return(ENOMEM);
	- }
	-
	- /* Create a dma tag for Common Layer's DMA'able memory (dma_mem). */
	- if (bus_dma_tag_create(sc->parent_tag, /* parent */
	- sc->alignment, /* alignment */
	- 0, /* boundary */
	- BUS_SPACE_MAXADDR, /* lowaddr */
	- BUS_SPACE_MAXADDR, /* highaddr */
	- NULL, NULL, /* filter, filterarg */
	- dma_mem_size, /* maxsize */
	- 1, /* nsegments */
	- BUS_SPACE_MAXSIZE, /* maxsegsize */
	- 0, /* flags */
	- NULL, /* lockfunc */
	- NULL, /* lockfuncarg */
	- &sc->cmd_tag /* tag */)) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x200B,
	- "Can't allocate DMA tag for Common Layer's "
	- "DMA'able memory",
	- ENOMEM);
	- return(ENOMEM);
	- }
	-
	- if (bus_dmamem_alloc(sc->cmd_tag, &sc->dma_mem,
	- BUS_DMA_NOWAIT, &sc->cmd_map)) {
	- /* Try a second time. */
	- if (bus_dmamem_alloc(sc->cmd_tag, &sc->dma_mem,
	- BUS_DMA_NOWAIT, &sc->cmd_map)) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x200C,
	- "Can't allocate DMA'able memory for the"
	- "Common Layer",
	- ENOMEM);
	- return(ENOMEM);
	- }
	- }
	-
	- bus_dmamap_load(sc->cmd_tag, sc->cmd_map, sc->dma_mem,
	- dma_mem_size, twa_map_load_callback,
	- &sc->dma_mem_phys, 0);
	-
	- /*
	- * Create a dma tag for data buffers; size will be the maximum
	- * possible I/O size (128kB).
	- */
	- if (bus_dma_tag_create(sc->parent_tag, /* parent */
	- sc->alignment, /* alignment */
	- 0, /* boundary */
	- BUS_SPACE_MAXADDR, /* lowaddr */
	- BUS_SPACE_MAXADDR, /* highaddr */
	- NULL, NULL, /* filter, filterarg */
	- TW_CL_MAX_IO_SIZE, /* maxsize */
	- max_sg_elements, /* nsegments */
	- TW_CL_MAX_IO_SIZE, /* maxsegsize */
	- BUS_DMA_ALLOCNOW, /* flags */
	- twa_busdma_lock, /* lockfunc */
	- sc->io_lock, /* lockfuncarg */
	- &sc->dma_tag /* tag */)) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x200F,
	- "Can't allocate DMA tag for data buffers",
	- ENOMEM);
	- return(ENOMEM);
	- }
	-
	- /*
	- * Create a dma tag for ioctl data buffers; size will be the maximum
	- * possible I/O size (128kB).
	- */
	- if (bus_dma_tag_create(sc->parent_tag, /* parent */
	- sc->alignment, /* alignment */
	- 0, /* boundary */
	- BUS_SPACE_MAXADDR, /* lowaddr */
	- BUS_SPACE_MAXADDR, /* highaddr */
	- NULL, NULL, /* filter, filterarg */
	- TW_CL_MAX_IO_SIZE, /* maxsize */
	- max_sg_elements, /* nsegments */
	- TW_CL_MAX_IO_SIZE, /* maxsegsize */
	- BUS_DMA_ALLOCNOW, /* flags */
	- twa_busdma_lock, /* lockfunc */
	- sc->io_lock, /* lockfuncarg */
	- &sc->ioctl_tag /* tag */)) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2010,
	- "Can't allocate DMA tag for ioctl data buffers",
	- ENOMEM);
	- return(ENOMEM);
	- }
	-
	- /* Create just one map for all ioctl request data buffers. */
	- if (bus_dmamap_create(sc->ioctl_tag, 0, &sc->ioctl_map)) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2011,
	- "Can't create ioctl map",
	- ENOMEM);
	- return(ENOMEM);
	- }
	-
	- /* Initialize request queues. */
	- tw_osli_req_q_init(sc, TW_OSLI_FREE_Q);
	- tw_osli_req_q_init(sc, TW_OSLI_BUSY_Q);
	-
	- if ((sc->req_ctx_buf = (struct tw_osli_req_context *)
	- malloc((sizeof(struct tw_osli_req_context) *
	- TW_OSLI_MAX_NUM_REQUESTS),
	- TW_OSLI_MALLOC_CLASS, M_WAITOK)) == NULL) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2012,
	- "Failed to allocate request packets",
	- ENOMEM);
	- return(ENOMEM);
	- }
	- bzero(sc->req_ctx_buf,
	- sizeof(struct tw_osli_req_context) * TW_OSLI_MAX_NUM_REQUESTS);
	-
	- for (i = 0; i < TW_OSLI_MAX_NUM_REQUESTS; i++) {
	- req = &(sc->req_ctx_buf[i]);
	- req->ctlr = sc;
	- if (bus_dmamap_create(sc->dma_tag, 0, &req->dma_map)) {
	- tw_osli_printf(sc, "request # = %d, error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2013,
	- "Can't create dma map",
	- i, ENOMEM);
	- return(ENOMEM);
	- }
	-
	- /* Initialize the ioctl wakeup/ timeout mutex */
	- req->ioctl_wake_timeout_lock = &(req->ioctl_wake_timeout_lock_handle);
	- mtx_init(req->ioctl_wake_timeout_lock, "tw_ioctl_wake_timeout_lock", NULL, MTX_DEF);
	-
	- /* Insert request into the free queue. */
	- tw_osli_req_q_insert_tail(req, TW_OSLI_FREE_Q);
	- }
	-
	- return(0);
	-}
	-
	-/*
	- * Function name: tw_osli_free_resources
	- * Description: Performs clean-up at the time of going down.
	- *
	- * Input: sc -- ptr to OSL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-static TW_VOID
	-tw_osli_free_resources(struct twa_softc *sc)
	-{
	- struct tw_osli_req_context *req;
	- TW_INT32 error = 0;
	-
	- tw_osli_dbg_dprintf(3, sc, "entered");
	-
	- /* Detach from CAM */
	- tw_osli_cam_detach(sc);
	-
	- if (sc->req_ctx_buf)
	- while ((req = tw_osli_req_q_remove_head(sc, TW_OSLI_FREE_Q)) !=
	- NULL) {
	- mtx_destroy(req->ioctl_wake_timeout_lock);
	-
	- if ((error = bus_dmamap_destroy(sc->dma_tag,
	- req->dma_map)))
	- tw_osli_dbg_dprintf(1, sc,
	- "dmamap_destroy(dma) returned %d",
	- error);
	- }
	-
	- if ((sc->ioctl_tag) && (sc->ioctl_map))
	- if ((error = bus_dmamap_destroy(sc->ioctl_tag, sc->ioctl_map)))
	- tw_osli_dbg_dprintf(1, sc,
	- "dmamap_destroy(ioctl) returned %d", error);
	-
	- /* Free all memory allocated so far. */
	- if (sc->req_ctx_buf)
	- free(sc->req_ctx_buf, TW_OSLI_MALLOC_CLASS);
	-
	- if (sc->non_dma_mem)
	- free(sc->non_dma_mem, TW_OSLI_MALLOC_CLASS);
	-
	- if (sc->dma_mem) {
	- bus_dmamap_unload(sc->cmd_tag, sc->cmd_map);
	- bus_dmamem_free(sc->cmd_tag, sc->dma_mem,
	- sc->cmd_map);
	- }
	- if (sc->cmd_tag)
	- if ((error = bus_dma_tag_destroy(sc->cmd_tag)))
	- tw_osli_dbg_dprintf(1, sc,
	- "dma_tag_destroy(cmd) returned %d", error);
	-
	- if (sc->dma_tag)
	- if ((error = bus_dma_tag_destroy(sc->dma_tag)))
	- tw_osli_dbg_dprintf(1, sc,
	- "dma_tag_destroy(dma) returned %d", error);
	-
	- if (sc->ioctl_tag)
	- if ((error = bus_dma_tag_destroy(sc->ioctl_tag)))
	- tw_osli_dbg_dprintf(1, sc,
	- "dma_tag_destroy(ioctl) returned %d", error);
	-
	- if (sc->parent_tag)
	- if ((error = bus_dma_tag_destroy(sc->parent_tag)))
	- tw_osli_dbg_dprintf(1, sc,
	- "dma_tag_destroy(parent) returned %d", error);
	-
	- /* Disconnect the interrupt handler. */
	- if ((error = twa_teardown_intr(sc)))
	- tw_osli_dbg_dprintf(1, sc,
	- "teardown_intr returned %d", error);
	-
	- if (sc->irq_res != NULL)
	- if ((error = bus_release_resource(sc->bus_dev,
	- SYS_RES_IRQ, sc->irq_res_id, sc->irq_res)))
	- tw_osli_dbg_dprintf(1, sc,
	- "release_resource(irq) returned %d", error);
	-
	- /* Release the register window mapping. */
	- if (sc->reg_res != NULL)
	- if ((error = bus_release_resource(sc->bus_dev,
	- SYS_RES_MEMORY, sc->reg_res_id, sc->reg_res)))
	- tw_osli_dbg_dprintf(1, sc,
	- "release_resource(io) returned %d", error);
	-
	- /* Destroy the control device. */
	- if (sc->ctrl_dev != (struct cdev *)NULL)
	- destroy_dev(sc->ctrl_dev);
	-
	- if ((error = sysctl_ctx_free(&sc->sysctl_ctxt)))
	- tw_osli_dbg_dprintf(1, sc,
	- "sysctl_ctx_free returned %d", error);
	-
	-}
	-
	-/*
	- * Function name: twa_detach
	- * Description: Called when the controller is being detached from
	- * the pci bus.
	- *
	- * Input: dev -- bus device corresponding to the ctlr
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-static TW_INT32
	-twa_detach(device_t dev)
	-{
	- struct twa_softc *sc = device_get_softc(dev);
	- TW_INT32 error;
	-
	- tw_osli_dbg_dprintf(3, sc, "entered");
	-
	- error = EBUSY;
	- if (sc->open) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2014,
	- "Device open",
	- error);
	- goto out;
	- }
	-
	- /* Shut the controller down. */
	- if ((error = twa_shutdown(dev)))
	- goto out;
	-
	- /* Free all resources associated with this controller. */
	- tw_osli_free_resources(sc);
	- error = 0;
	-
	-out:
	- return(error);
	-}
	-
	-/*
	- * Function name: twa_shutdown
	- * Description: Called at unload/shutdown time. Lets the controller
	- * know that we are going down.
	- *
	- * Input: dev -- bus device corresponding to the ctlr
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-static TW_INT32
	-twa_shutdown(device_t dev)
	-{
	- struct twa_softc *sc = device_get_softc(dev);
	- TW_INT32 error = 0;
	-
	- tw_osli_dbg_dprintf(3, sc, "entered");
	-
	- /* Disconnect interrupts. */
	- error = twa_teardown_intr(sc);
	-
	- /* Stop watchdog task. */
	- callout_drain(&(sc->watchdog_callout[0]));
	- callout_drain(&(sc->watchdog_callout[1]));
	-
	- /* Disconnect from the controller. */
	- if ((error = tw_cl_shutdown_ctlr(&(sc->ctlr_handle), 0))) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2015,
	- "Failed to shutdown Common Layer/controller",
	- error);
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: twa_busdma_lock
	- * Description: Function to provide synchronization during busdma_swi.
	- *
	- * Input: lock_arg -- lock mutex sent as argument
	- * op -- operation (lock/unlock) expected of the function
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-twa_busdma_lock(TW_VOID *lock_arg, bus_dma_lock_op_t op)
	-{
	- struct mtx *lock;
	-
	- lock = (struct mtx *)lock_arg;
	- switch (op) {
	- case BUS_DMA_LOCK:
	- mtx_lock_spin(lock);
	- break;
	-
	- case BUS_DMA_UNLOCK:
	- mtx_unlock_spin(lock);
	- break;
	-
	- default:
	- panic("Unknown operation 0x%x for twa_busdma_lock!", op);
	- }
	-}
	-
	-/*
	- * Function name: twa_pci_intr
	- * Description: Interrupt handler. Wrapper for twa_interrupt.
	- *
	- * Input: arg -- ptr to OSL internal ctlr context
	- * Output: None
	- * Return value: None
	- */
	-static TW_VOID
	-twa_pci_intr(TW_VOID *arg)
	-{
	- struct twa_softc sc = (struct twa_softc )arg;
	-
	- tw_osli_dbg_dprintf(10, sc, "entered");
	- tw_cl_interrupt(&(sc->ctlr_handle));
	-}
	-
	-/*
	- * Function name: tw_osli_fw_passthru
	- * Description: Builds a fw passthru cmd pkt, and submits it to CL.
	- *
	- * Input: sc -- ptr to OSL internal ctlr context
	- * buf -- ptr to ioctl pkt understood by CL
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_osli_fw_passthru(struct twa_softc sc, TW_INT8 buf)
	-{
	- struct tw_osli_req_context *req;
	- struct tw_osli_ioctl_no_data_buf *user_buf =
	- (struct tw_osli_ioctl_no_data_buf *)buf;
	- TW_TIME end_time;
	- TW_UINT32 timeout = 60;
	- TW_UINT32 data_buf_size_adjusted;
	- struct tw_cl_req_packet *req_pkt;
	- struct tw_cl_passthru_req_packet *pt_req;
	- TW_INT32 error;
	-
	- tw_osli_dbg_dprintf(5, sc, "ioctl: passthru");
	-
	- if ((req = tw_osli_get_request(sc)) == NULL)
	- return(EBUSY);
	-
	- req->req_handle.osl_req_ctxt = req;
	- req->orig_req = buf;
	- req->flags \|= TW_OSLI_REQ_FLAGS_PASSTHRU;
	-
	- req_pkt = &(req->req_pkt);
	- req_pkt->status = 0;
	- req_pkt->tw_osl_callback = tw_osl_complete_passthru;
	- /* Let the Common Layer retry the request on cmd queue full. */
	- req_pkt->flags \|= TW_CL_REQ_RETRY_ON_BUSY;
	-
	- pt_req = &(req_pkt->gen_req_pkt.pt_req);
	- /*
	- * Make sure that the data buffer sent to firmware is a
	- * 512 byte multiple in size.
	- */
	- data_buf_size_adjusted =
	- (user_buf->driver_pkt.buffer_length +
	- (sc->sg_size_factor - 1)) & ~(sc->sg_size_factor - 1);
	- if ((req->length = data_buf_size_adjusted)) {
	- if ((req->data = malloc(data_buf_size_adjusted,
	- TW_OSLI_MALLOC_CLASS, M_WAITOK)) == NULL) {
	- error = ENOMEM;
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2016,
	- "Could not alloc mem for "
	- "fw_passthru data_buf",
	- error);
	- goto fw_passthru_err;
	- }
	- /* Copy the payload. */
	- if ((error = copyin((TW_VOID *)(user_buf->pdata),
	- req->data,
	- user_buf->driver_pkt.buffer_length)) != 0) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2017,
	- "Could not copyin fw_passthru data_buf",
	- error);
	- goto fw_passthru_err;
	- }
	- pt_req->sgl_entries = 1; /* will be updated during mapping */
	- req->flags \|= (TW_OSLI_REQ_FLAGS_DATA_IN \|
	- TW_OSLI_REQ_FLAGS_DATA_OUT);
	- } else
	- pt_req->sgl_entries = 0; /* no payload */
	-
	- pt_req->cmd_pkt = (TW_VOID *)(&(user_buf->cmd_pkt));
	- pt_req->cmd_pkt_length = sizeof(struct tw_cl_command_packet);
	-
	- if ((error = tw_osli_map_request(req)))
	- goto fw_passthru_err;
	-
	- end_time = tw_osl_get_local_time() + timeout;
	- while (req->state != TW_OSLI_REQ_STATE_COMPLETE) {
	- mtx_lock(req->ioctl_wake_timeout_lock);
	- req->flags \|= TW_OSLI_REQ_FLAGS_SLEEPING;
	-
	- error = mtx_sleep(req, req->ioctl_wake_timeout_lock, 0,
	- "twa_passthru", timeout*hz);
	- mtx_unlock(req->ioctl_wake_timeout_lock);
	-
	- if (!(req->flags & TW_OSLI_REQ_FLAGS_SLEEPING))
	- error = 0;
	- req->flags &= ~TW_OSLI_REQ_FLAGS_SLEEPING;
	-
	- if (! error) {
	- if (((error = req->error_code)) \|\|
	- ((error = (req->state !=
	- TW_OSLI_REQ_STATE_COMPLETE))) \|\|
	- ((error = req_pkt->status)))
	- goto fw_passthru_err;
	- break;
	- }
	-
	- if (req_pkt->status) {
	- error = req_pkt->status;
	- goto fw_passthru_err;
	- }
	-
	- if (error == EWOULDBLOCK) {
	- /* Time out! */
	- if ((!(req->error_code)) &&
	- (req->state == TW_OSLI_REQ_STATE_COMPLETE) &&
	- (!(req_pkt->status)) ) {
	-#ifdef TW_OSL_DEBUG
	- tw_osli_printf(sc, "request = %p",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x7777,
	- "FALSE Passthru timeout!",
	- req);
	-#endif /* TW_OSL_DEBUG */
	- error = 0; /* False error */
	- break;
	- }
	- if (!(tw_cl_is_reset_needed(&(req->ctlr->ctlr_handle)))) {
	-#ifdef TW_OSL_DEBUG
	- tw_osli_printf(sc, "request = %p",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2018,
	- "Passthru request timed out!",
	- req);
	-#else /* TW_OSL_DEBUG */
	- device_printf((sc)->bus_dev, "Passthru request timed out!\n");
	-#endif /* TW_OSL_DEBUG */
	- tw_cl_reset_ctlr(&(req->ctlr->ctlr_handle));
	- }
	-
	- error = 0;
	- end_time = tw_osl_get_local_time() + timeout;
	- continue;
	- /*
	- * Don't touch req after a reset. It (and any
	- * associated data) will be
	- * unmapped by the callback.
	- */
	- }
	- /*
	- * Either the request got completed, or we were woken up by a
	- * signal. Calculate the new timeout, in case it was the latter.
	- */
	- timeout = (end_time - tw_osl_get_local_time());
	- } /* End of while loop */
	-
	- /* If there was a payload, copy it back. */
	- if ((!error) && (req->length))
	- if ((error = copyout(req->data, user_buf->pdata,
	- user_buf->driver_pkt.buffer_length)))
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x2019,
	- "Could not copyout fw_passthru data_buf",
	- error);
	-
	-fw_passthru_err:
	-
	- if (req_pkt->status == TW_CL_ERR_REQ_BUS_RESET)
	- error = EBUSY;
	-
	- user_buf->driver_pkt.os_status = error;
	- /* Free resources. */
	- if (req->data)
	- free(req->data, TW_OSLI_MALLOC_CLASS);
	- tw_osli_req_q_insert_tail(req, TW_OSLI_FREE_Q);
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_osl_complete_passthru
	- * Description: Called to complete passthru requests.
	- *
	- * Input: req_handle -- ptr to request handle
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osl_complete_passthru(struct tw_cl_req_handle *req_handle)
	-{
	- struct tw_osli_req_context *req = req_handle->osl_req_ctxt;
	- struct tw_cl_req_packet *req_pkt =
	- (struct tw_cl_req_packet *)(&req->req_pkt);
	- struct twa_softc *sc = req->ctlr;
	-
	- tw_osli_dbg_dprintf(5, sc, "entered");
	-
	- if (req->state != TW_OSLI_REQ_STATE_BUSY) {
	- tw_osli_printf(sc, "request = %p, status = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x201B,
	- "Unposted command completed!!",
	- req, req->state);
	- }
	-
	- /*
	- * Remove request from the busy queue. Just mark it complete.
	- * There's no need to move it into the complete queue as we are
	- * going to be done with it right now.
	- */
	- req->state = TW_OSLI_REQ_STATE_COMPLETE;
	- tw_osli_req_q_remove_item(req, TW_OSLI_BUSY_Q);
	-
	- tw_osli_unmap_request(req);
	-
	- /*
	- * Don't do a wake up if there was an error even before the request
	- * was sent down to the Common Layer, and we hadn't gotten an
	- * EINPROGRESS. The request originator will then be returned an
	- * error, and he can do the clean-up.
	- */
	- if ((req->error_code) && (!(req->flags & TW_OSLI_REQ_FLAGS_IN_PROGRESS)))
	- return;
	-
	- if (req->flags & TW_OSLI_REQ_FLAGS_PASSTHRU) {
	- if (req->flags & TW_OSLI_REQ_FLAGS_SLEEPING) {
	- /* Wake up the sleeping command originator. */
	- tw_osli_dbg_dprintf(5, sc,
	- "Waking up originator of request %p", req);
	- req->flags &= ~TW_OSLI_REQ_FLAGS_SLEEPING;
	- wakeup_one(req);
	- } else {
	- /*
	- * If the request completed even before mtx_sleep
	- * was called, simply return.
	- */
	- if (req->flags & TW_OSLI_REQ_FLAGS_MAPPED)
	- return;
	-
	- if (req_pkt->status == TW_CL_ERR_REQ_BUS_RESET)
	- return;
	-
	- tw_osli_printf(sc, "request = %p",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x201C,
	- "Passthru callback called, "
	- "and caller not sleeping",
	- req);
	- }
	- } else {
	- tw_osli_printf(sc, "request = %p",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x201D,
	- "Passthru callback called for non-passthru request",
	- req);
	- }
	-}
	-
	-/*
	- * Function name: tw_osli_get_request
	- * Description: Gets a request pkt from the free queue.
	- *
	- * Input: sc -- ptr to OSL internal ctlr context
	- * Output: None
	- * Return value: ptr to request pkt -- success
	- * NULL -- failure
	- */
	-struct tw_osli_req_context *
	-tw_osli_get_request(struct twa_softc *sc)
	-{
	- struct tw_osli_req_context *req;
	-
	- tw_osli_dbg_dprintf(4, sc, "entered");
	-
	- /* Get a free request packet. */
	- req = tw_osli_req_q_remove_head(sc, TW_OSLI_FREE_Q);
	-
	- /* Initialize some fields to their defaults. */
	- if (req) {
	- req->req_handle.osl_req_ctxt = NULL;
	- req->req_handle.cl_req_ctxt = NULL;
	- req->req_handle.is_io = 0;
	- req->data = NULL;
	- req->length = 0;
	- req->deadline = 0;
	- req->real_data = NULL;
	- req->real_length = 0;
	- req->state = TW_OSLI_REQ_STATE_INIT;/* req being initialized */
	- req->flags = 0;
	- req->error_code = 0;
	- req->orig_req = NULL;
	-
	- bzero(&(req->req_pkt), sizeof(struct tw_cl_req_packet));
	- }
	- return(req);
	-}
	-
	-/*
	- * Function name: twa_map_load_data_callback
	- * Description: Callback of bus_dmamap_load for the buffer associated
	- * with data. Updates the cmd pkt (size/sgl_entries
	- * fields, as applicable) to reflect the number of sg
	- * elements.
	- *
	- * Input: arg -- ptr to OSL internal request context
	- * segs -- ptr to a list of segment descriptors
	- * nsegments--# of segments
	- * error -- 0 if no errors encountered before callback,
	- * non-zero if errors were encountered
	- * Output: None
	- * Return value: None
	- */
	-static TW_VOID
	-twa_map_load_data_callback(TW_VOID arg, bus_dma_segment_t segs,
	- TW_INT32 nsegments, TW_INT32 error)
	-{
	- struct tw_osli_req_context *req =
	- (struct tw_osli_req_context *)arg;
	- struct twa_softc *sc = req->ctlr;
	- struct tw_cl_req_packet *req_pkt = &(req->req_pkt);
	-
	- tw_osli_dbg_dprintf(10, sc, "entered");
	-
	- if (error == EINVAL) {
	- req->error_code = error;
	- return;
	- }
	-
	- /* Mark the request as currently being processed. */
	- req->state = TW_OSLI_REQ_STATE_BUSY;
	- /* Move the request into the busy queue. */
	- tw_osli_req_q_insert_tail(req, TW_OSLI_BUSY_Q);
	-
	- req->flags \|= TW_OSLI_REQ_FLAGS_MAPPED;
	-
	- if (error == EFBIG) {
	- req->error_code = error;
	- goto out;
	- }
	-
	- if (req->flags & TW_OSLI_REQ_FLAGS_PASSTHRU) {
	- struct tw_cl_passthru_req_packet *pt_req;
	-
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_IN)
	- bus_dmamap_sync(sc->ioctl_tag, sc->ioctl_map,
	- BUS_DMASYNC_PREREAD);
	-
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_OUT) {
	- /*
	- * If we're using an alignment buffer, and we're
	- * writing data, copy the real data out.
	- */
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_COPY_NEEDED)
	- bcopy(req->real_data, req->data, req->real_length);
	- bus_dmamap_sync(sc->ioctl_tag, sc->ioctl_map,
	- BUS_DMASYNC_PREWRITE);
	- }
	-
	- pt_req = &(req_pkt->gen_req_pkt.pt_req);
	- pt_req->sg_list = (TW_UINT8 *)segs;
	- pt_req->sgl_entries += (nsegments - 1);
	- error = tw_cl_fw_passthru(&(sc->ctlr_handle), req_pkt,
	- &(req->req_handle));
	- } else {
	- struct tw_cl_scsi_req_packet *scsi_req;
	-
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_IN)
	- bus_dmamap_sync(sc->dma_tag, req->dma_map,
	- BUS_DMASYNC_PREREAD);
	-
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_OUT) {
	- /*
	- * If we're using an alignment buffer, and we're
	- * writing data, copy the real data out.
	- */
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_COPY_NEEDED)
	- bcopy(req->real_data, req->data, req->real_length);
	- bus_dmamap_sync(sc->dma_tag, req->dma_map,
	- BUS_DMASYNC_PREWRITE);
	- }
	-
	- scsi_req = &(req_pkt->gen_req_pkt.scsi_req);
	- scsi_req->sg_list = (TW_UINT8 *)segs;
	- scsi_req->sgl_entries += (nsegments - 1);
	- error = tw_cl_start_io(&(sc->ctlr_handle), req_pkt,
	- &(req->req_handle));
	- }
	-
	-out:
	- if (error) {
	- req->error_code = error;
	- req_pkt->tw_osl_callback(&(req->req_handle));
	- /*
	- * If the caller had been returned EINPROGRESS, and he has
	- * registered a callback for handling completion, the callback
	- * will never get called because we were unable to submit the
	- * request. So, free up the request right here.
	- */
	- if (req->flags & TW_OSLI_REQ_FLAGS_IN_PROGRESS)
	- tw_osli_req_q_insert_tail(req, TW_OSLI_FREE_Q);
	- }
	-}
	-
	-/*
	- * Function name: twa_map_load_callback
	- * Description: Callback of bus_dmamap_load for the buffer associated
	- * with a cmd pkt.
	- *
	- * Input: arg -- ptr to variable to hold phys addr
	- * segs -- ptr to a list of segment descriptors
	- * nsegments--# of segments
	- * error -- 0 if no errors encountered before callback,
	- * non-zero if errors were encountered
	- * Output: None
	- * Return value: None
	- */
	-static TW_VOID
	-twa_map_load_callback(TW_VOID arg, bus_dma_segment_t segs,
	- TW_INT32 nsegments, TW_INT32 error)
	-{
	- ((bus_addr_t )arg) = segs[0].ds_addr;
	-}
	-
	-/*
	- * Function name: tw_osli_map_request
	- * Description: Maps a cmd pkt and data associated with it, into
	- * DMA'able memory.
	- *
	- * Input: req -- ptr to request pkt
	- * Output: None
	- * Return value: 0 -- success
	- * non-zero-- failure
	- */
	-TW_INT32
	-tw_osli_map_request(struct tw_osli_req_context *req)
	-{
	- struct twa_softc *sc = req->ctlr;
	- TW_INT32 error = 0;
	-
	- tw_osli_dbg_dprintf(10, sc, "entered");
	-
	- /* If the command involves data, map that too. */
	- if (req->data != NULL) {
	- /*
	- * It's sufficient for the data pointer to be 4-byte aligned
	- * to work with 9000. However, if 4-byte aligned addresses
	- * are passed to bus_dmamap_load, we can get back sg elements
	- * that are not 512-byte multiples in size. So, we will let
	- * only those buffers that are 512-byte aligned to pass
	- * through, and bounce the rest, so as to make sure that we
	- * always get back sg elements that are 512-byte multiples
	- * in size.
	- */
	- if (((vm_offset_t)req->data % sc->sg_size_factor) \|\|
	- (req->length % sc->sg_size_factor)) {
	- req->flags \|= TW_OSLI_REQ_FLAGS_DATA_COPY_NEEDED;
	- /* Save original data pointer and length. */
	- req->real_data = req->data;
	- req->real_length = req->length;
	- req->length = (req->length +
	- (sc->sg_size_factor - 1)) &
	- ~(sc->sg_size_factor - 1);
	- req->data = malloc(req->length, TW_OSLI_MALLOC_CLASS,
	- M_NOWAIT);
	- if (req->data == NULL) {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x201E,
	- "Failed to allocate memory "
	- "for bounce buffer",
	- ENOMEM);
	- /* Restore original data pointer and length. */
	- req->data = req->real_data;
	- req->length = req->real_length;
	- return(ENOMEM);
	- }
	- }
	-
	- /*
	- * Map the data buffer into bus space and build the SG list.
	- */
	- if (req->flags & TW_OSLI_REQ_FLAGS_PASSTHRU) {
	- /* Lock against multiple simultaneous ioctl calls. */
	- mtx_lock_spin(sc->io_lock);
	- error = bus_dmamap_load(sc->ioctl_tag, sc->ioctl_map,
	- req->data, req->length,
	- twa_map_load_data_callback, req,
	- BUS_DMA_WAITOK);
	- mtx_unlock_spin(sc->io_lock);
	- } else if (req->flags & TW_OSLI_REQ_FLAGS_CCB) {
	- error = bus_dmamap_load_ccb(sc->dma_tag, req->dma_map,
	- req->orig_req, twa_map_load_data_callback, req,
	- BUS_DMA_WAITOK);
	- } else {
	- /*
	- * There's only one CAM I/O thread running at a time.
	- * So, there's no need to hold the io_lock.
	- */
	- error = bus_dmamap_load(sc->dma_tag, req->dma_map,
	- req->data, req->length,
	- twa_map_load_data_callback, req,
	- BUS_DMA_WAITOK);
	- }
	-
	- if (!error)
	- error = req->error_code;
	- else {
	- if (error == EINPROGRESS) {
	- /*
	- * Specifying sc->io_lock as the lockfuncarg
	- * in ...tag_create should protect the access
	- * of ...FLAGS_MAPPED from the callback.
	- */
	- mtx_lock_spin(sc->io_lock);
	- if (!(req->flags & TW_OSLI_REQ_FLAGS_MAPPED))
	- req->flags \|= TW_OSLI_REQ_FLAGS_IN_PROGRESS;
	- tw_osli_disallow_new_requests(sc, &(req->req_handle));
	- mtx_unlock_spin(sc->io_lock);
	- error = 0;
	- } else {
	- tw_osli_printf(sc, "error = %d",
	- TW_CL_SEVERITY_ERROR_STRING,
	- TW_CL_MESSAGE_SOURCE_FREEBSD_DRIVER,
	- 0x9999,
	- "Failed to map DMA memory "
	- "for I/O request",
	- error);
	- req->flags \|= TW_OSLI_REQ_FLAGS_FAILED;
	- /* Free alignment buffer if it was used. */
	- if (req->flags &
	- TW_OSLI_REQ_FLAGS_DATA_COPY_NEEDED) {
	- free(req->data, TW_OSLI_MALLOC_CLASS);
	- /*
	- * Restore original data pointer
	- * and length.
	- */
	- req->data = req->real_data;
	- req->length = req->real_length;
	- }
	- }
	- }
	-
	- } else {
	- /* Mark the request as currently being processed. */
	- req->state = TW_OSLI_REQ_STATE_BUSY;
	- /* Move the request into the busy queue. */
	- tw_osli_req_q_insert_tail(req, TW_OSLI_BUSY_Q);
	- if (req->flags & TW_OSLI_REQ_FLAGS_PASSTHRU)
	- error = tw_cl_fw_passthru(&sc->ctlr_handle,
	- &(req->req_pkt), &(req->req_handle));
	- else
	- error = tw_cl_start_io(&sc->ctlr_handle,
	- &(req->req_pkt), &(req->req_handle));
	- if (error) {
	- req->error_code = error;
	- req->req_pkt.tw_osl_callback(&(req->req_handle));
	- }
	- }
	- return(error);
	-}
	-
	-/*
	- * Function name: tw_osli_unmap_request
	- * Description: Undoes the mapping done by tw_osli_map_request.
	- *
	- * Input: req -- ptr to request pkt
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osli_unmap_request(struct tw_osli_req_context *req)
	-{
	- struct twa_softc *sc = req->ctlr;
	-
	- tw_osli_dbg_dprintf(10, sc, "entered");
	-
	- /* If the command involved data, unmap that too. */
	- if (req->data != NULL) {
	- if (req->flags & TW_OSLI_REQ_FLAGS_PASSTHRU) {
	- /* Lock against multiple simultaneous ioctl calls. */
	- mtx_lock_spin(sc->io_lock);
	-
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_IN) {
	- bus_dmamap_sync(sc->ioctl_tag,
	- sc->ioctl_map, BUS_DMASYNC_POSTREAD);
	-
	- /*
	- * If we are using a bounce buffer, and we are
	- * reading data, copy the real data in.
	- */
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_COPY_NEEDED)
	- bcopy(req->data, req->real_data,
	- req->real_length);
	- }
	-
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_OUT)
	- bus_dmamap_sync(sc->ioctl_tag, sc->ioctl_map,
	- BUS_DMASYNC_POSTWRITE);
	-
	- bus_dmamap_unload(sc->ioctl_tag, sc->ioctl_map);
	-
	- mtx_unlock_spin(sc->io_lock);
	- } else {
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_IN) {
	- bus_dmamap_sync(sc->dma_tag,
	- req->dma_map, BUS_DMASYNC_POSTREAD);
	-
	- /*
	- * If we are using a bounce buffer, and we are
	- * reading data, copy the real data in.
	- */
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_COPY_NEEDED)
	- bcopy(req->data, req->real_data,
	- req->real_length);
	- }
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_OUT)
	- bus_dmamap_sync(sc->dma_tag, req->dma_map,
	- BUS_DMASYNC_POSTWRITE);
	-
	- bus_dmamap_unload(sc->dma_tag, req->dma_map);
	- }
	- }
	-
	- /* Free alignment buffer if it was used. */
	- if (req->flags & TW_OSLI_REQ_FLAGS_DATA_COPY_NEEDED) {
	- free(req->data, TW_OSLI_MALLOC_CLASS);
	- /* Restore original data pointer and length. */
	- req->data = req->real_data;
	- req->length = req->real_length;
	- }
	-}
	-
	-#ifdef TW_OSL_DEBUG
	-
	-TW_VOID twa_report_stats(TW_VOID);
	-TW_VOID twa_reset_stats(TW_VOID);
	-TW_VOID tw_osli_print_ctlr_stats(struct twa_softc *sc);
	-TW_VOID twa_print_req_info(struct tw_osli_req_context *req);
	-
	-/*
	- * Function name: twa_report_stats
	- * Description: For being called from ddb. Calls functions that print
	- * OSL and CL internal stats for the controller.
	- *
	- * Input: None
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-twa_report_stats(TW_VOID)
	-{
	- struct twa_softc *sc;
	- TW_INT32 i;
	-
	- for (i = 0; (sc = devclass_get_softc(twa_devclass, i)) != NULL; i++) {
	- tw_osli_print_ctlr_stats(sc);
	- tw_cl_print_ctlr_stats(&sc->ctlr_handle);
	- }
	-}
	-
	-/*
	- * Function name: tw_osli_print_ctlr_stats
	- * Description: For being called from ddb. Prints OSL controller stats
	- *
	- * Input: sc -- ptr to OSL internal controller context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-tw_osli_print_ctlr_stats(struct twa_softc *sc)
	-{
	- twa_printf(sc, "osl_ctlr_ctxt = %p\n", sc);
	- twa_printf(sc, "OSLq type current max\n");
	- twa_printf(sc, "free %04d %04d\n",
	- sc->q_stats[TW_OSLI_FREE_Q].cur_len,
	- sc->q_stats[TW_OSLI_FREE_Q].max_len);
	- twa_printf(sc, "busy %04d %04d\n",
	- sc->q_stats[TW_OSLI_BUSY_Q].cur_len,
	- sc->q_stats[TW_OSLI_BUSY_Q].max_len);
	-}
	-
	-/*
	- * Function name: twa_print_req_info
	- * Description: For being called from ddb. Calls functions that print
	- * OSL and CL internal details for the request.
	- *
	- * Input: req -- ptr to OSL internal request context
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-twa_print_req_info(struct tw_osli_req_context *req)
	-{
	- struct twa_softc *sc = req->ctlr;
	-
	- twa_printf(sc, "OSL details for request:\n");
	- twa_printf(sc, "osl_req_ctxt = %p, cl_req_ctxt = %p\n"
	- "data = %p, length = 0x%x, real_data = %p, real_length = 0x%x\n"
	- "state = 0x%x, flags = 0x%x, error = 0x%x, orig_req = %p\n"
	- "next_req = %p, prev_req = %p, dma_map = %p\n",
	- req->req_handle.osl_req_ctxt, req->req_handle.cl_req_ctxt,
	- req->data, req->length, req->real_data, req->real_length,
	- req->state, req->flags, req->error_code, req->orig_req,
	- req->link.next, req->link.prev, req->dma_map);
	- tw_cl_print_req_info(&(req->req_handle));
	-}
	-
	-/*
	- * Function name: twa_reset_stats
	- * Description: For being called from ddb.
	- * Resets some OSL controller stats.
	- *
	- * Input: None
	- * Output: None
	- * Return value: None
	- */
	-TW_VOID
	-twa_reset_stats(TW_VOID)
	-{
	- struct twa_softc *sc;
	- TW_INT32 i;
	-
	- for (i = 0; (sc = devclass_get_softc(twa_devclass, i)) != NULL; i++) {
	- sc->q_stats[TW_OSLI_FREE_Q].max_len = 0;
	- sc->q_stats[TW_OSLI_BUSY_Q].max_len = 0;
	- tw_cl_reset_stats(&sc->ctlr_handle);
	- }
	-}
	-
	-#endif /* TW_OSL_DEBUG */
	Index: sys/dev/twa/tw_osl_includes.h
	===================================================================
	--- sys/dev/twa/tw_osl_includes.h
	+++ /dev/null
	@@ -1,76 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_OSL_INCLUDES_H
	-
	-#define TW_OSL_INCLUDES_H
	-
	-/*
	- * All header files needed by the OS Layer.
	- */
	-
	-#include <sys/param.h>
	-#include <sys/systm.h>
	-#include <sys/malloc.h>
	-#include <sys/kernel.h>
	-#include <sys/module.h>
	-#include <sys/sysctl.h>
	-#include <sys/bus.h>
	-#include <sys/conf.h>
	-#include <sys/clock.h>
	-#include <sys/disk.h>
	-#include <sys/stat.h>
	-#include <sys/devicestat.h>
	-#include <sys/taskqueue.h>
	-
	-#include <machine/bus.h>
	-#include <machine/resource.h>
	-#include <machine/stdarg.h>
	-
	-#include <vm/vm.h>
	-
	-#include <sys/rman.h>
	-
	-#include <dev/pci/pcireg.h>
	-#include <dev/pci/pcivar.h>
	-
	-#include <dev/twa/tw_osl_share.h>
	-#include <dev/twa/tw_cl_share.h>
	-#include <dev/twa/tw_osl_externs.h>
	-
	-#endif /* TW_OSL_INCLUDES_H */
	Index: sys/dev/twa/tw_osl_inline.h
	===================================================================
	--- sys/dev/twa/tw_osl_inline.h
	+++ /dev/null
	@@ -1,309 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_OSL_INLINE_H
	-
	-#define TW_OSL_INLINE_H
	-
	-/*
	- * Inline functions shared between OSL and CL, and defined by OSL.
	- */
	-
	-#include <dev/twa/tw_osl.h>
	-
	-/*
	- * Function name: tw_osl_init_lock
	- * Description: Initializes a lock.
	- *
	- * Input: ctlr_handle -- ptr to controller handle
	- * lock_name -- string indicating name of the lock
	- * Output: lock -- ptr to handle to the initialized lock
	- * Return value: None
	- */
	-#define tw_osl_init_lock(ctlr_handle, lock_name, lock) \
	- mtx_init(lock, lock_name, NULL, MTX_SPIN)
	-
	-/*
	- * Function name: tw_osl_destroy_lock
	- * Description: Destroys a previously initialized lock.
	- *
	- * Input: ctlr_handle -- ptr to controller handle
	- * lock -- ptr to handle to the lock to be
	- * destroyed
	- * Output: None
	- * Return value: None
	- */
	-#define tw_osl_destroy_lock(ctlr_handle, lock) \
	- mtx_destroy(lock)
	-
	-/*
	- * Function name: tw_osl_get_lock
	- * Description: Acquires the specified lock.
	- *
	- * Input: ctlr_handle -- ptr to controller handle
	- * lock -- ptr to handle to the lock to be
	- * acquired
	- * Output: None
	- * Return value: None
	- */
	-#define tw_osl_get_lock(ctlr_handle, lock) \
	- mtx_lock_spin(lock)
	-
	-/*
	- * Function name: tw_osl_free_lock
	- * Description: Frees a previously acquired lock.
	- *
	- * Input: ctlr_handle -- ptr to controller handle
	- * lock -- ptr to handle to the lock to be freed
	- * Output: None
	- * Return value: None
	- */
	-#define tw_osl_free_lock(ctlr_handle, lock) \
	- mtx_unlock_spin(lock)
	-
	-#ifdef TW_OSL_DEBUG
	-
	-/*
	- * Function name: tw_osl_dbg_printf
	- * Description: Prints passed info (prefixed by ctlr name)to syslog
	- *
	- * Input: ctlr_handle -- controller handle
	- * fmt -- format string for the arguments to follow
	- * ... -- variable number of arguments, to be printed
	- * based on the fmt string
	- * Output: None
	- * Return value: Number of bytes printed
	- */
	-#define tw_osl_dbg_printf(ctlr_handle, fmt, args...) \
	- twa_printf((ctlr_handle->osl_ctlr_ctxt), fmt, ##args)
	-
	-#endif /* TW_OSL_DEBUG */
	-
	-/*
	- * Function name: tw_osl_notify_event
	- * Description: Prints passed event info (prefixed by ctlr name)
	- * to syslog
	- *
	- * Input: ctlr_handle -- controller handle
	- * event -- ptr to a packet describing the event/error
	- * Output: None
	- * Return value: None
	- */
	-#define tw_osl_notify_event(ctlr_handle, event) \
	- twa_printf((ctlr_handle->osl_ctlr_ctxt), \
	- "%s: (0x%02X: 0x%04X): %s: %s\n", \
	- event->severity_str, \
	- event->event_src, \
	- event->aen_code, \
	- event->parameter_data + \
	- strlen(event->parameter_data) + 1, \
	- event->parameter_data)
	-
	-/*
	- * Function name: tw_osl_read_reg
	- * Description: Reads a register on the controller
	- *
	- * Input: ctlr_handle -- controller handle
	- * offset -- offset from Base Address
	- * size -- # of bytes to read
	- * Output: None
	- * Return value: Value read
	- */
	-#define tw_osl_read_reg tw_osl_read_reg_inline
	-static __inline TW_UINT32
	-tw_osl_read_reg_inline(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_INT32 offset, TW_INT32 size)
	-{
	- bus_space_tag_t bus_tag =
	- ((struct twa_softc *)(ctlr_handle->osl_ctlr_ctxt))->bus_tag;
	- bus_space_handle_t bus_handle =
	- ((struct twa_softc *)(ctlr_handle->osl_ctlr_ctxt))->bus_handle;
	-
	- if (size == 4)
	- return((TW_UINT32)bus_space_read_4(bus_tag, bus_handle,
	- offset));
	- else if (size == 2)
	- return((TW_UINT32)bus_space_read_2(bus_tag, bus_handle,
	- offset));
	- else
	- return((TW_UINT32)bus_space_read_1(bus_tag, bus_handle,
	- offset));
	-}
	-
	-/*
	- * Function name: tw_osl_write_reg
	- * Description: Writes to a register on the controller
	- *
	- * Input: ctlr_handle -- controller handle
	- * offset -- offset from Base Address
	- * value -- value to write
	- * size -- # of bytes to write
	- * Output: None
	- * Return value: None
	- */
	-#define tw_osl_write_reg tw_osl_write_reg_inline
	-static __inline TW_VOID
	-tw_osl_write_reg_inline(struct tw_cl_ctlr_handle *ctlr_handle,
	- TW_INT32 offset, TW_INT32 value, TW_INT32 size)
	-{
	- bus_space_tag_t bus_tag =
	- ((struct twa_softc *)(ctlr_handle->osl_ctlr_ctxt))->bus_tag;
	- bus_space_handle_t bus_handle =
	- ((struct twa_softc *)(ctlr_handle->osl_ctlr_ctxt))->bus_handle;
	-
	- if (size == 4)
	- bus_space_write_4(bus_tag, bus_handle, offset, value);
	- else if (size == 2)
	- bus_space_write_2(bus_tag, bus_handle, offset, (TW_INT16)value);
	- else
	- bus_space_write_1(bus_tag, bus_handle, offset, (TW_INT8)value);
	-}
	-
	-#ifdef TW_OSL_PCI_CONFIG_ACCESSIBLE
	-
	-/*
	- * Function name: tw_osl_read_pci_config
	- * Description: Reads from the PCI config space.
	- *
	- * Input: sc -- ptr to per ctlr structure
	- * offset -- register offset
	- * size -- # of bytes to be read
	- * Output: None
	- * Return value: Value read
	- */
	-#define tw_osl_read_pci_config(ctlr_handle, offset, size) \
	- pci_read_config( \
	- ((struct twa_softc *)(ctlr_handle->osl_ctlr_ctxt))->bus_dev, \
	- offset, size)
	-
	-/*
	- * Function name: tw_osl_write_pci_config
	- * Description: Writes to the PCI config space.
	- *
	- * Input: sc -- ptr to per ctlr structure
	- * offset -- register offset
	- * value -- value to write
	- * size -- # of bytes to be written
	- * Output: None
	- * Return value: None
	- */
	-#define tw_osl_write_pci_config(ctlr_handle, offset, value, size) \
	- pci_write_config( \
	- ((struct twa_softc *)(ctlr_handle->osl_ctlr_ctxt))->bus_dev, \
	- offset/PCIR_STATUS/, value, size)
	-
	-#endif /* TW_OSL_PCI_CONFIG_ACCESSIBLE */
	-
	-/*
	- * Function name: tw_osl_get_local_time
	- * Description: Gets the local time
	- *
	- * Input: None
	- * Output: None
	- * Return value: local time
	- */
	-#define tw_osl_get_local_time() \
	- (time_second - utc_offset())
	-
	-/*
	- * Function name: tw_osl_delay
	- * Description: Spin for the specified time
	- *
	- * Input: usecs -- micro-seconds to spin
	- * Output: None
	- * Return value: None
	- */
	-#define tw_osl_delay(usecs) DELAY(usecs)
	-
	-#ifdef TW_OSL_CAN_SLEEP
	-
	-/*
	- * Function name: tw_osl_sleep
	- * Description: Sleep for the specified time, or until woken up
	- *
	- * Input: ctlr_handle -- controller handle
	- * sleep_handle -- handle to sleep on
	- * timeout -- time period (in ms) to sleep
	- * Output: None
	- * Return value: 0 -- successfully woken up
	- * EWOULDBLOCK -- time out
	- * ERESTART -- woken up by a signal
	- */
	-#define tw_osl_sleep(ctlr_handle, sleep_handle, timeout) \
	- tsleep((TW_VOID *)sleep_handle, PRIBIO, NULL, timeout)
	-
	-/*
	- * Function name: tw_osl_wakeup
	- * Description: Wake up a sleeping process
	- *
	- * Input: ctlr_handle -- controller handle
	- * sleep_handle -- handle of sleeping process to be
	- woken up
	- * Output: None
	- * Return value: None
	- */
	-#define tw_osl_wakeup(ctlr_handle, sleep_handle) \
	- wakeup_one(sleep_handle)
	-
	-#endif /* TW_OSL_CAN_SLEEP */
	-
	-/* Allows setting breakpoints in the CL code for debugging purposes. */
	-#define tw_osl_breakpoint() breakpoint()
	-
	-/* Text name of current function. */
	-#define tw_osl_cur_func() __func__
	-
	-/* Copy 'size' bytes from 'src' to 'dest'. */
	-#define tw_osl_memcpy(dest, src, size) bcopy(src, dest, size)
	-
	-/* Zero 'size' bytes starting at 'addr'. */
	-#define tw_osl_memzero bzero
	-
	-/* Standard sprintf. */
	-#define tw_osl_sprintf sprintf
	-
	-/* Copy string 'src' to 'dest'. */
	-#define tw_osl_strcpy strcpy
	-
	-/* Return length of string pointed at by 'str'. */
	-#define tw_osl_strlen strlen
	-
	-/* Standard vsprintf. */
	-#define tw_osl_vsprintf vsprintf
	-
	-#endif /* TW_OSL_INLINE_H */
	Index: sys/dev/twa/tw_osl_ioctl.h
	===================================================================
	--- sys/dev/twa/tw_osl_ioctl.h
	+++ /dev/null
	@@ -1,108 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_OSL_IOCTL_H
	-
	-#define TW_OSL_IOCTL_H
	-
	-/*
	- * Macros and structures for OS Layer/Common Layer handled ioctls.
	- */
	-
	-#include <dev/twa/tw_cl_fwif.h>
	-#include <dev/twa/tw_cl_ioctl.h>
	-
	-#pragma pack(1)
	-/*
	- * We need the structure below to ensure that the first byte of
	- * data_buf is not overwritten by the kernel, after we return
	- * from the ioctl call. Note that cmd_pkt has been reduced
	- * to an array of 1024 bytes even though it's actually 2048 bytes
	- * in size. This is because, we don't expect requests from user
	- * land requiring 2048 (273 sg elements) byte cmd pkts.
	- */
	-typedef struct tw_osli_ioctl_no_data_buf {
	- struct tw_cl_driver_packet driver_pkt;
	- TW_VOID pdata; / points to data_buf */
	- TW_INT8 padding[488 - sizeof(TW_VOID *)];
	- struct tw_cl_command_packet cmd_pkt;
	-} TW_OSLI_IOCTL_NO_DATA_BUF;
	-
	-#pragma pack()
	-
	-/* ioctl cmds handled by the OS Layer */
	-#define TW_OSL_IOCTL_SCAN_BUS \
	- _IO('T', 200)
	-#define TW_OSL_IOCTL_FIRMWARE_PASS_THROUGH \
	- _IOWR('T', 202, TW_OSLI_IOCTL_NO_DATA_BUF)
	-
	-#include <sys/ioccom.h>
	-
	-#pragma pack(1)
	-
	-typedef struct tw_osli_ioctl_with_payload {
	- struct tw_cl_driver_packet driver_pkt;
	- TW_INT8 padding[488];
	- struct tw_cl_command_packet cmd_pkt;
	- union {
	- struct tw_cl_event_packet event_pkt;
	- struct tw_cl_lock_packet lock_pkt;
	- struct tw_cl_compatibility_packet compat_pkt;
	- TW_INT8 data_buf[1];
	- } payload;
	-} TW_OSLI_IOCTL_WITH_PAYLOAD;
	-
	-#pragma pack()
	-
	-/* ioctl cmds handled by the Common Layer */
	-#define TW_CL_IOCTL_GET_FIRST_EVENT \
	- _IOWR('T', 203, TW_OSLI_IOCTL_WITH_PAYLOAD)
	-#define TW_CL_IOCTL_GET_LAST_EVENT \
	- _IOWR('T', 204, TW_OSLI_IOCTL_WITH_PAYLOAD)
	-#define TW_CL_IOCTL_GET_NEXT_EVENT \
	- _IOWR('T', 205, TW_OSLI_IOCTL_WITH_PAYLOAD)
	-#define TW_CL_IOCTL_GET_PREVIOUS_EVENT \
	- _IOWR('T', 206, TW_OSLI_IOCTL_WITH_PAYLOAD)
	-#define TW_CL_IOCTL_GET_LOCK \
	- _IOWR('T', 207, TW_OSLI_IOCTL_WITH_PAYLOAD)
	-#define TW_CL_IOCTL_RELEASE_LOCK \
	- _IOWR('T', 208, TW_OSLI_IOCTL_WITH_PAYLOAD)
	-#define TW_CL_IOCTL_GET_COMPATIBILITY_INFO \
	- _IOWR('T', 209, TW_OSLI_IOCTL_WITH_PAYLOAD)
	-
	-#endif /* TW_OSL_IOCTL_H */
	Index: sys/dev/twa/tw_osl_share.h
	===================================================================
	--- sys/dev/twa/tw_osl_share.h
	+++ /dev/null
	@@ -1,107 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_OSL_SHARE_H
	-
	-#define TW_OSL_SHARE_H
	-
	-/*
	- * Macros, structures and functions shared between OSL and CL,
	- * and defined by OSL.
	- */
	-
	-#include <sys/param.h>
	-#include <sys/systm.h>
	-#include <sys/kernel.h>
	-#include <sys/clock.h>
	-#include <sys/lock.h>
	-#include <sys/mutex.h>
	-#include <sys/sysctl.h>
	-#include <sys/bus.h>
	-#include <sys/taskqueue.h>
	-
	-#include <machine/bus.h>
	-#include <machine/endian.h>
	-#include <machine/stdarg.h>
	-
	-#include <dev/pci/pcivar.h>
	-
	-#include <dev/twa/tw_osl_types.h>
	-#include "opt_twa.h"
	-
	-#ifdef TWA_DEBUG
	-#define TW_OSL_DEBUG TWA_DEBUG
	-#endif
	-
	-#ifdef TWA_ENCLOSURE_SUPPORT
	-#define TW_OSL_ENCLOSURE_SUPPORT
	-#endif
	-
	-#define TW_OSL_DRIVER_VERSION_STRING "3.80.06.003"
	-
	-#define TW_OSL_CAN_SLEEP
	-
	-#ifdef TW_OSL_CAN_SLEEP
	-typedef TW_VOID *TW_SLEEP_HANDLE;
	-#endif /* TW_OSL_CAN_SLEEP */
	-
	-#define TW_OSL_PCI_CONFIG_ACCESSIBLE
	-
	-#if _BYTE_ORDER == _BIG_ENDIAN
	-#define TW_OSL_BIG_ENDIAN
	-#else
	-#define TW_OSL_LITTLE_ENDIAN
	-#endif
	-
	-#ifdef TW_OSL_DEBUG
	-extern TW_INT32 TW_OSL_DEBUG_LEVEL_FOR_CL;
	-#endif /* TW_OSL_DEBUG */
	-
	-/* Possible return codes from/to Common Layer functions. */
	-#define TW_OSL_ESUCCESS 0 /* success */
	-#define TW_OSL_EGENFAILURE 1 /* general failure */
	-#define TW_OSL_ENOMEM ENOMEM /* insufficient memory */
	-#define TW_OSL_EIO EIO /* I/O error */
	-#define TW_OSL_ETIMEDOUT ETIMEDOUT /* time out */
	-#define TW_OSL_ENOTTY ENOTTY /* invalid command */
	-#define TW_OSL_EBUSY EBUSY /* busy -- try later */
	-#define TW_OSL_EBIG EFBIG /* request too big */
	-#define TW_OSL_EWOULDBLOCK EWOULDBLOCK /* sleep timed out */
	-#define TW_OSL_ERESTART ERESTART /* sleep terminated by a signal */
	-
	-#endif /* TW_OSL_SHARE_H */
	Index: sys/dev/twa/tw_osl_types.h
	===================================================================
	--- sys/dev/twa/tw_osl_types.h
	+++ /dev/null
	@@ -1,60 +0,0 @@
	-/*-
	- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
	- *
	- * Copyright (c) 2004-07 Applied Micro Circuits Corporation.
	- * Copyright (c) 2004-05 Vinod Kashyap.
	- * All rights reserved.
	- *
	- * Redistribution and use in source and binary forms, with or without
	- * modification, are permitted provided that the following conditions
	- * are met:
	- * 1. Redistributions of source code must retain the above copyright
	- * notice, this list of conditions and the following disclaimer.
	- * 2. Redistributions in binary form must reproduce the above copyright
	- * notice, this list of conditions and the following disclaimer in the
	- * documentation and/or other materials provided with the distribution.
	- *
	- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	- * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	- * SUCH DAMAGE.
	- *
	- * $FreeBSD$
	- */
	-
	-/*
	- * AMCC'S 3ware driver for 9000 series storage controllers.
	- *
	- * Author: Vinod Kashyap
	- * Modifications by: Adam Radford
	- */
	-
	-#ifndef TW_OSL_TYPES_H
	-
	-#define TW_OSL_TYPES_H
	-
	-/*
	- * typedefs shared between OSL and CL, and defined by OSL.
	- */
	-
	-typedef void TW_VOID;
	-typedef char TW_INT8;
	-typedef unsigned char TW_UINT8;
	-typedef short TW_INT16;
	-typedef unsigned short TW_UINT16;
	-typedef int TW_INT32;
	-typedef unsigned int TW_UINT32;
	-typedef long long TW_INT64;
	-typedef unsigned long long TW_UINT64;
	-
	-typedef time_t TW_TIME;
	-typedef struct mtx TW_LOCK_HANDLE;
	-
	-#endif /* TW_OSL_TYPES_H */
	Index: sys/i386/conf/GENERIC
	===================================================================
	--- sys/i386/conf/GENERIC
	+++ sys/i386/conf/GENERIC
	@@ -156,7 +156,6 @@
	device arcmsr # Areca SATA II RAID
	device ciss # Compaq Smart RAID 5*
	device ips # IBM (Adaptec) ServeRAID
	-device twa # 3ware 9000 series PATA/SATA RAID
	device tws # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller

	# RAID controllers
	Index: sys/i386/conf/NOTES
	===================================================================
	--- sys/i386/conf/NOTES
	+++ sys/i386/conf/NOTES
	@@ -614,13 +614,6 @@
	#
	device arcmsr # Areca SATA II RAID

	-#
	-# 3ware 9000 series PATA/SATA RAID controller driver and options.
	-# The driver is implemented as a SIM, and so, needs the CAM infrastructure.
	-#
	-options TWA_DEBUG # 0-10; 10 prints the most messages.
	-device twa # 3ware 9000 series PATA/SATA RAID
	-
	#
	# Adaptec FSA RAID controllers, including integrated DELL controllers,
	# the Dell PERC 2/QC and the HP NetRAID-4M
	Index: sys/modules/Makefile
	===================================================================
	--- sys/modules/Makefile
	+++ sys/modules/Makefile
	@@ -361,7 +361,6 @@
	tmpfs \
	${_toecore} \
	${_tpm} \
	- ${_twa} \
	twe \
	tws \
	uart \
	@@ -719,7 +718,6 @@
	_sdhci_acpi= sdhci_acpi
	_superio= superio
	_tpm= tpm
	-_twa= twa
	_vesa= vesa
	_viawd= viawd
	_vmd= vmd
	Index: sys/modules/twa/Makefile
	===================================================================
	--- sys/modules/twa/Makefile
	+++ /dev/null
	@@ -1,47 +0,0 @@
	-# $FreeBSD$
	-#
	-# Copyright (c) 2004-06 Applied Micro Circuits Corporation.
	-# All rights reserved.
	-#
	-# Redistribution and use in source and binary forms, with or without
	-# modification, are permitted provided that the following conditions
	-# are met:
	-# 1. Redistributions of source code must retain the above copyright
	-# notice, this list of conditions and the following disclaimer.
	-# 2. Redistributions in binary form must reproduce the above copyright
	-# notice, this list of conditions and the following disclaimer in the
	-# documentation and/or other materials provided with the distribution.
	-#
	-# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
	-# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
	-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
	-# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
	-# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	-# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
	-# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
	-# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	-# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	-# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
	-# SUCH DAMAGE.
	-#
	-
	-#
	-# 3ware driver for 9000 series storage controllers.
	-#
	-# Author: Vinod Kashyap
	-# Modifications by: Adam Radford
	-#
	-
	-KMOD = twa
	-.PATH: ${SRCTOP}/sys/dev/${KMOD}
	-SRCS= tw_osl_freebsd.c tw_osl_cam.c \
	- tw_cl_init.c tw_cl_io.c tw_cl_intr.c tw_cl_misc.c \
	- bus_if.h device_if.h pci_if.h opt_scsi.h opt_cam.h opt_twa.h
	-
	-# Uncomment the following line to turn on Enclosure Services support.
	-#CFLAGS+= -DTWA_ENCLOSURE_SUPPORT
	-
	-#CFLAGS+= -DTWA_DEBUG=0
	-CFLAGS+= -I${SRCTOP}/sys/dev/${KMOD}
	-
	-.include <bsd.kmod.mk>
	Index: tools/kerneldoc/subsys/Doxyfile-dev_twa
	===================================================================
	--- tools/kerneldoc/subsys/Doxyfile-dev_twa
	+++ /dev/null
	@@ -1,21 +0,0 @@
	-# Doxyfile 1.5.2
	-
	-# $FreeBSD$
	-
	-#---------------------------------------------------------------------------
	-# Project related configuration options
	-#---------------------------------------------------------------------------
	-PROJECT_NAME = "FreeBSD kernel TWA device code"
	-OUTPUT_DIRECTORY = $(DOXYGEN_DEST_PATH)/dev_twa/
	-EXTRACT_ALL = YES # for undocumented src, no warnings enabled
	-#---------------------------------------------------------------------------
	-# configuration options related to the input files
	-#---------------------------------------------------------------------------
	-INPUT = $(DOXYGEN_SRC_PATH)/dev/twa/ \
	- $(NOTREVIEWED)
	-
	-GENERATE_TAGFILE = dev_twa/dev_twa.tag
	-
	-@INCLUDE_PATH = $(DOXYGEN_INCLUDE_PATH)
	-@INCLUDE = common-Doxyfile
	-

File Metadata

Mime Type: text/plain
Expires: Wed, Oct 21, 3:30 AM (8 h, 5 m)
Storage Engine: blob
Storage Format: Raw Data
Storage Handle: 30573370
Default Alt Text: D33114.1792553434.diff (287 KB)

D33114.1792553434.diffNo OneTemporaryActions

D33114.1792553434.diffView Options

File Metadata

Event Timeline

D33114.1792553434.diff
No OneTemporary
Actions

D33114.1792553434.diff
View Options