IBM DS4000/DS5000 Fibre Channel Host Bus Adapter device driver version 7.00a for NetWare NOTE TO SERVICE – Reference RETAIN # N/A (C) Copyright International Business Machines Corporation 1999, 2007. All rights reserved. US Government Users Restricted Rights - Use, duplication, or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Note: Before using this information and the product it supports, read the general information in "Notices and trademarks” in this document. Refer to the IBM Support Web site or the Fibre Channel Host Bus Adapter (FC HBA) support CD that is shipped with your FC HBA for the IBM FAStT/DS4000 FC-2, FC2-133 single and dual port and FC 4Gbps single and dual port Host Bus Adapter Installation and User's guide, the latest information, and updated readme file. Last Update: 05/01/2009 Products supported: --------------------------------------------------------------------------- | FAStT Adapter | Qlogic Adapter | IBM Feature Code | IBM Option P/N| |---------------------------------------------------------------------------| |DS4000/FAStT FC2 | QLA2310FL-IBM-SP | FC2130 | 19K1246 | |---------------------------------------------------------------------------| |DS4000/FAStT FC2-133| QLA2340-IBM-SP | FC2104 | 24P0960 | | | | | 24P0961 | |---------------------------------------------------------------------------| |DS4000 FC 4Gb-Single| QLA2460-IBM-SP | FC2105 | 39M5894 | |---------------------------------------------------------------------------| |DS4000 FC 4Gb-Dual | QLA2462-IBM-SP | FC2106 | 39M5895 | |---------------------------------------------------------------------------| |System x Qlogic FC | QLE2460 | N/A | 39R6525 | |4Gb PCIe-Single | | | | |---------------------------------------------------------------------------| |System x Qlogic FC | QLE2462 | N/A | 39R6527 | |4Gb PCIe-Dual | | | | |---------------------------------------------------------------------------| |System x Qlogic FC | QLE2560 | N/A | 42D0501 | |8Gb PCIe-Single | | | | |---------------------------------------------------------------------------| |System x Qlogic FC | QLE2562 | N/A | 42D0510 | |8Gb PCIe-Dual | | | | --------------------------------------------------------------------------- -------------------------------------------------------------------- | DS4000/DS5000 | Controller Firmware | DS4000 Storage Manager | | Storage Subsystem | Version Required | Version Required | -------------------------------------------------------------------- | DS4100 | 06.12.56.00 or later| 10.50.xx.19 or later | | DS4200 | 07.50.12.00 or later| 10.50.xx.19 or later | | DS4300 (dual | | | | controller)| 06.60.08.xx or later| 10.50.xx.19 or later | | DS4400 | 06.12.56.00 or later| 10.50.xx.19 or later | | DS4500 | 06.60.08.xx or later| 10.50.xx.19 or later | | DS4700 | 07.50.12.00 or later| 10.50.xx.19 or later | | DS4800 | 07.50.12.00 or later| 10.50.xx.19 or later | | DS5100 | 07.50.12.00 or later| 10.50.xx.19 or later | | DS5300 | 07.50.12.00 or later| 10.50.xx.19 or later | -------------------------------------------------------------------- Contents -------- 1. OS Support 2. Supported Features 3. Creating the Driver Diskette 4. DS4000/DS5000 FC Host Bus Adapter BIOS configuration 4.1 Update IBM DS4000 FC HBA BIOS 4.2 Configure the NVRAM settings for the IBM DS4000/DS5000 FC HBAs 5. Installing the NetWare Device Driver 5.1. Initial OS installation with the QLogic Controller as Boot 5.2. Initial OS installation with the QLogic Controller as Add-On 5.3. Adding the IBM DS4000/DS5000 FC HBA to the Existing OS 5.3.1. Adding the Driver for the First Time 5.3.2. Adding additional IBM DS4000/DS5000 FC HBA 5.4. Updating the Existing IBM DS4000/DS5000 FC HBA Driver 6. Installing NetWare Native Multipath Failover Driver 7. Installing QLogic SANsurfer 8. Removing the Driver 9. Device Driver Optional Parameters 9.1. System Parameters 9.2. NVRAM Parameters 10. Known Issues and Limitations 11. Additional Notes 11.1. PCI Hot Plug 11.2. 256 LUN Support 11.3. Server Memory Recommendations 11.4 Debug Driver Information 11.5 Support for DMI and QMS 12. Diskette Content 13. WEB Sites and Support Phone Number 14. Trademarks and Notices 15. Disclaimer ====================================================================== 1. OS Support ------------- This version of the Novell NetWare device driver is supported only in the NetWare 6.5 Support Pack 6 (SP8) and later operating system environment. It is also supported with Novell NetWare 6.5 Cluster Solutions 1.8.2. This driver is not supported in NetWare 5.x or 6.0 operating system environment. Please contact IBM reseller or representatives if you have a need for such operating system environments support. ====================================================================== 2. Supported Features --------------------- * NetWare Multi-Processor Compliance * NetWare PCI Adapter Hot Plug Compliance * NetWare Driver Hot Replace Support * Driver 256 LUN Support * Driver Fabric Name Server Support * Driver Adapter Path LUN Failover * Driver Storage Port LUN Failover * Driver DMI and QMS Support ====================================================================== 3. Creating the Driver Diskette ------------------------------- Important: During NetWare OS installation, the driver must be on a floppy disk. The NetWare installer does not allow swapping of CDs during installation. 1. Download the IBM DS4000/DS5000 Fibre Channel Host Bus Adapter Device Driver: - Refer to the IBM DS4000/DS5000 support web site at http://www.ibm.com/servers/storage/support/disk/ and locate the IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter Device Driver or the appropriate device driver link for your Fibre Channel HBA model. Click the "Link to driver" hot-spot. 2. If prompted with "What would you like to do with this file?” choose "Save this program to disk". 3. Specify a temporary location on your hard drive and download the IBM DS4000/DS5000 Fibre Channel HBA NetWare device driver. 4. Insert a blank diskette in the diskette drive. Unpack (unzip) the file you just downloaded into the diskette, specifying A:\NWUPDATE\DRIVERS as "Unzip To Folder:” ======================================================================= 4. DS4000/DS5000 FC HBA BIOS configuration ----------------------------------- 4.1 Update IBM DS4000/DS5000 Fibre Channel Host Bus Adapter BIOS --------------------------------------------------------- IMPORTANT NOTE: The DS4000/DS5000 Fibre Channel Host Bus Adapter BIOS must be at the latest level that is available in the DS4000/DS5000 System Storage support web site for your IBM DS4000/DS5000 Fibre Channel Host Bus Adapter type. Currently, the latest BIOS version for the IBM FC-2 and FC2-133 HBA is 1.54. Whereas, the BIOS version for the FC 4Gbps/8Gbps single and dual port HBAs is 2.02. Please refer to readme file in the BIOS package of your Fibre Channel HBA for the information on how to update the BIOS and NVRAM settings of your Fibre Channel HBA. CAUTION: If you made modifications to FC HBA NVRAM settings, you will have to make the modifications again because the NVRAM will be reset to the new IBM default values. ====================================================================== 4.2 Configure the NVRAM settings for the IBM Fibre Channel Host Bus Adapter ---------------------------------------------------------------------------- All the NVRAM settings should be at the IBM defaults for the IBM FC 4Gbps adapters. For the FAStT/DS4000/DS5000 FC-2 and FC2-133 adapters, the only change to the default NVRAM settings is the change of the "LUNs per Target settings" from 0 to 256. - Advanced Adapter Settings LUNs per target - 256 To change the LUNs per Target NVRAM setting, perform the following steps. 1. As the server boots, press CTRL-Q when prompted Press for Fast!Util. 2. After the Fast!Util program loads, the display will depend on whether there are multiple IBM Fibre Channel Host Bus Adapter installed: If there are multiple IBM DS4000/DS5000 Fibre Channel Host Bus Adapters, a list of addresses occupied by those Host Adapters will appear. Using the arrow keys, select the desired adapter and press ENTER. The Fast!Util Options menu will then appear. Select Configuration Settings -> Advanced Adapter Settings to get to the Fast!Util menu to change the LUNs per Target setting. Press to save the change. If there are multiple Fibre Channel Host Bus Adapters in the server, use the Select Host Adapter option in the Fast!Util Options menu to select the other adapters for making changes to the LUNs per Target setting. For further information, refer to the appropriate IBM DS4000/DS5000 installation and User's guide for your Fibre Channel Host Bus Adapter model. 3. Reboot the server after updating the NVRAM on all of the Fibre Channel Host Bus Adapters that are installed in the server. ====================================================================== 5. Installing the Device Driver ------------------------------- This section is divided into the following four sections. · Initial operating system installation with the IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter (HBA) or other IBM DS4000/DS5000 Fibre Channel HBA as Boot · Initial operating system installation with the IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter (HBA) or other IBM DS4000/DS5000 Fibre Channel HBA as Add-on/New adapter · Adding the IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter (HBA) or other IBM DS4000/DS5000 Fibre Channel HBA to an existing NetWare 6.5 server · Updating the existing the IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter (HBA) or other IBM DS4000/DS5000 Fibre Channel HBA Device Driver ====================================================================== 5.1. Initial OS installation with the IBM DS4000/DS5000 Fibre Channel HBA as Boot --------------------------------------------------------------------------- Perform the following steps to initially install NetWare operating system on a Fibre Channel (FC) disk LUN attached to the DS4000/DS5000 FC 4Gbps Host Bus Adapter (HBA) or other IBM DS4000/DS5000 Fibre Channel HBA. 1. Enable the Host adapter BIOS NVRAM setting on the IBM DS4000/DS5000 Fibre Channel HBA and select the LUN created on the DS4000/DS5000 storage system as the boot device via the Fast!UTIL. To invoke Fast!UTIL, reboot the server and press when prompted during the server boot process. You might also need to disable the other disk devices that the server might see during boot if these disk devices are scanned before the devices from the Fibre Channel HBAs are being scanned. If the server BIOS allows changing the order of the bus devices scanned, make changes so that the PCI bus(es), in which the FC HBAs are installed, is(are) scanned before the on-board disk controller during the server boot process. 2. If you haven't already done so, install DOS on the boot disk. 3. Follow the standard NetWare instructions to begin the installation. 4. When prompted to "Choose Storage Drivers - Disk Drivers", press to install an unlisted driver. 5. Insert the IBM DS4000/DS5000 Fibre Channel HBA driver diskette, which has the *.ham driver that are included in this device driver package and press to specify the path to the *.HAM driver: A:\NWUPDATE\DRIVERS 6. Press to select the QL2x00.HAM driver. 7. Follow the standard NetWare installation procedure and complete the installation of the NetWare operating system. ====================================================================== 5.2. Initial OS installation with the IBM DS4000/DS5000 Fibre Channel HBA as Add-On/new adapter -------------------------------------------------------------------------- Perform the following steps to install NetWare and the IBM Fibre Channel HBA driver on the disk device that are not attached to the IBM DS4000/DS5000 Fibre Channel HBA ie. server internal IDE/SCSI drive. Same as section 5.1, except that at step 1, the IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter BIOS should be disable. Also ensure that the onboard disk controller is scanned first during the server boot process. ====================================================================== 5.3. Adding the IBM DS4000/DS5000 Fibre Channel HBA to the Existing NetWare Server --------------------------------------------------------------------------- Perform the following steps to add the driver to the existing NetWare system. 5.3.1. Adding the Driver for the First Time ------------------------------------------- For NetWare 6.5 installation only: 1. From the NetWare server prompt, load the appropriate NetWare program: LOAD HDETECT 2. Select and press . 3. Select and press . 4. Select the storage adapters and press . 5. Select and press . 6. Press to install an unlisted driver. 7. Insert the appropriate IBM DS4000/DS5000 Fibre Channel Host Bus Adapter driver diskette and press to specify the path to the QL2x00.HAM driver: A:\NWUPDATE\DRIVERS 8. Press to select the QL2x00.HAM driver. 9. Select and press to load the new driver. 10. Press to exit the installation. You can also perform the following steps to manually install the driver: 1. To manually copy the driver files (QL2X00.HAM and QL2X00.DDI) to the following directory: C:\NWSERVER\DRIVERS for NW6.x There are two ways to copy the files from the diskette to the above directory. a. Using the NetWare Remote Manager: Start the NetWare Remote Manager from the NetWare GUI (execute STARTX to start the NetWare GUI) and copy the files (drag-and- drop) from the floppy disk to one of the directories on the C: driver as specified above. b. Using the DOS copy command: Open the DOS prompt (if your server does not stop at the DOS prompt, press the F8 key just at the same time that the "DOS starting..." string is printed during a hardware boot, and respond to all the questions by pressing "Y", except for the last question "SERVER ?" to this respond press "N"); copy the files from the floppy disk to one of the directories specified above. 2. Start the server and manually load the driver for as many instances of Fibre Channel adapter ports as they are installed in the server; for each manual load take note of the slot number for each instance. 3. Start NWCONFIG and select "NCF Files Options" and select "Edit STARTUP.NCF File" to edit the file STARTUP.NCF; at the end of this file append the following load line once for each instance using the slot number n as noted in the previous paragraph: LOAD QL2X00.HAM SLOT=n /LUNS For example, if you have two IBM DS4000/DS5000 4Gbps FC HBAs adapter boards installed in server PCI bus slots 2 and 3 and if Novell native multi- pathing driver is also used , you would append the following lines LOAD QL2X00.HAM SLOT=2 /LUNS /ALLPATHS /PORTNAMES /XRETRY=400 LOAD QL2X00.HAM SLOT=3 /LUNS /ALLPATHS /PORTNAMES /XRETRY=400 4. Exit from the editor (saving the changes you made), exit from NWCONFIG, and restart the NetWare server. ====================================================================== 5.3.2. Adding Additional IBM DS4000/DS5000 Fibre Channel Host Bus Adapter ------------------------------------------------------------------ To support multiple adapters, add one LOAD command per Fibre Channel Host Bus Adapter port to your STARTUP.NCF file (LOAD QL2x00.HAM SLOT=n). The "SLOT=n" command line parameter is provided for specifying which adapter to configure for each load instance. If the driver is loaded manually from the server prompt and there are multiple adapters, the user will be required to pick a slot number from a list. The IBM DS4000/DS5000 FC Fibre Channel Host Bus Adapter driver reads the onboard NVRAM and supports all of the customized configuration parameters for the adapter and the attached Fibre Channel devices. Use the key combination during boot to access the adapter's configuration options. ====================================================================== 5.4. Updating the Existing IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter Driver -------------------------------------------------------------------- Perform the following steps if the driver is already installed and you want to install an updated version of the driver. For NetWare 6.5: 1. From the NetWare server prompt, load the appropriate NetWare program: LOAD HDETECT 2. Select and press . 3. Select and press . 4. Select the storage adapters and press . 5. Select and press . 6. Press to install an unlisted driver. 7. Insert the IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter driver diskette and press to specify the path to the QLogic HAM driver: A:\NWUPDATE\DRIVERS 8. Press to select the QLogic HAM driver. 9. Select and press to load the new driver. 10. Press to exit the installation. Note: For NetWare 6.0, one must follow a different procedure to update the existing Fibre Channel HBA NetWare device driver. The following procedure is provided for historical purpose only because the NetWare device driver version 6.90o, or higher, does not support NetWare 5.x and 6.0. 1. From the NetWare server prompt, load the NetWare program: LOAD NWCONFIG 2. Select "Driver options" from the main menu. 3. Select "Configure disk and storage device drivers". 4. Select "Deselect a selected driver". 5. Choose the QL2x00 HAM driver 6. Select "Select an additional driver". 7. Press to install an unlisted driver. 8. Insert the IBM DS4000/DS5000 FC 4Gbps Host Bus Adapter driver diskette and press to specify the path to the HAM driver: A:\NWUPDATE\DRIVERS 9. Press to select the QLogic HAM driver and select "Yes" to copy the driver from the diskette to the server directory. 10. Select "Select/Modify drivers parameter". 11. Insert the Slot Number and press . 12. Select "Save parameters and load driver". ====================================================================== 6. NetWare Failover ------------------- ATTENTION: The IBMSAN driver is the supported multi-path failover/failback driver ONLY for Novell NetWare 6.0 with Support Pack 2 or earlier. THIS DRIVE IS NOT SUPPORTED WITH THE IBM DS4000/DS5000 NetWare FC HBA DEVICE DRIVER VERSION 6.90H and higher. IBMSAN can be downloaded from the FAStT/DS4000/DS5000 Support Web site: http://www.ibm.com/servers/storage/support/disk/ for previous version of IBM DS4000/DS5000 NetWare FC HBA device driver. Please refer to the IBMSAN readme included in the IBMSAN package for installation instructions. IMPORTANT: THE DS4000/DS5000 FC HBA NETWARE DEVICE DRIVER VERSION 6.90H AND 6.90O (and later) IS SUPPORTED IN THE NETWARE 6.5 SP6 OR LATER OPERATING SYSTEMS ENVIRONMENT ONLY. Currently, IBM supports only the Novell native NetWare Failover driver as the DS4000/DS5000 storage subsystem multipath and failover driver in Novell NetWare operating system environments. For the Novell native NetWare Failover, you must use Novell NetWare 6.0 with Support Pack 3 or higher or NetWare 6.5 with Support Pack 2-5. In addition, in the DS4000/DS5000 Storage Manager Subsystem Management window, you must select the 'NetWare Failover' Host Type for the storage partition that the Fibre Channel HBA ports in the NetWare server are defined. However, DO NOT USE THE NOVELL MULTIPATH DRIVER MODULES THAT ARE AVAILABLE WITH NetWare 6.5 SUPPORT PACK 5 OR EARLIER or in NetWare 6.0 with Support Pack 3 or higher. There is a fix to the "LUN bouncing between controllers" problem in NetWare 6.5 SP5 and earlier with the IBM DS4000/DS5000 storage subsystems. The problem was observed in large (>6 nodes) NetWare 6.5 cluster with cluster nodes sharing the same IBM DS4000/DS5000 storage subsystem LUNs and being configured with the NetWare native multipath driver. When the nodes in the clusters scan all available Fibre Channel paths to discover LUNs as part of a server boot up process or as the result of executing "scan all LUNs", "scan for new devices" or "list failover devices" commands, the DS4000/DS5000 storage subsystem LUNs will moved between controller A and B momentary. In certain cases, this "LUN bouncing between controllers" behavior could cause the server to mark all paths to a LUN as failed and deactivate it. The fixes required new version of the following Novell NetWare 6.5 CDM and NLM modules. The version and released date of the CDM and NLM modules that these fixes were first incorporated into were as followed: 1. MM.NLM - NetWare 6.5 Media Manager Version 3.22 April 4, 2006 2. NWPA.NLM - NetWare 6.5 NetWare Peripheral Architecture NLM Version 3.21 March 27, 2006 3. SCSIHD.CDM - Novell NetWare SCSI Fixed Disk Custom Device Module Version 3.03.06 March 29, 2006 4. LSIMPE.CDM - LSI Multipath Enhancer Version 1.02.01 March 13, 2006 These files can be downloaded from the Novell support web site - http://support.novell.com/cgi-bin/search/searchtid.cgi?/2973674.htm. Do not use any of the above modules whose release dates and versions are earlier than as stated above. Do not use any of the above modules whose release dates and versions are earlier than as stated above. Furthermore, newer version of these Novell NetWare 6.5 CDM and NLM modules might also be available in the Novell support web site. IBM recommends using the newer versions of these NetWare 6.5 CDM and NLM modules when they are available from the Novell support web site or NetWare 6.5 support packs. In addition, the new Novell NetWare 6.5 native multipath driver requires the Automatic Logical Drive Transfer (ADT, aka. Automatic Volume Transfer (AVT)) mode/function to be disable. The ADT/AVT mode/function must be enable for previous version of LSIMPE.CDM, SCSIHD.CDM, MM.NLM and NWPA.NLM. It must be disabled for the stated above release dates and versions of LSIMPE.CDM, SCSIHD.CDM, MM.NLM and NWPA.NLM modules. To disable the AVT/ADT function so that the new Novell native multipath driver can be used, you must run the DS4000/DS5000 "DisableAVT_Netware.script" SMcli script file. This file is placed in the script folder of the CD- ROM or the DS4000/DS5000 storage manager host software package for Novell NetWare. ATTENTION: If you do not install NetWare 6.5, or if you use an older version of the Novell native multipath driver in NetWare 6.5, do not load the new DS4000/DS5000 NVRAM file that disable AVT/ADT. NetWare hosts with the new Novell native multipath driver (NetWare 6.5) and the previous Novell native multipath driver (NetWare 6.0, or NetWare 6.5 without the new native multipath driver installed) cannot share the same DS4000/DS5000 storage subsystem using the same NetWare failover host type. This is because the different ADT/AVT enable/disable settings of the different Novell native multipath driver versions conflict with each other. RECOMMENDATION: If your system has NetWare version 6.0 (or earlier) installed, it is recommended that you upgrade to NetWare 6.5 to take advantage of this new Novell multipath driver. Always check the Novell support web site for the latest version of the LSIMPE.CDM. In some cases, the IBM Support web site for DS4000/DS5000 product will have the LSIMPE.CDM that one should use until Novell releases the new maintenance pack that incorporates changes between Novell NetWare schedule maintenance releases. Check the README that is included in the DS4000/DS5000 Storage Manager version 9 host software package for Novell NetWare OS for more info. Perform the following steps to configured the NetWare native multipath driver. 1. Install the LSIMPE.CDM by copying it to the c:\NWSERVER or to the location on the server's DOS partition where the server.exe file resides. 2. Install the NWPA.NLM by copying it to the c:\NWSERVER or to the location on the server's DOS partition where the server.exe file resides. 3. Similarly, install the MM.NLM and the SCSIHD.CDM modules 4. Down the server and then restart the server for changes to take effect. 5. After the Device Driver is installed, edit the STARTUP.NCF for the following changes: - Add 'SET MULTI-PATH SUPPORT = ON' - Add 'Load LSIMPE.CDM' before the SCSIHD.CDM driver - Add the 'AEN' command line option to SCSIHD.CDM, example: 'Load SCSIHD.CDM AEN' - Disable all other multi-pathing support. Add /ALLPATHS option to all of the LOAD statements for the IBM DS4000/DS5000 FC HBA ports. Your STARTUP.NCF file should look somewhat like the following: SET Multi-path Support = ON LOAD MPS14.PSM LOAD IDECD.CDM LOAD IDEHD.CDM LOAD LSIMPE.CDM LOAD SCSIHD.CDM AEN LOAD IDEATA.HAM SLOT=10007 LOAD ADPT160M.HAM SLOT=10010 LOAD ADPT160M.HAM SLOT=10011 LOAD QL2300.HAM SLOT=4 /LUNS /ALLPATHS /PORTNAMES /XRETRY=400 LOAD QL2300.HAM SLOT=5 /LUNS /ALLPATHS /PORTNAMES /XRETRY=400 5. Down and restart the server. If your configuration is correct you will not need to set the failover priorities for your LUNs. The MM.NLM module, with a generic enhancer built into the NWPA.NLM module, and the DS4000/DS5000 storage subsystem customization LSIMPE.CDM module provides the multipath support for the DS4000/DS5000 storage subsystem LUNs. The MM.NLM automatically checks for duplicates paths to LUNs and matches them into one. All of the original devices are retyped into FAILOVER DEVICES, which are hidden from the file systems. Each path to the device can be assigned a priority level starting at 0. The lower the number, the higher the priority. This number will determine which path to take. When the current path failed, the path with the next highest priority will be selected. For more info on the Novell multi-path documentation, please refer to the following web link - http://support.novell.com/cgi-bin/search/searchtid.cgi?/10070244.htm. An example of the "list failover devices" output from NetWare console on an IBM DS4000/DS5000 storage device: CHICL1N8:list failover devices 0xAD [V597-A3-D0:11] IBM 1742-900 rev:0520 Up 0x1F [V597-A3-D0:11] IBM 1742-900 rev:0 Priority=1 port = 11100 select Up 0x66 [V597-A4-D0:11] IBM 1742-900 rev:0 Priority=2 port = 651E00 0xAE [V597-A4-D0:18] IBM 1742-900 rev:0520 Up 0x24 [V597-A3-D0:18] IBM 1742-900 rev:0 Priority=2 port = 11100 Up 0x6B [V597-A4-D0:18] IBM 1742-900 rev:0 Priority=1 port = 651E00 select where - "A3" is the adapter number, from the Operating System's view... - "D0" is the "disk" number, from the Operating System's view; which also corresponds to the SSID of the LUN on the DS4000/DS5000 unit. - "port = [number]" is assigned by the OS, and designates each specific controller on the DS4000/DS5000 unit. The port with the "select" label is the "preferred" controller/owner of the LUN in question (as of the initial load of the MM.NLM), and should also have the higher priority path assignment ====================================================================== 7. Install QLogic SANsurfer --------------------------- If QLogic SANsurfer (aka FAStT_MSJ) for Windows/NetWare is not currently installed, you should install it now. The QLogic SANsurfer program must be installed in a Microsoft Windows host that is on the same IP subnet as the NetWare servers. In addition, the Windows host must have the Novell client for Windows installed. During the QLogic SANsurfer program installation, one can specify the installation of the NetWare qlremote agent on the NetWare servers. This program and the NetWare qlremote agent are needed for Fibre path diagnostics and to manage the IBM DS4000/DS5000 Fibre Channel HBA installed in the NetWare servers. See the QLogic SANsurfer publication for instructions on the installation procedure for QLogic SANsurfer and the qlremote agent for WIN32 / NetWare. ====================================================================== 8. Removing the Driver ---------------------- For NetWare 6.5: 1. From the NetWare server prompt, load the appropriate NetWare program: LOAD HDETECT 2. Select and press . 3. Select and press . 4. Select the storage adapters and press . 5. Select and press . 6. Select the correct driver and press . 7. Power down the server and remove the adapter from the server. If you do not want to remove the DS4000/DS5000 Fibre Channel HBA from the server hardware and want to disable the device driver only, perform the following steps: 1. Start NWCONFIG and select "NCF Files Options" and select "Edit STARTUP.NCF File" to edit the file STARTUP.NCF; at the start of each line that loads QL2X00.HAM, insert a pound character (#) to comment out that line; spaces are allowed after the pound character; be sure to do this on all lines that load the QLogic driver. For example, the load lines for two driver instances commented out with the pound (#) character are shown below: # LOAD QL2X00.HAM SLOT=2 /LUNS # LOAD QL2X00.HAM SLOT=3 /LUNS 2. Exit from the editor (saving the changes you made), exit from NWCONFIG, and restart the NetWare server. 3. At any time that you want to reload the QLogic driver again, use NWCONFIG (as in step 0) to edit STARTUP.NCF and remove the pound (#) characters that you inserted in step 1; exit the editor (saving any changes you made), exit NWCONFIG, and restart the server. Note for NetWare 6.0, one must use NWCONFIG instead of HDETECT to remove the device driver. 1. From the NetWare server prompt, load the appropriate NetWare program: LOAD NWCONFIG 2. Select "Driver options" from the main menu. 3. Select "Configure disk and storage device drivers". 4. Select "Deselect a selected driver". ====================================================================== 9. Driver Optional Parameters ------------------------------ The HAM driver has two types of optional parameters: 1. System parameters (specified at driver load time) 2. NVRAM parameters (stored in the adapter NVRAM) These parameters are described in the following subsections. NOTE: Below is a list of all parameters supported by the 7.00a driver. However, for the specific recommended parameters to use for the "new" Novell NetWare 6.5 native multipath driver with the 7.00a HBA driver, refer to the STARTUP.NCF file editing example in section 6 "NetWare Failover". ====================================================================== 9.1. System Parameters ----------------------- Driver optional system parameters are specified on the driver load line. There are two types of driver optional parameters: 1. Global - the parameter acts on all driver instances. 2. Instance - the parameter acts only on the individual driver instance in which it is specified. SLOT=n (Instance) Specifies to NetWare the PCI slot number n of the adapter for which to load this driver instance. LOAD QL2x00.HAM SLOT=3 /LUNS (Instance) Specifies to NetWare to scan for all LUNs during the load of this driver instance (otherwise NetWare will only scan for LUN zero devices). This is required for all DS4000/DS5000 storage subsystems. The scanned LUN number range is 0 to (n - 1) where n is specified by the /MAXLUNS=n option. LOAD QL2x00.HAM SLOT=3 /LUNS /ALLPATHS (Global) Disables the adapter failover and reports to NetWare all devices on all adapter paths. This is required when failover is being handled by an upper layer driver or CDM such as Novell NetWare's native multi-pathing driver. LOAD QL2x00.HAM SLOT=3 /LUNS /ALLPATHS /ALLPORTS (Global) Same as function as /PORTNAMES (see next option). LOAD QL2X00.HAM SLOT=3 /LUNS /ALLPORTS /PORTNAMES (Global) - Disables device port failover on each adapter and reports all ports as individual devices to NetWare. This allows upper layer modules to do failover across a storage subsystem's multiple ports. - internally manages devices by portname rather than nodename. This is required when the LUNs in the storage subsystems do not have a 1:1 correspondence across portnames. LOAD QL2X00.HAM SLOT=3 /LUNS /PORTNAMES /GANXT (Instance) Use GANXT sequence rather than single GNNFT to query the fabric name server. Use for compatibility with name servers that do not support the GNNFT command. LOAD QL2X00.HAM SLOT=3 /LUNS /GANXT /LOGINRETRY=n (Global) Sets the fabric port login retry count to n. When a fabric port login fails, it is retried up to n times. Each retry is delayed one second. LOAD QL2X00.HAM SLOT=3 /LUNS /LOGINRETRY=n /LINKDOWN=n (Global) Sets the link down timeout period to n seconds. This is the timeout period for adapter link down. The expiration of this timeout triggers adapter path failover if multiple paths are available. LOAD QL2X00.HAM SLOT=3 /LUNS /LINKDOWN=n /PORTDOWN=n (Global) Sets the port down timeout period to n seconds. This is the timeout period for storage port down. The expiration of this timeout triggers storage port failover if multiple ports are available. LOAD QL2X00.HAM SLOT=3 /LUNS /PORTDOWN=n /FAILBACK=n (Global) Sets the failback delay time period to n seconds. This is the time period that elapses when some port update has event occurred before any attempt is made to failback paths/ports. LOAD QL2X00.HAM SLOT=3 /LUNS /FAILBACK=n /BACKOFF (Global) Disables failback of I/O after a broken link or a broken port has been restored and made operational. LOAD QL2X00.HAM SLOT=3 /LUNS /BACKOFF /IGNCFG (Global) Avoids reading the configuration file QL2x00.CFG, and allows all devices to be reported to NetWare. This file is located in the default DOS directory (C:\NWSERVER), and is written by QLogic SANsurfer software or the CFG.NLM test tool. LOAD QL2X00.HAM SLOT=3 /LUNS /IGNCFG /REQCFG (Global) Requires reading the configuration file QL2x00.CFG, and ignores all devices not specified therein, or ignores all devices if this file does not exist. This file is located in the default DOS directory (C:\NWSERVER), and is written by QLogic SANsurfer software or the CFG.NLM test tool. LOAD QL2X00.HAM SLOT=3 /LUNS /REQCFG /FDMI (Instance) Registers this adapter's FC-4 symbolic nodename with the fabric name server for FDMI. LOAD QL2X00.HAM SLOT=3 /LUNS /FDMI /HPLEDS (Instance) Enables HP LED control mode. With this option, the LEDs will blink according to HP specification. LOAD QL2X00.HAM SLOT=3 /LUNS /HPLEDS /CONSOLE (Global) Sends system alerts to the server console when failover and failback events occur. LOAD QL2X00.HAM SLOT=3 /LUNS /CONSOLE /FIRSTLID=n (Instance) Sets the loop id of the first logged in device to the value n (hexadecimal). Used for debugging purposes only. LOAD QL2X00.HAM SLOT=3 /LUNS /FIRSTLID=B7 /ELEVATOR=n (Instance) Sets the elevator threshold to n. NetWare uses this value to determine when to do scatter/gather list sorting. /MAXLUNS=n (Global) Sets the maximum number of LUNs allowed during the LUN scan. The range of LUNs searched is 0 to (n-1). LOAD QL2X00.HAM SLOT=3 /LUNS /MAXLUNS=256 /SKIPREPORT (Instance) Avoids executing the SCSI Report LUNs command during LUN scanning. Otherwise, if this option is not used, the information returned by the Report LUNs is used to optimize the LUN scan by doing SCSI Inquiries to the LUNs listed in the Report LUNs information only. LOAD QL2X00.HAM SLOT=3 /LUNS /SKIPREPORT /INQUIRY (Instance) Force SCSI Inquiry commands to be used during the LUN scan regardless of the LUNs listed by the SCSI Report LUNs command. The existence of a particular LUN is determined solely by the results of the Inquiry to that LUN. LOAD QL2X00.HAM SLOT=3 /LUNS /INQUIRY /COMPLEXTAG (Instance) Enables SCSI complex tag queuing; the hacb preserve order control flag maps to ordered queue, and the hacb priority control flag maps to head of queue. Otherwise, if the absence of these hacb control flags maps to simple queue tag. LOAD QL2X00.HAM SLOT=3 /LUNS /COMPLEXTAG /STARTUNIT (Instance) Sends the SCSI Start Unit command to each LUN during the LUN scan; on some multiple port RAID subsystems, this may cause LUN ownership to transfer across ports. LOAD QL2X00.HAM SLOT=3 /LUNS /STARTUNIT /XRETRY=n (Instance) Extends the retry count to n for any IO command that experiences failure with SCSI SK/ASC/ASCQ sense info code in this list - 2/04/01, 6/8B/02. LOAD QL2X00.HAM SLOT=3 /LUNS /XRETRY=10 /XTIMEOUT=n (Instance) Extends the timeout value to n seconds for any IO command that experiences a timeout (ie. held by the storage device for a long period of time). LOAD QL2X00.HAM SLOT=3 /LUNS /XTIMEOUT=120 /QUALIFIED (Instance) Extends the Inquiry data LUN present test to allow RAID subsystem LUNs to be filtered as follows: LUNs returning 20 XX are reported, LUNs returning 20 00 are ignored. LOAD QL2X00.HAM SLOT=3 /LUNS /QUALIFIED /LUNZERO (Instance) Reports to NetWare LUN 0 regardless of its actual existence as determined by the response to the SCSI Inquiry command, as long as the port was logged in. This is used for RAID subsystems that do not report any LUNs on a port until the subsystem is configured for the first time. LOAD QL2X00.HAM SLOT=3 /ALLPATHS /LUNZERO /SPLITNONE (Instance) Disables the split completion timeout. This option overrides the value in NVRAM. LOAD QL2X00.HAM SLOT=3 /LUNS /SPLITNONE /ZIOMODE=n (Instance) Sets the Zero Interupt Operation (ZIO) mode to n. The allowed values of n are as follows: 0 ZIO mode disabled 5 ZIO mode 5 (interrupt after delay) 6 ZIO mode 6 (interrupt after delay or if activity ceases) This option overrides the value in NVRAM. LOAD QL2X00.HAM SLOT=3 /LUNS /ZIOMODE=6 /ZIODELAY=n (Instance) Sets the ZIO interrupt delay period in units of 100 us. This option overrides the value in NVRAM. LOAD QL2X00.HAM SLOT=3 /LUNS /ZIOMODE=6 /ZIODELAY=10 ====================================================================== 9.2. NVRAM Parameters ---------------------- The adapter parameters are stored in the adapter's NVRAM. These parameters are also referred to as the adapter NVRAM parameters. The adapter NVRAM parameters are modified by pressing when prompted by the adapter during server hardware boot to enter Fast!UTIL program or by using the QLogic SANsurfer program and the NetWare qlremote agent. In Fast!UTIL, first select the host adapter, and then select "Configuration Settings". Then, under the following headings, you will see these parameters (This NVRAM parameter listing is applicable only when the adapter BIOS is of version 1.35 or higher for the IBM DS4000/DS5000 FC-2 or FC2-133 HBAs or version 1.12 for the FC 4Gbps HBAs. For adapter with older BIOS, the NVRAM parameters are categorized in a different order) Host Adapter Settings --------------------- + Frame Size : Loop Reset Delay + Adapter Hard Loop Id + Hard Loop Id + Connection Options + Fibre Channel Tape Support + Data Rate Advanced Adapter Settings ------------------------- + Execution Throttle + LUNs Per Target + Enable LIP Reset + Enable LIP Full Login + Enable Target Reset + Login Retry Count + Port Down Retry Count + Link Down Timeout + Drivers Load Risc Code : Extended Error Logging (for FC-2 and FC2-133 HBA only) = RIO Operation Mode = Interrupt Delay Timer @ Enable Interupt (for FC 4Gbps HBA only) Key: + Used by HAM driver (change effects behavior) @ Default value expected by HAM driver (do Not change) = Overridden by HAM driver when specified in the LOAD statement (change has no effect) : Not used by HAM driver (change has no effect) NOTES ----- 0. The QLA200 adapter does not use any of these NVRAM parameters. 1. Enable both of the following to support FC-Tape (see "Extended Firmware Settings" above): Fibre Channel Tape Support 2. The timeout for port down is set [in seconds] by (see "Advanced Adapter Settings" above): Port Down Retry Count The timeout for link down is set [in seconds] by (see "Advanced Adapter Settings" above): Link Down Timeout For best results, set these parameters to 30 or higher (seconds). ====================================================================== 10. Known Issues and Limitations -------------------------------- - The Storage Manager Agent should be unloaded when not used for system configuration. When the Storage Manager Agent is loaded and a fibre path suffers a failure it is possible for the NetWare server to abend. The Storage Manager Agent can be unloaded by following these steps. 1. At the server console type: java -show look for the line "devmgr.versioned.agent.AgentServer.......................QQQ" where QQQ is the process id. 2. At the server console type: java -killQQQ where QQQ is the process id from the java -show command.. - NetWare 6 may install the following in your 'startup.ncf' file" - LOAD QLCSFTE.CDM. You must either remove or comment this line out of your startup.ncf file for use with the IBM FAStT/DS4000/DS5000 Storage Subsystems. - In the xSeries server model x260, x366 and x460 configurations, the DS4000/DS5000 FC 4Gbps single or dual port Fibre Channel Host Bus Adapter (HBA) might cause the servers to intermittently reboot under heavy Fiber Channel I/O. The server RSA II logs might indicate several PERR (PCI Error) machine checks occurred followed by NMI (non-maskable interrupt), which will cause the system to reboot. The problem occurs when the DS4000/DS5000 FC 4Gbps single or dual port Fibre Channel HBA operates at PCI-X mode 2 speed (266 MHz). The workaround is to operate the adapter in PCI-X 2.0 mode 1 (133 MHz). A new version of server BIOS is required for this work-around. The version of this BIOS package will be 1.10 or later. The xSeries server BIOS can be downloaded at the xSeries server support web site - http://www.pc.ibm.com/support. After the new server BIOS is installed, for each PCI-X Slot, the BIOS workaround will determine which PCI-X 2.0 mode to run using the following options which are set in the Setup Configuration Utility: 1. Setup Option 1 (default) - if adapter is not mode 2 capable, then run in mode designated on the adapter. - if adapter is mode 2 capable and if adapter has a Fiber Channel class code, then run in mode 1 - if adapter is mode 2 capable and if adapter does not have a Fibre Channel class code, then run in mode 2 2. Setup Option 2 - User Selectable PCI-X 2.0 mode 1 3. Setup Option 3 - User Selectable PCI-X 2.0 mode 2 if the adapter is mode 2 capable. The BIOS workaround has minor to no performance impact to the operation of the 4Gbps Fiber Channel HBA's while operating in PCI-X 2.0 mode 1 (133MHz). ====================================================================== 11. Additional Notes -------------------- 11.1. PCI Hot Plug ------------------ NOTE: This procedure addresses only the driver-related portion of the PCI Hot Swap operation. The server must have PCI slots that are capable of being powered up and down independently. Depending on the server platform, this entire operation may be controlled by a platform specific software module supplied by the platform vendor. In such a case, these commands are issued directly to the driver and do not require the operator to enter them. The IBM Netfinity PCI Hot Plug System Bus Driver for Novell NetWare V5.1 can be downloaded from the IBM Support Site for your specific server model at http://www.pc.ibm.com/support/us. PCI Hot Plug works by unloading a specific instance of the driver associated with a specific host adapter. It allows a failed adapter to be removed and replaced by an adapter of the same type without disrupting the operation of other adapters of the same type. The steps necessary to perform a hot swap operation are listed below. 1. Type "List Devices" or "List Storage Adapters" at the server console. You will see a list similar to this example: [V591-A0-D0:0] SCSI HARD DISK [V591-A0-D1:0] SCSI HARD DISK [V591-A1-D3:0] SCSI HARD DISK [V591-A1-D4:0] SCSI HARD DISK 2. If it were desired to remove host adapter instance 0 controlling the first two hard drives, the operator would enter the following command: REMOVE STORAGE ADAPTER A0 3. After the host adapter has been replaced, the operator would make the new adapter operational by loading another instance of the driver using the normal load command syntax. For example: LOAD QL2x00.HAM SLOT=3 ====================================================================== 11.2. 256 LUN Support --------------------- To find all LUNS, load the driver with the /LUNS qualifier. For example: LOAD QL2x00.HAM SLOT=3 /LUNS Or, if the driver is already loaded, use the command SCAN ALL LUNS at the server console prompt to find all LUNS (new and existing LUNS). To specify the maximum lun number, use the /MAXLUNS=n option in the driver LOAD statement or set the FC HBA "Luns Per Target" NVRAM parameter using Fast!Util to the required number of luns per target ID. If not using multiple luns per target, set this parameter to 0. The device driver treats this setting as only 1 lun is allowed per target. To set the LUNs Per Target parameter, perform the following steps 1. Reboot the hardware, and press when QL2x00 banner appears to start the Fast!UTIL program. 2. Select the desired adapter. 3. Select Configuration Settings. 4. Select Advanced Adapter Settings. 5. Select LUNs Per Target and change the LUNs Range. 6. Press ESC twice and select Save. 7. Press ESC again and select Reboot. ====================================================================== 11.3 Server Memory Recommendations ---------------------------------- In general, at least 1024MB of memory is recommended. For large numbers of large volumes, at least 2048MB or higher is recommended. ====================================================================== 11.4 Debug Driver Information ----------------------------- This package also includes the debug version of the QL2300 adapter driver. This driver is identified as ql2300d.HAM. The debug driver provides additional driver information that is dumped to an additional console screen. The format for the debug information is as follows. HAM DRIVER DEBUG SCREEN ======================= Every 1 second: A0=0/1234 <0,0> {0,0} 5678 A1=0/1234 <0,0> {0,0} 5678 A0 adapter instance index 0 A1 adapter instance index 1 0/1234 0 unresolved interrupts / 1234 total number of interrupts <0,0> 0 io command timeouts ,0 mailbox command timeouts {0,0} 0 io commands in pending queue structures , 0 io commands currently executing on fibre 5678 5678 seconds since last interrupt occurred Notes: a. Unresolved interrupt is when the isp indicates that it has an interrupt pending, but when the driver reads the completion registers it finds no work to do. b. The total number of io commands outstanding from NetWare is the sum x+y of the numbers inside braces {x,y}. Every 10 seconds: FindTimedOutCommands(An): Searching... An adapter instance index n This indicates that the driver is searching for any IO commands that have not completed and have expired their timeout counter; when none are found, nothing is printed; otherwise the timed out IO commands are identified and then requeued for retry. ====================================================================== 11.5 Support for DMI and QMS ----------------------------- This driver supports DMI (Desktop Management Interface) allowing QLogic adapter information and statistics to be queried by a remote Windows client running the IBM FAStTMSJ. Please refer to the readme file for FAStTMSJ and the IBM FAStTMSJ User's Guide for further information. ====================================================================== 12. Diskette Content -------------------- QL2x00.ddi Driver installation description file QL2x00.ham NetWare HAM driver for supporting the IBM Fibre Channel HBA QL2x00d.ddi Driver installation description file QL2x00d.ham NetWare Debug HAM driver for supporting the IBM Fibre Channel HBA i2xNW680Freadme.TXT This file cfg.nlm Driver configuration utility ====================================================================== 13. WEB Sites and Support Phone Number -------------------------------------- 13.1 IBM System Storage Disk Storage Systems Technical Support web site: http://www.ibm.com/servers/storage/support/disk/ 13.2 IBM System Storage Marketing Web Site: http://www.ibm.com/servers/storage/disk 13.3 If you have any questions about this update, or problem applying the update go to the following HelpCenter World Telephone Numbers URL: http://www.ibm.com/planetwide ====================================================================== 14. Trademarks and Notices -------------------------- 14.1 The following terms are trademarks of the IBM Corporation in the United States or other countries or both: IBM System Storage the e-business logo xSeries HelpCenter Linux is a registered trademark of Linus Torvalds. NetWare is a registered trademark of Novell Inc. Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United states, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Other company, product, and service names may be trademarks or service marks of others. ====================================================================== 15. Disclaimer -------------- 15.1 THIS DOCUMENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. IBM DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE AND MERCHANTABILITY WITH RESPECT TO THE INFORMATION IN THIS DOCUMENT. BY FURNISHING THIS DOCUMENT, IBM GRANTS NO LICENSES TO ANY PATENTS OR COPYRIGHTS. 15.2 Note to Government Users Note to U.S. Government Users -- Documentation related to restricted rights -- Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corporation.