Summary of the content on the page No. 1
QLogic HCA and QLogic OFED Software
Install Guide
QLogic OFED Version 1.4
IB0056101-00 G.02
Preliminary
Draft
Summary of the content on the page No. 2
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S Information furnished in this manual is believed to be accurate and reliable. However, QLogic Corporation assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties which may result from its use. QLogic Corporation reserves the right to change product specifications at any time without notice. Applications described in this document for any of these products are for i
Summary of the content on the page No. 3
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S Replace EM64T with 64-bit Intel Xeon. “Supported Linux Distributions” on page 5-1 Added more information on VNIC interface. “OpenSM” on page 6-3 Split the Install section into three separate sec- “Software Installation” on page 5-1 tions. “Configuring Drivers and Services” on page 6-1 “Installation Verification and Additional Settings” on page 7-1 Modified installation checklist. Clarified which driv- “Software Installa
Summary of the content on the page No. 4
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S Sections rearranged, renamed, and expanded due “Choose the Appropriate Download Files” on to multiple install methods. Previous sections: page 5-4 Installing the InfiniPath and Open fabrics RPMs “Install QLogicIB-Basic with the Installer Tool” on Downloading and Unpacking the InfiniPath and page 5-7 OpenFabrics Software “About rpm Installation” on page 5-13 “Install QLogic OFED Using Rocks” on page 5-20 “Install QLogi
Summary of the content on the page No. 5
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S New Appendix for Write Combining. MTRR set- “Write Combining” on page B-1 tings information moved here. Minor wording change to introduction section. “Software Installation” on page 5-1 Added new issue in Troubleshooting. “openmpi_gcc Fails to Install Because of Depen- dency on gfortran (RHEL 4)” on page A-4 Added new issue in Troubleshooting. “Outdated ipath_ether Configuration Setup Generates Error” on page A-5 Config
Summary of the content on the page No. 6
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S Changed title of Troubleshooting issue. Was “Outdated ipath_ether Configuration Setup Generates Error” on page A-5 “ipath_ether Configuration Setup Gener- ates Error” Change main configuration file from /etc/sys- Table C-1 on page C-1 and “SRP” on page 6-4. config/infinipath to /etc/infini- band/openib.conf. The infinipath file is still to enable or disable the ipath_mtrr script. Updated note about setting datagram mo
Summary of the content on the page No. 7
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S Adapter Settings section name changed. Also “Adapter and Other Settings” on page 7-1 added new bullet point “Check the PCIe bus width” Updated tables of RPMs. Moved all documentation All tables in D Package Descriptions RPMs together. New table for OpenSM-Devel RPMs. Updated OtherMPIs. Deleted table 5-7 RPMs to Install, as information is “Using rpm to Install InfiniPath and OpenFabrics” repeated in “Using rpm to Instal
Summary of the content on the page No. 8
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S Notes Page viii Preliminary IB0056101-00 G.02 Draft
Summary of the content on the page No. 9
Table of Contents 1 Introduction Who Should Read this Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1 How this Guide is Organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3 Conventions Used in this Guide.
Summary of the content on the page No. 10
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S Hardware Installation for QLE7240, QLE7280, or QLE7140 with PCI Express Riser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-9 Dual Adapter Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-9 Installation Steps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-9 Hardware Installation for QHT7140 with HTX Riser . . . . . . . . . .
Summary of the content on the page No. 11
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 A Uninstalling Software with Rocks or Platform OCS . . . . . . . . . . . . . . . 5-25 Downgrading RPMs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-25 6 Configuring Drivers and Services InfiniPath and OpenFabrics Driver Overview . . . . . . . . . . . . . . . . . . . . . . . . 6-1 OpenFabrics Drivers and Services Configuration and Startup. . . . . . . . . . . 6-1 Configuring the IPoIB Netwo
Summary of the content on the page No. 12
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S ibsrpdm Command Hangs When Two HCAs are Installed but Only Unit 1 is Connected to the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-5 Outdated ipath_ether Configuration Setup Generates Error . . . . . A-5 B Write Combining Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1 Verify Write Combining is Working . . . . . . . . .
Summary of the content on the page No. 13
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 A List of Figures Figure Page 4-1 QLogic QLE7280 with IBA7220 ASIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7 4-2 QLogic QLE7140 Card with Riser, Top View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7 4-3 QLogic QHT7040/QHT7140 Full and Low Profile Cards with Riser, Top View . . . . 4-8 4-4 PCIe Slot in a Typical Motherboard . . . . . . . . . . . . . . . . . . . . . . . . .
Summary of the content on the page No. 14
QLogic HCA and QLogic OFED Software Install Guide QLogic OFED Version 1.4 S Notes xiv Preliminary IB0056101-00 G.02 Draft
Summary of the content on the page No. 15
1 Introduction This chapter describes the contents, intended audience, and organization of the QLogic HCA and QLogic OFED Software Install Guide. The QLogic HCA and QLogic OFED Software Install Guide contains instructions for installing the QLogic Host Channel Adapters (HCAs) and the QLogic InfiniPath and OpenFabrics software. The following adapters are covered in this guide: ® QLE7140 PCI Express (PCIe) QLE7240 PCI Express QLE7280 PCI Express QHT7040/QHT7140 HyperTransport Expansion (HTX
Summary of the content on the page No. 16
Introduction Overview S Section 3, Step-by-Step Installation Checklist, provides a high-level overview of the hardware and software installation procedures. Section 4, Hardware Installation, includes instructions for installing the QLogic QLE7140, QLE7240, QLE7280, QHT7040, and QHT7140 HCAs. Section 5, Software Installation, includes instructions for installing the QLogic InfiniPath and OpenFabrics software. Section 6, Configuring Drivers and Services, includes instructions for configur
Summary of the content on the page No. 17
Introduction Interoperability A QLogic OFED OpenFabrics software is interoperable with other vendors’ InfiniBand Host Channel Adapters (HCAs) running compatible OpenFabrics releases. There are several options for subnet management in your cluster: Use the embedded Subnet Manager (SM) in one or more managed switches supplied by your InfiniBand switch vendor. Use a host-based Subnet Manager. QLogic provides one, HSM, as a part of the InfiniBand fabric Suite download. Use the Open source Su
Summary of the content on the page No. 18
Introduction Documentation S Table 1-1. Typographical Conventions (Continued) Convention Meaning concept Italic font is used for emphasis and concepts, as well as for docu- mentation names/titles. user input Bold fixed-space font is used for literal items in commands or con- structs that you type. $ Indicates a command line prompt. # Indicates a command line prompt as root. [] Brackets enclose optional elements of a command or program con- struct. ... Ellipses indicate that a preceding element
Summary of the content on the page No. 19
Introduction Contact Information A Contact Information Support Headquarters QLogic Corporation 4601 Dean Lakes Blvd Shakopee, MN 55379 USA QLogic Web Site www.qlogic.com Technical Support Web Site support.qlogic.com Technical Support Email support@qlogic.com Technical Training Email tech.training@qlogic.com Additional contact information is available from the Contact Support area of the Techni- cal Support Web Site. IB0056101-00 G.02 Preliminary 1-5 Draft
Summary of the content on the page No. 20
Introduction Contact Information S Notes 1-6 Preliminary IB0056101-00 G.02 Draft
