Dell Mellanox Family Of Adapters Owners Manual User For ConnectX 3 10/40 Gigabit Ethernet PowerEdge Servers

2014-11-13

• User Manual for Mellanox ConnectX®-3 10/40 Gigabit Ethernet Adapters for Dell PowerEdge Servers
  • Restrictions and Disclaimers
  • Export Regulations
  • Revision History
  • About this Manual
    • Intended Audience
    • Related Documentation
    • Document Conventions
    • Technical Support
  • 1 Introduction
    • 1.1 Functional Description
    • 1.2 Features
      • 1.2.1 Single Root IO Virtualization (SR-IOV)
      • 1.2.2 Remote Direct Memory Access (RDMA)
    • 1.3 Supported Operating Systems/Distributions
  • 2 Adapter Card Interfaces
    • 2.1 I/O Interfaces
      • 2.1.1 Ethernet QSFP+/ SFP+ Interface
      • 2.1.2 LED Assignment
  • 3 Installing the Hardware
    • 3.1 System Requirements
      • 3.1.1 Hardware
      • 3.1.2 Operating Systems/Distributions
      • 3.1.3 Software Stacks
      • 3.1.4 Co-requisites
    • 3.2 Safety Precautions
    • 3.3 Pre-installation Checklist
    • 3.4 Installation Instructions
      • 3.4.1 For Adapters
      • 3.4.2 For Mezzanine Cards
    • 3.5 Connecting the Network Cables
      • 3.5.1 Inserting a Cable into the Adapter Card
      • 3.5.2 Removing a Cable from the Adapter Card
    • 3.6 Identifying the Card in A System
      • 3.6.1 On Linux
  • 4 Driver Installation and Configuration
    • 4.1 Linux Driver
      • 4.1.1 Installation Requirements
      • 4.1.2 Downloading Mellanox OFED
      • 4.1.3 Installing Mellanox OFED
        • 4.1.3.1 Pre-installation Notes
      • 4.1.4 Installation Script
        • 4.1.4.1 mlnxofedinstall Return Codes
      • 4.1.5 Installation Procedure
      • 4.1.6 Installation Results
      • 4.1.7 Post-installation Notes
      • 4.1.8 Uninstalling Mellanox OFED
    • 4.2 Linux Driver Features
      • 4.2.1 iSCSI Extensions for RDMA (iSER)
      • 4.2.2 iSER Initiator
      • 4.2.3 Quality of Service (QoS) Ethernet
        • 4.2.3.1 Mapping Traffic to Traffic Classes
        • 4.2.3.2 Plain Ethernet Quality of Service Mapping
        • 4.2.3.3 RoCE Quality of Service Mapping
        • 4.2.3.4 Raw Ethernet QP Quality of Service Mapping
        • 4.2.3.5 Map Priorities with tc_wrap.py/mlnx_qos
        • 4.2.3.6 Quality of Service Properties
        • 4.2.3.7 Quality of Service Tools
      • 4.2.4 Ethernet Time-Stamping
        • 4.2.4.1 Enabling Time Stamping
        • 4.2.4.2 Getting Time Stamping
        • 4.2.4.3 Querying Time Stamping Capabilities via ethtool
      • 4.2.5 RoCE Time Stamping
        • 4.2.5.1 Query Capabilities
        • 4.2.5.2 Creating Time Stamping Completion Queue
        • 4.2.5.3 Polling a Completion Queue
        • 4.2.5.4 Querying the Hardware Time
      • 4.2.6 Flow Steering
        • 4.2.6.1 Enable/Disable Flow Steering
        • 4.2.6.2 Flow Domains and Priorities
      • 4.2.7 Single Root IO Virtualization (SR-IOV)
        • 4.2.7.1 System Requirements
        • 4.2.7.2 Setting Up SR-IOV
        • 4.2.7.3 Enabling SR-IOV and Para Virtualization on the Same Setup
        • 4.2.7.4 Assigning a Virtual Function to a Virtual Machine
        • 4.2.7.5 Uninstalling SR-IOV Driver
        • 4.2.7.6 Configuring Pkeys and GUIDs under SR-IOV
        • 4.2.7.7 Ethernet Virtual Function Configuration when Running SR-IOV
      • 4.2.8 Ethtool
      • 4.2.9 Ethernet Performance Counters
    • 4.3 VMware Driver
      • 4.3.1 Installing and Running the Driver
        • 4.3.1.1 Installing and Running the VIB Driver on ESXi-5.x
      • 4.3.2 Installing and Running the offline_bundle Driver on ESXi-5.x
      • 4.3.3 Removing the VIB/offline_bundle Driver
    • 4.4 Windows
      • 4.4.1 Installation Requirements
        • 4.4.1.1 Required Disk Space for Installation
      • 4.4.2 Software Requirements
        • 4.4.2.1 Installer Privileges
      • 4.4.3 Downloading Mellanox WinOF
      • 4.4.4 Installing Mellanox WinOF
        • 4.4.4.1 Attended Installation
        • 4.4.4.2 Unattended Installation
      • 4.4.5 Uninstalling Mellanox WinOF
      • 4.4.6 Windows Performance Tuning
    • 4.5 WinOF Features
      • 4.5.1 Configuring Quality of Service (QoS)
      • 4.5.2 RDMA over Converged Ethernet
        • 4.5.2.1 RoCE Configuration
        • 4.5.2.2 Configuring Router (PFC only)
      • 4.5.3 Deploying Windows Server 2012 and 2012 R2 with SMB Direct
        • 4.5.3.1 Hardware and Software Prerequisites
        • 4.5.3.2 SMB Configuration Verification
        • 4.5.3.3 Verifying SMB Connection
        • 4.5.3.4 Verifying SMB Events that Confirm RDMA Connection
  • 5 Remote Boot
    • 5.1 iSCSI Boot
      • 5.1.1 RHEL6.4/RHEL6.5
        • 5.1.1.1 Configuring the iSCSI Target Machine
        • 5.1.1.2 Installing RHEL6.4/RHEL6.5 on a Remote Storage over iSCSI
        • 5.1.1.3 SAN-Booting the Diskless Client with FlexBoot
      • 5.1.2 Booting Windows from an iSCSI Target
        • 5.1.2.1 Configuring the WDS, DHCP and iSCSI Servers
        • 5.1.2.2 Configuring the Client Machine
        • 5.1.2.3 Installing iSCSI
      • 5.1.3 SLES11 SP3
        • 5.1.3.1 Configuring the iSCSI Target Machine
        • 5.1.3.2 Configuring the DHCP Server
        • 5.1.3.3 Installing SLES11 SP3 on a Remote Storage over iSCSI
        • 5.1.3.4 Using PXE Boot Services for Booting the SLES11 SP3 from the iSCSI Target
    • 5.2 PXE Boot
      • 5.2.1 SLES11 SP3
        • 5.2.1.1 Configuring the PXE Server
  • 6 Firmware
  • 7 Dell Lifecycle ControllerTroubleshooting
    • 7.1 General
    • 7.2 Linux
    • 7.3 Windows
  • 8 Specifications
  • 9 Regulatory
    • 9.1 Regulatory Statements
      • 9.1.1 FCC Statements (USA)
      • 9.1.2 EN Statements (Europe)
      • 9.1.3 ICES Statements (Canada)
      • 9.1.4 VCCI Statements (Japan)
      • 9.1.5 KCC Certification (Korea)
  • Appendix A: Configuration for Mellanox Adapters through System Setup
  • Appendix B: Safety Warnings
  • Appendix C: Avertissements de sécurité d’installation (Warnings in French)
  • Appendix D: Sicherheitshinweise (Warnings in German)
  • Appendix E: Advertencias de seguridad para la instalación (Warnings in Spanish)