Solarflare Server Adapter User Guide (2013) SF 103837 CD 10 Issue
User Manual: Pdf
Open the PDF directly: View PDF 
.
Page Count: 389
Solarflare Server Adapter
User Guide
Solarflare® Server Adapter User Guide
• Introduction...Page 1
• Installation...Page 19
• Solarflare Adapters on Linux...Page 39
• Solarflare Adapters on Windows...Page 107
• Solarflare Adapters on VMware...Page 252
• Solarflare Adapters on Solaris...Page 279
• SR-IOV Virtualization Using KVM...Page 323
• SR-IOV Virtualization for XenServer...Page 340
• Solarflare Adapters on Mac 0S X...Page 350
• Solarflare Boot ROM Agent...Page 360
Information in this document is subject to change without notice.
© 2008-2013 Solarflare Communications Inc. All rights reserved.
Trademarks used in this text are registered trademarks of Solarflare Communications Inc; Adobe is
a trademark of Adobe Systems. Microsoft and Windows are registered trademarks of Microsoft
Corporation.
Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.
Other trademarks and trade names may be used in this document to refer to either the entities
claiming the marks and names or their products. Solarflare Communications Inc. disclaims any
proprietary interest in trademarks and trade names other than its own.
SF-103837-CD
Last revised: December 2013
Issue 10
Issue 10
© Solarflare Communications 2013
i
Solarflare Server Adapter
User Guide
Table of Contents
Table of Contents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Virtual NIC Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Product Specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Software Driver Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.4 Solarflare AppFlex™ Technology Licensing.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.5 Open Source Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.6 Support and Download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.7 Regulatory Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.8 Regulatory Approval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Chapter 2: Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1 Solarflare Network Adapter Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 Fitting a Full Height Bracket (optional) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 Inserting the Adapter in a PCI Express (PCIe) Slot . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4 Attaching a Cable (RJ-45) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.5 Attaching a Cable (SFP+) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.6 Supported SFP+ Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.7 Supported SFP+ 10G SR Optical Transceivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.8 Supported SFP+ 10G LR Optical Transceivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.9 Supported SFP 1000BASE-T Transceivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.10 Supported 1G Optical Transceivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.11 Supported Speed and Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.12 LED States. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.13 Solarflare Mezzanine Adapter: SFN5802K. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.14 Solarflare Mezzanine Adapters: SFN5812H and SFN5814H. . . . . . . . . . . . . . . . . . 34
2.15 Solarflare Mezzanine Adapter SFN6832F-C61 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.16 Solarflare Mezzanine Adapter SFN6832F-C62 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.17 Solarflare Precision Time Synchronization Adapters . . . . . . . . . . . . . . . . . . . . . . . 38
2.18 Solarflare SFA6902F ApplicationOnload™ Engine . . . . . . . . . . . . . . . . . . . . . . . . . 38
Chapter 3: Solarflare Adapters on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 Linux Platform Feature Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.3 Solarflare RPMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4 Installing Solarflare Drivers and Utilities on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.5 Red Hat Enterprise Linux Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.6 SUSE Linux Enterprise Server Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.7 Unattended Installations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.8 Unattended Installation - Red Hat Enterprise Linux . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.9 Unattended Installation - SUSE Linux Enterprise Server . . . . . . . . . . . . . . . . . . . . . 48
Issue 10
© Solarflare Communications 2013
ii
Solarflare Server Adapter
User Guide
3.10
3.11
3.12
3.13
3.14
3.15
3.16
3.17
3.18
3.19
3.20
3.21
3.22
Hardware Timestamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Configuring the Solarflare Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Setting Up VLANs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Setting Up Teams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Running Adapter Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Running Cable Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Linux Utilities RPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Configuring the Boot ROM with sfboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Upgrading Adapter Firmware with Sfupdate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
License Install with sfkey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Performance Tuning on Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Module Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Linux ethtool Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Chapter 4: Solarflare Adapters on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.2 Windows Feature Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.3 Installing the Solarflare Driver Package on Windows. . . . . . . . . . . . . . . . . . . . . . . 110
4.4 Adapter Drivers Only Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.5 Full Solarflare Package Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.6 Install Drivers and Options From a Windows Command Prompt . . . . . . . . . . . . . 121
4.7 Unattended Installation - Windows Server 2008, 2008 R2, Windows 7 and Windows
Server 2012 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
4.8 Unattended Installation - Windows Server 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.9 Managing Adapters with SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.10 Managing Adapters Remotely with SAM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.11 Using SAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.12 Using SAM to Configure Adapter Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
4.13 Segmentation Offload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.14 Using SAM to Configure Teams and VLANs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
4.15 Using SAM to View Statistics and State Information . . . . . . . . . . . . . . . . . . . . . . 167
4.16 Using SAM to Run Adapter and Cable Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . 168
4.17 Using SAM for Boot ROM Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
4.18 Managing Firmware with SAM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
4.19 Configuring Network Adapter Properties in Windows. . . . . . . . . . . . . . . . . . . . . 180
4.20 Windows Command Line Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
4.21 Sfboot: Boot ROM Configuration Tool. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
4.22 Sfupdate: Firmware Update Tool. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
4.23 Sfteam: Adapter Teaming and VLAN Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
4.24 Sfcable: Cable Diagnostics Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
4.25 Sfnet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
4.26 Completion codes (%errorlevel%) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
4.27 Teaming and VLANs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
4.28 Performance Tuning on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Issue 10
© Solarflare Communications 2013
iii
Solarflare Server Adapter
User Guide
4.29 Windows Event Log Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
Chapter 5: Solarflare Adapters on VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
5.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
5.2 VMware Feature Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
5.3 Installing Solarflare Drivers and Utilities on VMware. . . . . . . . . . . . . . . . . . . . . . . 254
5.4 Configuring Teams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
5.5 Configuring VLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
5.6 Running Adapter Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
5.7 Configuring the Boot ROM with Sfboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
5.8 Upgrading Adapter Firmware with Sfupdate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
5.9 Performance Tuning on VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Chapter 6: Solarflare Adapters on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
6.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
6.2 Solaris Platform Feature Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
6.3 Installing Solarflare Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
6.4 Unattended Installation Solaris 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
6.5 Unattended Installation Solaris 11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
6.6 Configuring the Solarflare Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
6.7 Setting Up VLANs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
6.8 Solaris Utilities Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
6.9 Configuring the Boot ROM with sfboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
6.10 Upgrading Adapter Firmware with Sfupdate . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
6.11 Performance Tuning on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
6.12 Module Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
6.13 Kernel and Network Adapter Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
Chapter 7: SR-IOV Virtualization Using KVM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
7.1 Supported Platforms and Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
7.2 Linux KVM SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
7.3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
7.4 Configuration Red Hat 6.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
7.5 Configuration Red Hat 6.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
7.6 Performance Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
7.7 Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
Chapter 8: SR-IOV Virtualization for XenServer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
8.1 Supported Platforms and Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
8.2 XenServer6 SR-IOV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
8.3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
8.4 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
8.5 Performance Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
Issue 10
© Solarflare Communications 2013
iv
Solarflare Server Adapter
User Guide
Chapter 9: Solarflare Adapters on Mac 0S X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
9.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
9.2 Supported Hardware Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
9.3 Mac 0S X Platform Feature Set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
9.4 Thunderbolt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
9.5 Driver Install. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
9.6 Interface Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
9.7 Tuning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
9.8 Driver Properties via sysctl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
9.9 Firmware Update. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
9.10 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
Chapter 10: Solarflare Boot ROM Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
10.1 Configuring the Solarflare Boot ROM Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
10.2 PXE Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
10.3 iSCSI Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
10.4 Configuring the iSCSI Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
10.5 Configuring the Boot ROM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
10.6 DHCP Server Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
10.7 Installing an Operating System to an iSCSI target. . . . . . . . . . . . . . . . . . . . . . . . . 372
10.8 Default Adapter Settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
Issue 10
© Solarflare Communications 2013
v
Solarflare Server Adapter
User Guide
Chapter 1: Introduction
This is the User Guide for Solarflare® Server Adapters. This chapter covers the following topics:
• Virtual NIC Interface...Page 1
• Advanced Features and Benefits...Page 2
• Product Specifications...Page 4
• Software Driver Support on page 13
• Solarflare AppFlex™ Technology Licensing....Page 13
• Open Source Licenses...Page 14
• Support and Download...Page 15
• Regulatory Information...Page 15
• Regulatory Approval...Page 16
NOTE: Throughout this guide the term Onload refers to both OpenOnload® and EnterpriseOnload®
unless otherwise stated. Users of Onload should refer to the Onload User Guide, SF-104474-CD,
which describes procedures for download and installation of the Onload distribution, accelerating
and tuning the application using Onload to achieve minimum latency and maximum throughput.
1.1 Virtual NIC Interface
Solarflare’s VNIC architecture provides the key to efficient server I/O and is flexible enough to be
applied to multiple server deployment scenarios. These deployment scenarios include:
• Kernel Driver – This deployment uses an instance of a VNIC per CPU core for standard operating
system drivers. This allows network processing to continue over multiple CPU cores in parallel.
The virtual interface provides a performance-optimized path for the kernel TCP/IP stack and
contention-free access from the driver, resulting in extremely low latency and reduced CPU
utilization.
• Accelerated Virtual I/O – The second deployment scenario greatly improves I/O for virtualized
platforms. The VNIC architecture can provide a VNIC per Virtual Machine, giving over a
thousand protected interfaces to the host system, granting any virtualized (guest) operating
system direct access to the network hardware. Solarflare's hybrid SR-IOV technology, unique to
Solarflare Ethernet controllers, is the only way to provide bare-metal I/O performance to
virtualized guest operating systems whilst retaining the ability to live migrate virtual machines.
• OpenOnload™ – The third deployment scenario aims to leverage the host CPU(s) to full
capacity, minimizing software overheads by using a VNIC per application to provide a kernel
bypass solution. Solarflare has created both an open-source and Enterprise class highperformance application accelerator that delivers lower and more predictable latency and
higher message rates for TCP and UDP-based applications, all with no need to modify
applications or change the network infrastructure. To learn more about the open source
Issue 10
© Solarflare Communications 2013
1
Solarflare Server Adapter
User Guide
OpenOnload project or EnterpriseOnload, download the Onload user guide (SF-104474-CD) or
contact your reseller.
Advanced Features and Benefits
Virtual NIC support
The core of Solarflare technology. Protected VNIC interfaces can
be instantiated for each running guest operating system or
application, giving it a direct pipeline to the Ethernet network.
This architecture provides the most efficient way to maximize
network and CPU efficiency. The Solarflare Ethernet controller
supports up to 1024 vNIC interfaces per port.
On IBM System p servers equipped with Solarflare adapters,
each adapter is assigned to a single Logical Partition (LPAR)
where all VNICS are available to the LPAR.
PCI Express
Implements PCI Express 3.0.
High Performance
Support for 40G Ethernet interfaces and a new internal
datapath micro architecture.
Hardware Switch Fabric
Full hardware switch fabric in silicon capable of steering any
flow based on Layer 2, Layer 3 or application level protocols
between physical and virtual interfaces. Supporting an open
software defined network control plane with full PCI-IOV
virtualization acceleration for high performance guest operating
systems and virtual applications.
Improved flow processing
The addition of dedicated parsing, filtering, traffic shaping and
flow steering engines which are capable of operating flexibly
and with an optimal combination of a full hardware data plane
with software based control plane.
TX PIO
Transmit Programmed input/output is the direct transfer of data
to the adapter without CPU involvement. As an alternative to
the usual bus master DMA method, TX PIO improves latency
and is especially useful for smaller packets.
Multicast Replication
Received multicast packets are replicated in hardware and
delivered to multiple receive queues.
Sideband management
NCSI RMII interface for base board management integration.
SMBus interface for legacy base board management integration.
Issue 10
© Solarflare Communications 2013
2
Solarflare Server Adapter
User Guide
PCI Single-Root-IOV, SR-IOV,
capable
127 Virtual functions per port.
Flexible deployment of 1024 channels between Virtual and
Physical Functions.
Support Alternate Routing ID (ARI).
SR-IOV is not supported for Solarflare adapters on IBM System p
servers.
10-gigabit Ethernet
Supports the ability to design a cost effective, high performance
10 Gigabit Ethernet solution.
Receive Side Scaling (RSS)
IPv4 and IPv6 RSS raises the utilization levels of multi-core
servers dramatically by distributing I/O load across all CPUs and
cores.
Stateless offloads
Through the addition of hardware based TCP segmentation and
reassembly offloads, VLAN, VxLAN and FCOE offloads.
Transmit rate pacing (per
queue)
Provides a mechanism for enforcing bandwidth quotas across all
guest operating systems. Software re-programmable on the fly
to allow for adjustment as congestion increases on the network.
Jumbo frame support
Support for up to 9216 byte jumbo frames.
MSI-X support
1024 MSI-X interrupt support enables higher levels of
performance.
Can also work with MSI or legacy line based interrupts.
Ultra low latency
Cut through architecture. < 7μs end to end latency with
standard kernel drivers, < 3μs with Onload drivers.
Remote boot
Support for PXE boot 2.1 and iSCSI Boot provides flexibility in
cluster design and diskless servers (see Solarflare Boot ROM
Agent on page 360).
Network boot is not supported for Solarflare adapters on IBM
System p servers.
MAC address filtering
Enables the hardware to steer packets based on the MAC
address to a VNIC.
Hardware timestamps
The Solarflare Flareon™ SFN7000 series adapters can support
hardware timestamping for all received network packets including PTP.
The SFN5322F and SFN6322F adapters can generate hardware
timestamps of PTP packets.
Issue 10
© Solarflare Communications 2013
3
Solarflare Server Adapter
User Guide
1.2 Product Specifications
Solarflare Flareon™ Network Adapters
Solarflare Flareon™ Ultra SFN7322F Dual-Port 10GbE PCIe 3.0 Server I/O Adapter
Part number
SFN7322F
Controller silicon
SFC9120
Power
5.9W typical
PCI Express
8 lanes Gen 3 (8.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes (factory enabled)
PTP and hardware timestamps
Yes (factory enabled)
1PPS
Optional bracket and cable assembly - not factory installed.
SR-IOV
Yes
Network ports
2 x SFP+ (10G/1G)
Solarflare Flareon™ Ultra SFN7122F Dual-Port 10GbE PCIe 3.0 Server I/O Adapter
Issue 10
Part number
SFN7122F
Controller silicon
SFC9120
Power
5.9W typical
PCI Express
8 lanes Gen 3 (8.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes (factory enabled)
PTP and hardware timestamps
AppFlex™ license required
1PPS
Optional bracket and cable assembly - not factory installed.
SR-IOV
Yes
Network ports
2 x SFP+ (10G/1G)
© Solarflare Communications 2013
4
Solarflare Server Adapter
User Guide
Solarflare Flareon™ SFN7022F Dual-Port 10GbE PCIe 3.0 Server I/O Adapter
Issue 10
Part number
SFN7022F
Controller silicon
SFC9120
Power
5.9W typical
PCI Express
8 lanes Gen 3 (8.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
AppFlex™ license required
PTP and hardware timestamps
AppFlex™ license required
1PPS
Optional bracket and cable assembly - not factory installed.
SR-IOV
Yes
Network ports
2 x SFP+ (10G/1G)
© Solarflare Communications 2013
5
Solarflare Server Adapter
User Guide
Solarflare Onload Network Adapters
Solarflare SFN5121T Dual-Port 10GBASE-T Server Adapter
Part number
SFN5121T
Controller silicon
SFL9021
Power
12.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Network ports
2 x 10GBASE-T (10G/1G/100M)
Solarflare SFN5122F Dual-Port 10G SFP+ Server Adapter
Part number
SFN5122F
Controller silicon
SFC9020
Power
4.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Network ports
2 x SFP+ (10G/1G)
Solarflare SFN4112F Single-Port SFP+ Server Adapter
Part number
SFN4112F
Controller silicon
SFC4000
Power
< 4.5W typical, <5.2W max (with direct attach module)
< 5.3W typical, <6.1W max (with 10GBASE-SR SFP+ module)
Issue 10
PCI Express
8 lanes Gen1 (2.5GT/s)
Virtual NIC support
1024 vNIC interfaces
Supports OpenOnload
Yes
© Solarflare Communications 2013
6
Solarflare Server Adapter
User Guide
SR-IOV
No
Network ports
1 x SFP+ (10G)
Solarflare SFN5322F Dual-Port 10GbE Precision Time Stamping Server Adapter
Part number
SFN5322F
Controller silicon
SFC9020
Power
4.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Network ports
2 x SFP+ (10G/1G)
Solarflare SFN6122F Dual-Port 10GbE SFP+ Server Adapter
Part number
SFN6122F
Controller silicon
SFC9020
Power
5.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes1
Network ports
2 x SFP+ (10G/1G)
Regulatory Product Code
S6102
1. SR-IOV is not supported for Solarflare adapters on IBM System p servers.
Issue 10
© Solarflare Communications 2013
7
Solarflare Server Adapter
User Guide
Solarflare SFN6322F Dual-Port 10GbE SFP+ Server Adapter
Part number
SFN6122F
Controller silicon
SFC9020
Power
5.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Network ports
2 x SFP+ (10G/1G)
Solarflare SFA6902F Dual-Port 10GbE SFP+ ApplicationOnload™ Engine
Issue 10
Part number
SFA6902F
Controller silicon
SFC9020
Power
25W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Network ports
2 x SFP+ (10G/1G)
© Solarflare Communications 2013
8
Solarflare Server Adapter
User Guide
Solarflare Performant Network Adapters
Solarflare SFN5161T Dual-Port 10GBASE-T Server Adapter
Part number
SFN5161T
Controller silicon
SFL9021
Power
12.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s)
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
No
SR-IOV
Yes
Network ports
2 x 10GBASE-T (10G/1G/100M)
Solarflare SFN5151T Single-Port 10GBASE-T Server Adapter
Part number
SFN5151T
Controller silicon
SFL9021
Power
7.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s)
Virtual NIC support
1024 vNIC interfaces
Supports OpenOnload
No
SR-IOV
Yes
Network ports
1 x 10GBASE-T (10G/1G/100M)
Solarflare SFN5162F Dual-Port 10G SFP+ Server Adapter
Issue 10
Part number
SFN5162F
Controller silicon
SFC9020
Power
4.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s)
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
No
SR-IOV
Yes1
© Solarflare Communications 2013
9
Solarflare Server Adapter
User Guide
Network ports
2 x SFP+ (10G/1G)
Solarflare SFN5152F Single-Port 10G SFP+ Server Adapter
Part number
SFN5152F
Controller silicon
SFC9020
Power
4.0W typical
PCI Express
8 lanes Gen2 (5.0GT/s)
Virtual NIC support
1024 vNIC interfaces
Supports OpenOnload
No
SR-IOV
Yes
Network ports
1 x SFP+ (10G/1G)
1. SR-IOV is not supported for Solarflare adapters on IBM System p servers.
Issue 10
© Solarflare Communications 2013
10
Solarflare Server Adapter
User Guide
Solarflare Mezzanine Adapters
Solarflare SFN5812H Dual-Port 10G Ethernet Mezzanine Adapter
Part number
SFN5812H
Controller silicon
SFC9020
Power
3.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Ports
2 x 10GBASE-KX4 backplane transmission
Solarflare SFN5814H Quad-Port 10G Ethernet Mezzanine Adapter
Part number
SFN5814H
Controller silicon
2 x SFC9020
Power
7.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Ports
4 x 10GBASE-KX4 backplane transmission
Solarflare SFN5802K Dual-Port 10G Ethernet Mezzanine Adapter
Issue 10
Part number
SFN5802K
Controller silicon
SFC9020
Power
7.8W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
© Solarflare Communications 2013
11
Solarflare Server Adapter
User Guide
Ports
2 x 10GBASE-KR backplane transmission
Solarflare SFN6832F Dual-Port 10GbE SFP+ Mezzanine Adapter
Part number
SFN6832F-C61 for DELL PowerEdge C6100 series
SFN6832F-C62 for DELL PowerEdge C6200 series
Controller silicon
SFC9020
Power
5.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Ports
2 x SFP+ (10G/1G)
Regulatory Product Code
S6930
Solarflare SFN6822F Dual-Port 10GbE SFP+ FlexibleLOM Onload Server Adapter
Issue 10
Part number
SFN6822F
Controller silicon
SFC9020
Power
5.9W typical
PCI Express
8 lanes Gen2 (5.0GT/s), 127 SR-IOV virtual functions per port
Virtual NIC support
1024 vNIC interfaces per port
Supports OpenOnload
Yes
SR-IOV
Yes
Ports
2 x SFP+ (10G/1G)
© Solarflare Communications 2013
12
Solarflare Server Adapter
User Guide
1.3 Software Driver Support
• Windows® Server 2003 (32 bit and 64 bit).
• Windows® Server 2008 (32 bit and 64 bit) - including R2 release.
• Windows® Server 2012.
• Windows® 7 (32 bit and 64 bit).
• Windows® XP (32 bit and 64 bit).
• Microsoft® Hyper-V™ Server 2008 R2.
• Linux® 2.6 and 3.x Kernels (32 bit and 64 bit) for the following distributions: RHEL 5, 6 and MRG.
SLES 10, 11 and SLERT.
• VMware® ESX™ 5.0 and ESXi™ 5.1, vSphere™ 4.0 and vSphere™ 4.1.
• Citrix XenServer™ 5.6, 6.0 and Direct Guest Access.
• Linux® KVM.
• Solaris™ 10 updates 8, 9 and 10 and Solaris™ 11 (GLDv3).
• Mac OS X Snow Leopard 10.6.8 (32 bit and 64 bit), OS X Lion 10.7.0 and later releases, OS X
Mountain Lion 10.8.0 and later, OS X Mavericks 10.9.
Solarflare SFN5162F and SFN6122F adapters are supported on the IBM POWER architecture (PPC64)
running RHEL 6.4 on IBM System p servers.
Drivers supporting the SFN7000 series adapters are currently only available for Linux platforms.
The Solarflare accelerated network middleware, OpenOnload and EnterpriseOnload, is supported
on all Linux variants listed above, and is available for all Solarflare Onload network adapters.
Solarflare are not aware of any issues preventing OpenOnload installation on other Linux variants
such as Ubuntu, Gentoo, Fedora and Debian variants.
1.4 Solarflare AppFlex™ Technology Licensing.
Solarflare AppFlex technology allows Solarflare server adapters to be selectively configured to
enable on-board applications. AppFlex licenses are required to enable selected functionality on the
Solarflare Flareon™ adapters and the AOE ApplicationOnload™ Engine.
Customers can obtain access to AppFlex applications via their Solarflare sales channel by obtaining
the corresponding AppFlex authorization code. The authorization code allows the customer to
generate licenses at the MyAppFlex page at https://support.solarflare.com/myappflex.
The sfkey utility application is used to install the generated license key file on selected adapters. For
detailed instructions for sfkey and license installation refer to License Install with sfkey on page 75.
Issue 10
© Solarflare Communications 2013
13
Solarflare Server Adapter
User Guide
1.5 Open Source Licenses
1.4.1 Solarflare Boot Manager
The Solarflare Boot Manager is installed in the adapter's flash memory. This program is free
software; you can redistribute it and/or modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2 of the License, or (at your option)
any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without
even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
The latest source code for the Solarflare Boot Manager can be download from https://
support.solarflare.com/. If you require an earlier version of the source code, please e-mail
support@solarflare.com.
1.4.2 Controller Firmware
The firmware running on the SFC9xxx controller includes a modified version of libcoroutine. This
software is free software published under a BSD license reproduced below:
Copyright (c) 2002, 2003 Steve Dekorte
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted
provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list of conditions and the
following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and
the following disclaimer in the documentation and/or other materials provided with the
distribution.
Neither the name of the author nor the names of other contributors may be used to endorse or
promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY
WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Issue 10
© Solarflare Communications 2013
14
Solarflare Server Adapter
User Guide
1.6 Support and Download
Solarflare network drivers, RPM packages and documentation are available for download from
https://support.solarflare.com/.
Software and documentation for OpenOnload is available from www.openonload.org.
1.7 Regulatory Information
Warnings
Do not install the Solarflare network adapter in hazardous areas where highly combustible or
explosive products are stored or used without taking additional safety precautions. Do not expose
the Solarflare network adapter to rain or moisture.
The Solarflare network adapter is a Class III SELV product intended only to be powered by a certified
limited power source.
The equipment has been tested and found to comply with the limits for a Class B digital device,
pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection
against harmful interference in a residential installation. The equipment generates, uses and can
radiate radio frequency energy and, if not installed and used in accordance with the instructions,
may cause harmful interference to radio communications. However, there is no guarantee that
interference will not occur in a particular installation.
If the equipment does cause harmful interference to radio or television reception, which can be
determined by turning the equipment off and on, the user is encouraged to try to correct the
interference by one or more of the following measures:
• Reorient or relocate the receiving antenna.
• Increase the separation between the equipment and receiver.
• Connect the equipment into an outlet on a circuit different from that to which the receiver is
connected.
• Consult the dealer or an experienced radio/TV technician for help.
Changes or modifications not expressly approved by Solarflare Communications, the party
responsible for FCC compliance, could void the user's authority to operate the equipment.
This Class B digital apparatus complies with Canadian ICES-003.
Cet appareil numérique de la classe B est conforme à la norme NMB-003 du Canada.
Underwriters Laboratory Inc ('UL') has not tested the performance or reliability of the security or
signaling aspects of this product. UL has only tested for fire, shock or casualty hazards as outlined in
the UL's Standard for Safety UL 60950-1. UL Certification does not cover the performance or
reliability of the security or signaling aspects of this product. UL makes no representations,
warranties or certifications whatsoever regarding the performance or reliability of any security or
signaling related functions of this product.
Issue 10
© Solarflare Communications 2013
15
Solarflare Server Adapter
User Guide
Laser Devices
The laser safety of the equipment has been verified using the following certified laser device module
(LDM):
Manufactuer
Model
Avago Technologies
Finisar Corporation
AFBR-703SDZ
FTLX8571D3BCL
CDRH Accession
No
9720151-072
9210176-094
Mark of
conformity
TUV
TUV
File No
R72071411
R72080250
When installed in a 10Gb ETHERNET NETWORK INTERFACE CARD FROM THE Solarflare SFN5000,
SFN6000 or SFN7000 SERIES, the laser emission levels remain under Class I limits as specified in the
FDA regulations for lasers, 21 CFR Part 1040.
The decision on what LDMs to use is made by the installer. For example, equipment may use one of
a multiple of different LDMs depending on path length of the laser communication signal. This
equipment is not basic consumer ITE.
The equipment is installed and maintained by qualified staff from the end user communications
company or subcontractor of the end user organization. The end product user and/or installer are
solely responsible for ensuring that the correct devices are utilized in the equipment and the
equipment with LDMs installed complies with applicable laser safety requirements.
1.8 Regulatory Approval
The information in this section is applicable to SFN5121T, SFN5151T, SFN5161T, SFN5152F and
SFN5162F Solarflare network adapters:
Category
Specification
Europe
EMC
Safety1
RoHS
US
Canada
Europe
US
Canada
CB
Europe
Details
BS EN 55022:2006
BS EN 55024:1998 +A1:2001 +A2:2003
FCC Part 15 Class B
ICES 003/NMB-003 Class B
BS EN 60950-1:2006 +A11:2009
UL 60950-1 2nd Ed.
CSA C22.2 60950-1-07 2nd Ed.
IEC 60950-1:2005 2nd Ed.
Complies with EU directive 2002/95/EC
1. The safety assessment has been concluded on this product as a component/sub-assembly only.
Additional Regulatory Information for SFN5122F, SFN5322F, SFN6122F, SFN6322F
, SFA6902F, SFN7022F,SFN7122F and SFN7322F adapters.
ሶቯቒ㍔⫀⑵䚕孔函䷘榊㽱椫⹂呹尞Ⓟ◣巿↩᧤9&&,᧥ቑ㲨䄥⪉ቈሲኌኖ $ ㍔⫀㔏
嫢孔函ቊሼᇭሶቑ孔函ት⹅ㄼ䜿⬒ቊ∎䞷ሼቮቋ榊㽱ⰷ⹂ትㆤሰ怆ሶሼሶቋሯሥቭቡሼᇭ
ቀቑቫሩቍ椫⹂ሯ䤉䞮ሺቂ椪ᇬ∎䞷劔ቒ拸⒖ቍ⺍㉫ሯ㉔尐ቋቍቮ⫃⚗ሯሥቭቡሼ
Issue 10
© Solarflare Communications 2013
16
Solarflare Server Adapter
User Guide
巵⛙∎䞷劔᧶
抨㢾䟁櫭䤓彖岙䞱❐᧨⦷⻔⇞䤓䜿⬒₼∎䞷㣑᧨♾厌㦒抯㒟⺓櫊㞍᧨⦷抨䲽㍔㽐ₚ᧨∎
䞷劔㦒嬺尐㻑㘰♥㩟K拸䠅䤓⺜䷥
$ 鞾韥韥 꽺ꓩ끞ꗞꭖ뭪겕韥韥 넩韥韥鱉꽺ꓩ끞 $ 鞾 냱ꈑ놹녅볁놶뼞麦ꈒ냹뼑
韥韥넩꿙鱽 볅ꎙ녅 鿅鱉 ꩡ끞녅鱉 넩 뇅냹 늱넍뼍겑韥 ꗉꄱꐥ 閵뇊뀭넍 덵꾢꾅 ꩡ끞뼍鱉
阸냹ꑞ놶냱ꈑ뼞鱽鲙
Category
Specification
Europe
EMC
Safety1
RoHS
US
Canada
Taiwan
Japan
South Korea
Australia
Europe
US
Canada
CB
Europe
Details
BS EN 55022:2010 + A1:2007
BS EN 55024:1998 +A1:2001 +A2:2003
FCC Part 15 Class B
ICES 003/NMB-003 Class B
CNS 13438:2006 Class B
VCCI Regulations V-3:2010 Class B
KCC KN-22, KN-24
AS/NZS CISPR 22:2009
BS EN 60950-1:2006 +A11:2009
UL 60950-1 2nd Ed.
CSA C22.2 60950-1-07 2nd Ed.
IEC 60950-1:2005 2nd Ed.
Complies with EU directive 2011/65/EU
1. The safety assessment has been concluded on this product as a component/sub-assembly only.
Additional Regulatory Information for SFN5812H, SFN5814H SFN5802K and
SFN6832F adapters.
ሶቯቒ㍔⫀⑵䚕孔函䷘榊㽱椫⹂呹尞Ⓟ◣巿↩᧤9&&,᧥ቑ㲨䄥⪉ቈሲኌኖ $ ㍔⫀㔏
嫢孔函ቊሼᇭሶቑ孔函ት⹅ㄼ䜿⬒ቊ∎䞷ሼቮቋ榊㽱ⰷ⹂ትㆤሰ怆ሶሼሶቋሯሥቭቡሼᇭ
ቀቑቫሩቍ椫⹂ሯ䤉䞮ሺቂ椪ᇬ∎䞷劔ቒ拸⒖ቍ⺍㉫ሯ㉔尐ቋቍቮ⫃⚗ሯሥቭቡሼ
巵⛙∎䞷劔᧶
抨㢾䟁櫭䤓彖岙䞱❐᧨⦷⻔⇞䤓䜿⬒₼∎䞷㣑᧨♾厌㦒抯㒟⺓櫊㞍᧨⦷抨䲽㍔㽐ₚ᧨∎
䞷劔㦒嬺尐㻑㘰♥㩟K拸䠅䤓⺜䷥
Issue 10
© Solarflare Communications 2013
17
Solarflare Server Adapter
User Guide
Category
Specification
Europe
EMC
Safety1
RoHS
US
Canada
Taiwan
Japan
Australia
Europe
US
Canada
CB
Europe
Details
BS EN 55022:2006
BS EN 55024:1998 +A1:2001 +A2:2003
FCC Part 15 Class B
ICES 003/NMB-003 Class B
CNS 13438:2006 Class A
VCCI Regulations V-3:2010 Class A
AS/NZS CISPR 22:2009
BS EN 60950-1:2006 +A11:2009
UL 60950-1 2nd Ed.
CSA C22.2 60950-1-07 2nd Ed.
IEC 60950-1:2005 2nd Ed.
Complies with EU directive 2002/95/EC
1. The safety assessment has been concluded on this product as a component/sub-assembly only.
Issue 10
© Solarflare Communications 2013
18
Solarflare Server Adapter
User Guide
Chapter 2: Installation
This chapter covers the following topics:
• Solarflare Network Adapter Products...Page 20
• Fitting a Full Height Bracket (optional)...Page 21
• Inserting the Adapter in a PCI Express (PCIe) Slot...Page 22
• Attaching a Cable (RJ-45)...Page 23
• Attaching a Cable (SFP+)...Page 24
• Supported SFP+ Cables...Page 26
• Supported SFP+ 10G SR Optical Transceivers...Page 27
• Supported SFP+ 10G LR Optical Transceivers on page 28
• Supported SFP 1000BASE-T Transceivers...Page 28
• Supported 1G Optical Transceivers...Page 29
• Supported Speed and Mode...Page 29
• LED States...Page 31
• Solarflare Mezzanine Adapter: SFN5802K...Page 32
• Solarflare Mezzanine Adapters: SFN5812H and SFN5814H...Page 34
• Solarflare Mezzanine Adapter SFN6832F-C61...Page 35
• Solarflare Mezzanine Adapter SFN6832F-C62...Page 37
• Solarflare Precision Time Synchronization Adapters...Page 38
• Solarflare SFA6902F ApplicationOnload™ Engine...Page 38
CAUTION: Servers contain high voltage electrical components. Before removing the server cover,
disconnect the mains power supply to avoid the risk of electrocution.
Static electricity can damage computer components. Before handling computer components,
discharge static electricity from yourself by touching a metal surface, or wear a correctly fitted antistatic wrist band.
Issue 10
© Solarflare Communications 2013
19
Solarflare Server Adapter
User Guide
2.1 Solarflare Network Adapter Products
Solarflare Flareon™ adapters
- Solarflare Flareon Ultra SFN7322F Dual-Port 10GbE PCIe 3.0 Server I/O Adapter
- Solarflare Flareon Ultra SFN7122F Dual-Port 10GbE PCIe 3.0 Server I/O Adapter
- Solarflare Flareon SFN7022F Dual-Port 10GbE PCIe 3.0 Server I/O Adapter
Solarflare Onload adapters
- Solarflare SFN6322F Dual-Port 10GbE Precision Time Stamping Server Adapter
- Solarflare SFN6122F Dual-Port 10GbE SFP+ Server Adapter
- Solarflare SFA6902F Dual-Port 10GbE ApplicationOnload™ Engine
- Solarflare SFN5322F Dual-Port 10GbE Precision Time Stamping Server Adapter
- Solarflare SFN5122F Dual-Port 10G SFP+ Server Adapter
- Solarflare SFN5121T Dual-Port 10GBASE-T Server Adapter
- Solarflare SFN4112F Single-Port SFP+ Server Adapter
Solarflare Performant network adapters
- Solarflare SFN5161T Dual-Port 10GBASE-T Server Adapter
- Solarflare SFN5151T Single-Port10GBASE-T Server Adapter
- Solarflare SFN5162F Dual-Port 10G SFP+ Server Adapter
- Solarflare SFN5152F Single-Port 10G SFP+ Server Adapter
Solarflare Mezzanine adapters
- Solarflare SFN5802K Dual-Port 10G Ethernet Mezzanine Adapter for HP BladeSystem c-Class
- Solarflare SFN5812H Dual-Port 10G Ethernet Mezzanine Adapter for IBM BladeCenter
- Solarflare SFN5814H Quad-Port 10G Ethernet Mezzanine Adapter for IBM BladeCenter
- Solarflare SFN6832F-C61 Dual-Port 10GbE SFP+ Mezzanine Adapter for DELL PowerEdge
C6100 series servers.
- Solarflare SFN6832F-C62 Dual-Port 10GbE SFP+ Mezzanine Adapter for DELL PowerEdge
C6200 series servers.
- Solarflare SFN6822F Dual-Port 10GbE SFP+ FlexibleLOM Onload Server Adapter
Solarflare network adapters can be installed on Intel/AMD x86 based 32 bit or 64 bit servers. The
network adapter must be inserted into a PCIe x8 OR PCIe x 16 slot for maximum performance. Refer
to PCI Express Lane Configurations on page 241 for details.
Solarflare SFN5162F and SFN6122F adapters are supported on the IBM POWER architecture (PPC64)
running RHEL 6.4 on IBM System p servers.
Issue 10
© Solarflare Communications 2013
20
Solarflare Server Adapter
User Guide
2.2 Fitting a Full Height Bracket (optional)
Solarflare adapters are supplied with a low-profile bracket fitted to the adapter. A full height bracket
has also been supplied for PCIe slots that require this type of bracket.
To fit a full height bracket to the Solarflare adapter:
1
From the back of the adapter, remove the screws securing the bracket.
2
Slide the bracket away from the adapter.
3
Taking care not the overtighten the screws, attach the full height bracket to the adapter.
Issue 10
© Solarflare Communications 2013
21
Solarflare Server Adapter
User Guide
2.3 Inserting the Adapter in a PCI Express (PCIe) Slot
1
Shut down the server and unplug it from the mains. Remove the server cover to access the
PCIe slots in the server.
2
Locate an 8-lane or 16-lane PCIe slot (refer to the server manual if necessary) and insert the
Solarflare card.
3
Secure the adapter bracket in the slot.
4
Replace the cover and restart the server.
5
After restarting the server, the host operating system may prompt you to install drivers for the
new hardware. Click Cancel or abort the installation and refer to the relevant chapter in this
manual for how to install the Solarflare adapter drivers for your operating system.
Issue 10
© Solarflare Communications 2013
22
Solarflare Server Adapter
User Guide
2.4 Attaching a Cable (RJ-45)
The Solarflare SFN5121T 10GBASE-T Server Adapter connects to the Ethernet network using a
copper cable fitted with an RJ-45 connector (shown below).
RJ-45 Cable Specifications
Table 1 below lists the recommended cable specifications for various Ethernet port types.
Depending on the intended use, attach a suitable cable. For example, to achieve 10 Gb/s
performance, use a Category 6 cable. To achieve the desired performance, the adapter must be
connected to a compliant link partner, such as an IEEE 802.3an-compliant gigabit switch.
Table 1: RJ-45 Cable Specification
Issue 10
Port type
Connector
Media Type
Maximum
Distance
10GBASE-T
RJ-45
Category 6A
100m (328 ft.)
Category 6 unshielded twisted pairs (UTP)
55m (180 ft.)
Category 5E
55m (180 ft.)
1000BASE-T
RJ-45
Category 5E, 6, 6A UTP
100m (328 ft.)
100BASE-TX
RJ-45
Category 5E, 6, 6A UTP
100m (328 ft.)
© Solarflare Communications 2013
23
Solarflare Server Adapter
User Guide
2.5 Attaching a Cable (SFP+)
The Solarflare SFP+ Server Adapters can be connected to the network using either an SFP+ Direct
Attach cable or a fiber optic cable.
Attaching the SFP+ Direct Attach Cable:
1
Turn the cable so that the connector retention tab and gold fingers are on the same side as the
network adapter retention clip.
Push the cable connector straight in to the adapter socket until it clicks into place.
Removing the SFP+ Direct Attach Cable:
1
Pull straight back on the release ring to release the cable retention tab. Alternatively, you can
lift the retention clip on the adapter to free the cable if necessary.
2
Slide the cable free from the adapter socket.
Attaching a fiber optic cable:
WARNING
Do not look directly into the fiber transceiver or cables
as the laser beams can damage your eyesight.
1
Remove and save the fiber optic connector cover.
2
Insert a fiber optic cable into the ports on the network adapter bracket as shown. Most
connectors and ports are keyed for proper orientation. If the cable you are using is not keyed,
Issue 10
© Solarflare Communications 2013
24
Solarflare Server Adapter
User Guide
check to be sure the connector is oriented properly (transmit port connected to receive port
on the link partner, and vice versa).
Removing a fiber optic cable:
WARNING
Do not look directly into the fiber transceiver or cables
as the laser beams can damage your eyesight.
1
Remove the cable from the adapter bracket and replace the fiber optic connector cover.
2
Pull the plastic or wire tab to release the adapter bracket.
3
Hold the main body of the adapter bracket and remove it from the adapter.
Issue 10
© Solarflare Communications 2013
25
Solarflare Server Adapter
User Guide
2.6 Supported SFP+ Cables
Table 2 is a list of supported SFP+ cables that have been tested by Solarflare. Solarflare is not aware
of any issues preventing the use of other brands of SFP+ cables (of up to 5m in length) with Solarflare
network adapters. However, only cables in the table below have been fully verified and are therefore
supported.
Table 2: Supported SFP+ Direct Attach Cables
Issue 10
Manufacturer
Product Code
Cable Length
Arista
CAB-SFP-SFP-1M
1m
Arista
CAB-SFP-SFP-3M
3m
Cisco
SFP-H10GB-CU1M
1m
Cisco
SFP-H10GB-CU3M
3m
Cisco
SFP-H10GB-CU5M
5m
HP
J9283A/B Procurve
3m
Juniper
EX-SFP-10GE-DAC-1m
1m
Juniper
EX-SFP-10GE-DAC-3m
3m
Molex
74752-1101
1m
Molex
74752-2301
3m
Molex
74752-3501
5m
Molex
74752-9093
1m
37-0960-01 / 0K585N
Molex
74752-9094
3m
37-0961-01 / 0J564N
Molex
74752-9096
5m
37-0962-01 / 0H603N
Panduit
PSF1PXA1M
1m
Panduit
PSF1PXA3M
3m
Panduit
PSF1PXD5MBU
5m
Siemon
SFPP30-01
1m
Siemon
SFPP30-02
2m
Siemon
SFPP30-03
3m
Siemon
SFPP24-05
5m
© Solarflare Communications 2013
Notes
Not supported on SFN4112F
adapters – due to cable being
non-compliant with the SFP+
spec with respect to I2C.
26
Solarflare Server Adapter
User Guide
Table 2: Supported SFP+ Direct Attach Cables
Manufacturer
Product Code
Cable Length
Tyco
2032237-2 D
1m
Tyco
2032237-4
3m
Notes
The Solarflare SFA6902F adapter has been tested and certified with direct attach cables up to 3m in
length.
2.7 Supported SFP+ 10G SR Optical Transceivers
Table 3 is a list of supported SFP+10G SR optical transceivers that have been tested by Solarflare.
Solarflare is not aware of any issues preventing the use of other brands of 10G SR transceivers with
Solarflare network adapters. However, only transceivers in the table below have been fully verified
and are therefore supported.
Table 3: Supported SFP+ 10G Optical SR Transceivers
Issue 10
Manufacturer
Product Code
Notes
Avago
AFBR-703SDZ
10G
Avago
AFBR-703SDDZ
Dual speed 1G/10G optic. Not
supported on SFN4112F adapters.
Avago
AFBR-703SMZ
10G
Arista
SFP-10G-SR
10G
Finisar
FTLX8571D3BCL
10G
Finisar
FTLX8571D3BCV
Dual speed 1G/10G optic. Not
supported on SFN4112F adapters.
HP
456096-001
Also labelled as 455883-B21 and
455885-001
Intel
AFBR-703SDZ
10G
JDSU
PLRXPL-SC-S43-22-N
10G
Juniper
AFBR-700SDZ-JU1
10G
MergeOptics
TRX10GVP2010
10G
Solarflare
SFM-10G-SR
10G
© Solarflare Communications 2013
27
Solarflare Server Adapter
User Guide
2.8 Supported SFP+ 10G LR Optical Transceivers
Table 4 is a list of supported SFP+10G LR optical transceivers that have been tested by Solarflare.
Solarflare is not aware of any issues preventing the use of other brands of 10G LR transceivers with
Solarflare network adapters. However, only transceivers in the table below have been fully verified
and are therefore supported.
Table 4: Supported SFP+ 10G LR Optical Transceivers
Manufacturer
Product Code
Notes
Avago
AFCT-701SDZ
10G single mode fiber
Finisar
FTLX1471D3BCL
10G single mode fiber
2.9 Supported SFP 1000BASE-T Transceivers
Table 5 is a list of supported SFP 1000BASE-T transceivers that have been tested by Solarflare.
Solarflare is not aware of any issues preventing the use of other brands of 1000BASE-T transceivers
with the Solarflare network adapters. However, only transceivers in the table below have been fully
verified and are therefore supported.
NOTE: 1000BASE-T transceivers are not supported on the SFN4112F SFP+ server network adapter.
Table 5: Supported SFP 1000BASE-T Transceivers
Issue 10
Manufacturer
Product Code
Arista
SFP-1G-BT
Avago
ABCU-5710RZ
Cisco
30-1410-03
Dell
FCMJ-8521-3-(DL)
Finisar
FCLF-8521-3
Finisar
FCMJ-8521-3
HP
453156-001
453154-B21
3COM
3CSFP93
© Solarflare Communications 2013
28
Solarflare Server Adapter
User Guide
2.10 Supported 1G Optical Transceivers
Table 6 is a list of supported 1G transceivers that have been tested by Solarflare. Solarflare is not
aware of any issues preventing the use of other brands of 1G transceivers with Solarflare network
adapters. However, only transceivers in the table below have been fully verified and are therefore
supported.
NOTE: 1G optical transceivers are not supported on the SFN4112F SFP+ server network adapter.
Table 6: Supported 1G Transceivers
Manufacturer
Product Code
Type
Avago
AFBR-5710PZ
1000Base-SX
Cisco
GLC-LH-SM
1000Base-LX/LH
Finisar
FTLF8519P2BCL
1000Base-SX
Finisar
FTLF8519P3BNL
1000Base-SX
Finisar
FTLF1318P2BCL
1000Base-LX
Finisar
FTLF1318P3BTL
1000Base-LX
HP
453153-001
453151-B21
1000Base-SX
2.11 Supported Speed and Mode
Solarflare network adapters support either SFP/SFP+ or Base-T standards.
On Base-T adapters three speeds are supported 100Mbps, 1Gbps and 10Gbps. The adapters use
auto negotiation to automatically select the highest speed supported in common with the link
partner.
On SFP+ adapters the currently inserted SFP module (transceiver) determines the supported speeds,
typically SFP modules only support a single speed. Some Solarflare SFP+ adapters support dual
speed optical modules that can operate at either 1Gbps or 10Gbps. However, these modules do not
auto-negotiate link speed and operate at the maximum (10G) link speed unless explicitly configured
to operate at a lower speed (1G).
Issue 10
© Solarflare Communications 2013
29
Solarflare Server Adapter
User Guide
The tables below summarizes the speeds supported by Solarflare network adapters.
Table 7: SFN4112F SFP+ Adapter
Supported Modes
Auto neg speed
Speed
SFP+ direct attach cable
No
10G
SFP+ optical module (10G)
No
10G
Comment
Table 8: SFN5xxx,SFN6xxx and SFN7xxx SFP+ Adapters
Supported Modes
Auto neg speed
Speed
Comment
SFP+ direct attach cable
No
10G
SFP+ optical module (10G)
No
10G
SFP optical module (1G)
No
1G
SFP+ optical module (10G/1G)
No
10G or 1G
Dual speed modules run at
the maximum speed (10G)
unless explicitly configured to
the lower speed (1G)
SFP 1000BASE-T module
No
1G
These modules support only
1G and will not link up at
100Mbps
Table 9: SFN5121T, SFN5151T, SFN5161T 10GBASE-T Adapters
Issue 10
Supported Modes
Auto neg speed
Speed
Comment
100Base-T
Yes
100Mbps
1000Base-TX
Yes
1Gbps
10GBase-T
Yes
10Gbps
Typically the interface is set
to auto negotiation speed
and automatically selects the
highest speed supported in
common with it’s link partner.
If the link partner is set to
100Mbps, with no autoneg,
the adapter will use “parallel
detection” to detect and
select 100Mbps speed. If
needed any of the three
speeds can be explicitly
configured
© Solarflare Communications 2013
30
Solarflare Server Adapter
User Guide
100Base-T in a Solarflare adapter back-to-back (no intervening switch) configuration will not work
and is not supported.
2.12 LED States
There are two LEDs on the Solarflare network adapter transceiver module. LED states are as follows:
Table 10: LED States
Adapter Type
LED Description
State
SFP/SFP+
Link
Green (solid) at all speeds
Activity
Flashing green when network traffic is present
LEDs are OFF when there is no link present
BASE-T
Speed
Green (solid) 10Gbps
Yellow (solid) 100/1000Mbps
Activity
Flashing green when network traffic is present
LEDs are OFF when there is no link present
Issue 10
© Solarflare Communications 2013
31
Solarflare Server Adapter
User Guide
2.13 Solarflare Mezzanine Adapter: SFN5802K
The Solarflare SFN5802K is a Dual-Port 10G Ethernet Mezzanine Adapter for the HP BladeSystem cClass.
The HP BladeSystem blade supports up to two Solarflare mezzanine adapters.
Inserting the Mezzanine Adapter into the BladeSystem Server.
1
The blade should be extracted from the server in order to install the mezzanine adapter.
2
Remove the blade top cover and identify the screw posts towards the rear of the blade
(Figure 1) where the adapter will be secured (two screw posts per adapter - refer to the
BladeSystem manual if necessary).
Figure 1: Locate the Mezzanine Adapter screw posts
3
It may be necessary to remove the internal battery tray if fitted. Align the mezzanine port
connector with the backplane connector block and screw posts. Press home gently ensuring
that the adapter is firmly and correctly seated in the connector block.
4
Hand tighten the adapter retaining screws.
5
Replace the internal battery tray and replace the blade top cover.
Issue 10
© Solarflare Communications 2013
32
Solarflare Server Adapter
User Guide
With a mezzanine adapter installed into the lower connector block, a second adapter can be
installed into the remaining connector block (see Figure 2)
Figure 2: Installing a second mezzanine adapter
CAUTION: It is important that the spring loaded screws are not over tightened to avoid damage to
the adapter.
Issue 10
© Solarflare Communications 2013
33
Solarflare Server Adapter
User Guide
2.14 Solarflare Mezzanine Adapters: SFN5812H and SFN5814H
The Solarflare SFN5812H Dual-Port and SFN5814H Quad-Port are 10G Ethernet Mezzanine Adapters
for the IBM BladeCenter.
Solarflare mezzanine adapters are supported on the IBM BladeCenter E, H and S chassis, HS22,
HS22V and HX5 servers. The IBM BladeCenter blade supports a single Solarflare mezzanine adapter.
1
The blade should be extracted from the BladeCenter in order to install the mezzanine adapter.
2
Remove the blade top cover and locate the two retaining posts towards the rear of the blade (Figure 3). Refer to the BladeCenter manual if necessary.
Figure 3: Installing the Mezzanine Adapter
3
Issue 10
Hinge the adapter under the retaining posts, as illustrated, and align the mezzanine port
connector with the backplane connector block.
© Solarflare Communications 2013
34
Solarflare Server Adapter
User Guide
4
Lower the adapter, taking care to align the side positioning/retaining posts with the recesses in
the adapter. See Figure 4.
Figure 4: In position mezzanine adapter
5
Press the port connector gently into the connector block ensuring that the adapter is firmly
and correctly seated in the connector block.
6
Replace the blade top cover.
7
When removing the adapter raise the release handle (shown on Figure 4) to ease the adapter
upwards until it can be freed from the connector block.
2.15 Solarflare Mezzanine Adapter SFN6832F-C61
The Solarflare SFN6832F-C61 is a Dual-Port SFP+ are 10GbE Mezzanine Adapters for the DELL
PowerEdge C6100 series rack server. Each DELL PowerEdge node supports a single Solarflare
mezzanine adapter.
1
Issue 10
The node should be extracted from the rack server in order to install the mezzanine adapter.
Refer to the PowerEdge rack server manual if necessary.
© Solarflare Communications 2013
35
Solarflare Server Adapter
User Guide
Figure 5: SFN6832F-C61 - Installing into the rack server node
2
Secure the side retaining bracket as shown in Figure 5 (top diagram)
3
Fit riser PCB card into the slot as shown in Figure 5 (top diagram). Note that the riser card only
fits one way.
4
Offer the adapter to the node and ensure it lies underneath the chassis cover.
5
Lower the adapter into position making sure to connect the adapter slot with the to of the PCB
riser card.
6
Secure the adapter using the supplied screws at the positions shown in the diagram.
Issue 10
© Solarflare Communications 2013
36
Solarflare Server Adapter
User Guide
2.16 Solarflare Mezzanine Adapter SFN6832F-C62
The Solarflare SFN6832F-C61 is a Dual-Port SFP+ are 10GbE Mezzanine Adapters for the DELL
PowerEdge C6200 series rack server. Each DELL PowerEdge node supports a single Solarflare
mezzanine adapter.
1
The node should be extracted from the rack server in order to install the mezzanine adapter.
Refer to the PowerEdge rack server manual if necessary.
Figure 6: SFN6832F-C62 - Installing into the rack server node
2
Fit the PCB riser card to the underside connector on the adapter.
3
Offer the adapter to the rack server node ensuring it lies underneath the chassis cover.
4
Lower to adapter to connect the riser PCB card into the slot in the node.
5
Secure the adapter with the supplied screws at the points shown in the diagram.
Issue 10
© Solarflare Communications 2013
37
Solarflare Server Adapter
User Guide
2.17 Solarflare Precision Time Synchronization Adapters
The Solarflare SFN7122F1, SFN7322F, SFN6322F and SFN5322F adapters are dual-port SFP+ 10GbE
adapters that can generate hardware timestamps for PTP packets in support of a network precision
time protocol deployment compliant with the IEEE 1588-2008 specification.
Customers requiring configuration instructions for these adapters and Solarflare PTP in a PTP
deployment should refer to the Solarflare Enhanced PTP User Guide SF-109110-CD.
1. Requires an AppFlex™ license - refer to Solarflare AppFlex™ Technology Licensing. on page 13.
2.18 Solarflare SFA6902F ApplicationOnload™ Engine
The ApplicationOnload™ Engine (AOE) SFA6902F is a full length PCIe form factor adapter that
combines an ultra-low latency adapter with a tightly coupled ’bump-in-the-wire’ FPGA.
For details of installation and configuring applications that run on the AOE refer to the Solarflare AOE
User’s Guide (SF-108389-CD). For details on developing custom applications to run on the FPGA refer
to the AOE Firmware Development Kit User Guide (SF-108390-CD).
Issue 10
© Solarflare Communications 2013
38
Solarflare Server Adapter
User Guide
Chapter 3: Solarflare Adapters on Linux
This chapter covers the following topics on the Linux® platform:
• System Requirements...Page 39
• Linux Platform Feature Set...Page 40
• Solarflare RPMs...Page 41
• Installing Solarflare Drivers and Utilities on Linux...Page 43
• Red Hat Enterprise Linux Distributions...Page 43
• SUSE Linux Enterprise Server Distributions...Page 44
• Unattended Installations...Page 45
• Unattended Installation - Red Hat Enterprise Linux...Page 47
• Unattended Installation - SUSE Linux Enterprise Server...Page 48
• Hardware Timestamps...Page 49
• Configuring the Solarflare Adapter...Page 49
• Configuring Receive/Transmit Ring Buffer Size...Page 50
• Setting Up VLANs...Page 51
• Setting Up Teams...Page 52
• Running Adapter Diagnostics...Page 53
• Running Cable Diagnostics...Page 54
• Linux Utilities RPM...Page 55
• Configuring the Boot ROM with sfboot...Page 56
• Upgrading Adapter Firmware with Sfupdate...Page 70
• License Install with sfkey...Page 75
• Performance Tuning on Linux...Page 77
• Module Parameters...Page 97
• Linux ethtool Statistics...Page 99
3.1 System Requirements
Refer to Software Driver Support on page 13 for supported Linux Distributions.
NOTE: SUSE Linux Enterprise Server 11 includes a version of the Solarflare network adapter Driver.
This driver does not support the SFN512x family of adapters. To update the supplied driver, see
SUSE Linux Enterprise Server Distributions on page 44
Issue 10
© Solarflare Communications 2013
39
Solarflare Server Adapter
User Guide
NOTE: Red Hat Enterprise Linux versions 5.5 and 6.0 include a version of the SFN4112F Solarflare
adapter driver. This driver does not support the SFN512x family of adapters. Red Hat Enterprise
Linux 5.6 and 6.1 includes a version of the Solarflare network driver for the SFN512x family of
adapters. To update the supplied driver, see Installing Solarflare Drivers and Utilities on Linux on
page 43
3.2 Linux Platform Feature Set
Table 11 lists the features supported by Solarflare adapters on Red Hat and SUSE Linux distributions.
Table 11: Linux Feature Set
Fault diagnostics
Support for comprehensive adapter and cable fault diagnostics
and system reports.
• See Running Adapter Diagnostics on page 53
Firmware updates
Support for Boot ROM, Phy transceiver and adapter firmware
upgrades.
• See Upgrading Adapter Firmware with Sfupdate on page 70
Hardware Timestamps
Solarflare Flareon SFN7122F1 and SFN7322F adapters support
the hardware timestamping of all received packets - including
PTP packets.
The Linux kernel must support the SO_TIMESTAMPING socket
option (2.6.30+) to allow the driver to support hardware packet
timestamping. Therefore hardware packet timestamping is not
available in RHEL 5
1. SFN7122F requires an AppFlex license - for details refer to Solarflare
AppFlex™ Technology Licensing. on page 13.
Jumbo frames
Support for MTUs (Maximum Transmission Units) from
1500 bytes to 9216 bytes.
• See Configuring Jumbo Frames on page 51
PXE and iSCSI booting
Support for diskless booting to a target operating system via
PXE or iSCSI boot.
• See Configuring the Boot ROM with sfboot on page 56
• See Solarflare Boot ROM Agent on page 360
PXE or iSCSI boot are not supported for Solarflare adapters on
IBM System p servers.
Issue 10
© Solarflare Communications 2013
40
Solarflare Server Adapter
User Guide
Table 11: Linux Feature Set
Receive Side Scaling (RSS)
Support for RSS multi-core load distribution technology.
• See Receive Side Scaling (RSS) on page 83
SR-IOV
Support for XenServer6 PCIe Single Root-IO Virtualization and
Linux KVM SR-IOV.
• See SR-IOV Virtualization for XenServer on page 340
• See SR-IOV Virtualization Using KVM on page 323
SR-IOV is not supported for Solarflare adapters on IBM System
p servers.
Standby and Power
Management
Solarflare adapters support Wake On LAN on Linux. These
settings are only available if the adapter has auxiliary power
supplied by a separate cable.
Task offloads
Support for TCP Segmentation Offload (TSO), Large Receive
Offload (LRO), and TCP/UDP/IP checksum offload for improved
adapter performance and reduced CPU processing
requirements.
• See Configuring Task Offloading on page 50
Teaming
Improve server reliability and bandwidth by combining physical
ports, from one or more Solarflare adapters, into a team,
having a single MAC address and which function as a single port
providing redundancy against a single point of failure.
• See Setting Up Teams on page 52
Virtual LANs (VLANs)
Support for multiple VLANs per adapter.
• See Setting Up VLANs on page 51
3.3 Solarflare RPMs
Solarflare supply RPM packages in the following formats:
• DKMS
• Source RPM
Where possible, it is recommended that you use the DKMS RPM if you have DKMS installed.
Issue 10
© Solarflare Communications 2013
41
Solarflare Server Adapter
User Guide
DKMS RPM
Dynamic Kernel Module Support (DKMS) is a framework where device driver source can reside
outside the kernel source tree. It supports an easy method to rebuild modules when kernels are
upgraded.
Execute the command dkms --version to determine whether DKMS is installed.
To install the Solarflare driver DKMS package execute the following command:
rpm -i sfc-dkms-.noarch.rpm
Building the Source RPM
These instructions may be used to build a source RPM package for use with Linux distributions or
kernel versions where DKMS or KMP packages are not suitable.
NOTE: RPMs can be installed for multiple kernel versions.
1
First, the kernel headers for the running kernel must be installed at /lib/modules/
/build. On Red Hat systems, install the appropriate kernel-smpdevel or kernel-devel package. On SUSE systems install the kernel-source package.
2
To build a source RPM for the running kernel version from the source RPM, enter the following
at the command-line:
rpmbuild --rebuild
Where package_name is the full path to the source RPM (see the note below).
3
To build for a different kernel to the running system, enter the following command:
rpmbuild --define 'kernel ' --rebuild
4
Install the resulting RPM binary package, as described in Installing Solarflare Drivers and
Utilities on Linux.
NOTE: The location of the generated RPM is dependent on the distribution and often the version
of the distribution and the RPM build tools.
The RPM build process should print out the location of the RPM towards the end of the build
process, but it can be hard to find amongst the other output.
Typically the RPM will be placed in /usr/src//RPMS//, where is
distribution specific. Possible folders include Red Hat, packages or extra. The RPM file will be
named using the same convention as the Solarflare provided pre-built binary RPMs.
The command: find /usr/src -name "*sfc*.rpm” will list the locations of all Solarflare
RPMs.
Issue 10
© Solarflare Communications 2013
42
Solarflare Server Adapter
User Guide
3.4 Installing Solarflare Drivers and Utilities on Linux
• Red Hat Enterprise Linux Distributions...Page 43
• SUSE Linux Enterprise Server Distributions...Page 44
• Building the Source RPM...Page 42
Linux drivers for Solarflare are available in DKMS and source RPM packages. The source RPM can be
used to build binary RPMs for a wide selection of distributions and kernel variants. This section
details how to install the resultant binary RPM.
Solarflare recommend using DKMS RPMs if the DKMS framework is available. See DKMS RPM on
page 42 for more details.
NOTE: The Solarflare adapter should be physically installed in the host computer before installing
the driver. The user must have root permissions to install the adapter drivers.
3.5 Red Hat Enterprise Linux Distributions
These instructions cover installation and configuration of the Solarflare network adapter drivers on
Red Hat Enterprise Linux Server. Refer to Software Driver Support on page 13 for details of
supported Linux distributions.
Refer to Building the Source RPM on page 42 for directions on creating the binary RPM.
1
Install the RPMs:
[root@myhost1]# rpm -ivh kernel-module-sfc-RHEL6-2.6.32279.el6.x86_64-3.3.0.6262-1.x86_64.rpm
2
There are various tools that can be used for configuring the Solarflare Server Adapter:
a) The NetworkManager service and associated GUI tools. For more information about his
refer to https://wiki.gnome.org/NetworkManager.
3
b) Solarflare recommend using the Network Administration Tool (NEAT) to configure the new
network interface. NEAT is a GUI based application and therefore requires an X server to run.
c) Alternatively the command line program Kudzu can be used. However, you may find when
kudzu is run that you are NOT presented with an option to configure the new network
interface. If this occurs, carefully clear details of the Solarflare Server Adapter from the
hardware database by removing all entries with “vendor id: 1924” in the /etc/
sysconfig/hwconf file. Running kudzu again should now provide an option to configure
the newly added network interface.
Apply the new network settings:
a) NEAT provides an option to Activate the new interface. The new network interface can
then be used immediately (there is no need to reboot or restart the network service).
b) If you are not using NEAT you will need to reboot, or alternatively restart the networking
service, by typing the following before the new Solarflare interface can be used:
[root@myhost1]# service network restart
Issue 10
© Solarflare Communications 2013
43
Solarflare Server Adapter
User Guide
3.6 SUSE Linux Enterprise Server Distributions
These instructions cover installation and configuration of the Solarflare Network Adapter drivers on
SUSE Linux Enterprise Server. Refer to Software Driver Support...Page 13 for details of supported
distributions.
Refer to Building the Source RPM on page 42 for directions on creating the binary RPM.
1
The Solarflare drivers are currently classified as 'unsupported' by SUSE Enterprise Linux 10
(SLES10). To allow unsupported drivers to load in SLES10, edit the following file:
/etc/sysconfig/hardware/config
find the line:
LOAD_UNSUPPORTED_MODULES_AUTOMATICALLY=no
and change no to yes.
For SLES 11, edit the last line in /etc/modprobe.d/unsupported-modules to:
allow_unsupported_modules 1
2
Install the RPMs:
[root@myhost1]# rpm -ivh kernel-module-sfc-2.6.5-7.244-smp-2.1.01110.sf.1.SLES9.i586.rpm
3
Run YaST to configure the Solarflare Network Adapter. When you select the Ethernet
Controller, the Configuration Name will take one of the following forms:
a) eth-bus-pci-dddd:dd:dd.N where N is either 0 or 1.
b) eth-id-00:0F:53:XX:XX:XX
Once configured, the Configuration Name for the correct Ethernet Controller will change to
the second form, and an ethX interface will appear on the host. If the incorrect Ethernet
Controller is chosen and configured, then the Configuration Name will remain as eth-buspci-dddd:dd:dd.1 after configuration by YaST, and an ethX interface will not appear on
the system. In this case, you should remove the configuration for this Ethernet Controller, and
configure the other Ethernet Controller of the pair.
Issue 10
© Solarflare Communications 2013
44
Solarflare Server Adapter
User Guide
3.7 Unattended Installations
Building Drivers and RPMs for Unattended Installation
Linux unattended installation requires building two drivers:
• A minimal installation Solarflare driver that only provides networking support. This driver is
used for network access during the installation process.
• An RPM that includes full driver support. This RPM is used to install drivers in the resultant Linux
installation.
Figure 7: Unattended Installation RPM
Figure 7 shows how the unattended installation process works.
1
Build a minimal Solarflare driver needed for use in the installation kernel (Kernel A in the
diagram above). This is achieved by defining “sfc_minimal” to rpmbuild. This macro disables
hardware monitoring, MTD support (used for access to the adapters flash), I2C and debugfs.
This results in a driver with no dependencies on other modules and allows networking support
from the driver during installation.
# as normal user
$ mkdir -p /tmp/rpm/BUILD
$ rpm -i sfc--1.src.rpm
$ rpmbuild -bc -D 'sfc_minimal=1' -D 'kernel=' \
/tmp/rpm/SPECS/sfc.spec
2
The Solarflare minimal driver sfc.ko can be found in /tmp/rpm/BUILD/sfc-/
linux_net/sfc.ko. Integrate this minimal driver into your installer kernel, either by
creating a driver disk incorporating this minimal driver or by integrating this minimal driver
into initrd.
3
Build a full binary RPM for your Target kernel and integrate this RPM into your Target (Kernel
B).
Issue 10
© Solarflare Communications 2013
45
Solarflare Server Adapter
User Guide
Driver Disks for Unattended Installations
Solarflare are preparing binary driver disks to help avoid the need to build the minimal drivers
required in unattended installations. Please contact Solarflare support to obtain these driver disks
Table 12 shows the various stages of an unattended installation process:
Table 12: Installation Stages
Issue 10
In Control
Stages of Boot
Setup needed
BIOS
PXE code on the adapter
runs.
Adapter must be in PXE boot mode.
See PXE Support on page 361.
SF Boot ROM (PXE)
DHCP request from PXE (SF
Boot ROM).
DHCP server filename and nextserver options.
SF Boot ROM (PXE)
TFTP request for filename to
next-server, e.g. pxelinux.0
TFTP server.
pxelinux
TFTP retrieval of pxelinux
configuration.
pxelinux configuration on TFTP
server.
pxelinux
TFTP menu retrieval of Linux
kernel image initrd.
pxelinux configuration
Linux kernel/installer
Installer retrieves kickstart
configuration, e.g. via HTTP.
Kickstart/AutoYaST configuration.
Target Linux kernel
kernel reconfigures network
adapters.
DHCP server.
Kernel, kernel command, initrd
© Solarflare Communications 2013
46
Solarflare Server Adapter
User Guide
3.8 Unattended Installation - Red Hat Enterprise Linux
Documentation for preparing for a Red Hat Enterprise Linux network installation can be found at:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Installation_Guide/s1begininstall-perform-nfs-x86.html
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/
index.html
The prerequisites for a Network Kickstart installation are:
• Red Hat Enterprise Linux installation media.
• A Web server and/or FTP Server for delivery of the RPMs that are to be installed.
• A DHCP server for IP address assignments and to launch PXE Boot.
• A TFTP server for download of PXE Boot components to the machines being kickstarted.
• The BIOS on the computers to be Kickstarted must be configured to allow a network boot.
• A Boot CD-ROM or flash memory that contains the kickstart file or a network location where the
kickstart file can be accessed.
• A Solarflare driver disk.
Unattended Red Hat Enterprise Linux installations are configured with Kickstart. The documentation
for Kickstart can be found at:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Installation_Guide/chredhat-config-kickstart.html
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/chkickstart2.html
To install Red Hat Enterprise you need the following:
1
A modified initrd.img file with amended modules.alias and modules.dep which
incorporates the Solarflare minimal driver for the installation kernel.
To modules.alias, add the following entries:
2
alias:
pci:v00001924d00000813sv*sd*bc*sc*i*
alias:
pci:v00001924d00000803sv*sd*bc*sc*i*
alias:
pci:v00001924d00000710sv*sd*bc*sc*i*
alias:
pci:v00001924d00000703sv*sd*bc*sc*i*
Identify the driver dependencies using the modinfo command:
modinfo ./sfc.ko | grep depends
depends:
i2c-core,mii,hwmon,hwmon-vid,i2c-algo-bit mtdcore mtdpart
All modules listed as depends must be present in the initrd file image. In addition the user
should be aware of further dependencies which can be resolved by adding the following lines
to the modules.dep file:
Issue 10
© Solarflare Communications 2013
47
Solarflare Server Adapter
User Guide
sfc: i2c-core mii
i2c-algo-bit:
mtdpart:
hwmon
hwmon-vid
i2c-algo-bit
mtdcore
mtdpart *
i2c-core
mtdcore
*For Red Hat Enterprise Linux from version 5.5 add mdio to this line.
3
A configured kickstart file with the Solarflare Driver RPM manually added to the %Post
section. For example:
%post
/bin/mount -o ro :/ /mnt
/bin/rpm -Uvh /mnt/
/bin/umount /mnt
3.9 Unattended Installation - SUSE Linux Enterprise Server
Unattended SUSE Linux Enterprise Server installations are configured with AutoYaST. The
documentation for AutoYaST can be found at:
http://www.suse.com/~ug/autoyast_doc/index.html
The prerequisites for a Network AutoYaST installation are:
• SUSE Linux Enterprise installation media.
• A DHCP server for IP address assignments and to launch PXE Boot.
• A NFS or FTP server to provide the installation source.
• A TFTP server for the download of the kernel boot images needed to PXE Boot.
• A boot server on the same Ethernet segment.
• An install server with the SUSE Linux Enterprise Server OS.
• An AutoYaST configuration server that defines rules and profiles.
• A configured AutoYast Profile (control file).
Further Reading
• SUSE Linux Enterprise Server remote installation:
http://www.novell.com/documentation/sles10/sles_admin/?page=/documentation/sles10/
sles_admin/data/cha_deployment_remoteinst.html
• SUSE install with PXE Boot:
http://en.opensuse.org/SuSE_install_with_PXE_boot
Issue 10
© Solarflare Communications 2013
48
Solarflare Server Adapter
User Guide
3.10 Hardware Timestamps
The Solarflare Flareon SFN7000 series adapters can support hardware timestamping for all received
network packets.
The Linux kernel must support the SO_TIMESTAMPING socket option (2.6.30+) therefore hardware
packet timestamping is not supported on RHEL 5.
For more information about using the kernel timestamping API, users should refer to the Linux
documentation: http://lxr.linux.no/linux/Documentation/networking/timestamping.txt
3.11 Configuring the Solarflare Adapter
Ethtool is a standard Linux tool that you can use to query and change Ethernet adapter settings,
including those for Solarflare adapters. Ethtool can be downloaded from http://sourceforge.net/
projects/gkernel/files/ethtool/.
The general command for ethtool is as follows:
ethtool <-option>
Where X is the identifier of the interface. Note that root access will be required to configure adapter
settings. Refer to the Linux online manual (man ethtool) for details of the options that are
available for ethtool.
Configuring Speed and Modes
Solarflare adapters by default automatically negotiate the connection speed to the maximum
supported by the link partner. On the 10GBASE-T adapters “auto” instructs the adapter to negotiate
the highest speed supported in common with it’s link partner. On SFP+ adapters, “auto” instructs the
adapter to use the highest link speed supported by the inserted SFP+ module. On 10GBASE-T and
SFP+ adapters, any other value specified will fix the link at that speed, regardless of the capabilities
of the link partner, which may result in an inability to establish the link. Dual speed SFP+ modules
operate at their maximum (10G) link speed unless explicitly configured to operate at a lower speed
(1G).
The following commands demonstrate ethtool to configure the network adapter Ethernet settings.
Identify interface configuration settings:
ethtool ethX
Set link speed:
ethtool -s ethX speed 1000|100
To return the connection speed to the default auto-negotiate, enter:
ethtool -s autoneg on
Configure auto negotiation:
ethtool -s ethX autoneg [on|off]
Set auto negotiation advertised speed 1G:
Issue 10
© Solarflare Communications 2013
49
Solarflare Server Adapter
User Guide
ethtool -s ethX advertise 0x20
Set autonegotiation advertised speed 10G:
ethtool -s ethX advertise 0x1000
Set autonegotiation advertised speeds 1G and 10G:
ethtool -s ethX advertise 0x1020
Identify interface auto negotiation pause frame setting:
ethtool -a ethX
Configure auto negotiation of pause frames:
ethtool -A ethX autoneg on [rx on|off] [tx on|off]
Configuring Task Offloading
Solarflare adapters support transmit (Tx) and receive (Rx) checksum offload, as well as TCP
segmentation offload. To ensure maximum performance from the adapter, all task offloads should
be enabled, which is the default setting on the adapter. For more information, see Performance
Tuning on Linux on page 77.
To change offload settings for Tx and Rx, use the ethtool command:
ethtool --offload [rx on|off] [tx on|off]
Configuring Receive/Transmit Ring Buffer Size
By default receive and transmit ring buffers on the Solarflare adapter support 1024 descriptors. The
user can identify and reconfigure ring buffer sizes using the ethtool command.
To identify the current ring size:
ethtool -g ethX
To set the new transmit or receive ring size to value N
ethtool -G ethX [rx N| tx N]
The ring buffer size must be a value between 128 and 4096. Buffer size can also be set directly in the
modprobe.conf file or add the options line to a file under the /etc/modprobe.d directory e.g.
options sfc
rx_ring=4096
Using the modprobe method sets the value for all Solarflare interfaces. Then reload the driver for
the option to become effective:
modprobe -r sfc
modprobe sfc
Issue 10
© Solarflare Communications 2013
50
Solarflare Server Adapter
User Guide
Configuring Jumbo Frames
Solarflare adapters support frame sizes from 1500 bytes to 9216 bytes. For example, to set a new
frame size (MTU) of 9000 bytes, enter the following command:
ifconfig mtu 9000
To make the changes permanent, edit the network configuration file for ; for example,
/etc/sysconfig/network-scripts/ifcfg-eth1 and append the following configuration
directive, which specifies the size of the frame in bytes:
MTU=9000
Standby and Power Management
Solarflare adapters support Wake on LAN and Wake on Magic Packet setting on Linux. You need to
ensure that Wake on LAN has been enabled on the BIOS correctly and your adapter has auxiliary
power via a separate cable before configuring Wake on LAN features.
In SUSE Linux Enterprise Server, you can use the YaST WOL module to configure Wake on LAN or you
can use the ethtool wol g setting.
In Red Hat Enterprise Linux you can use the ethtool wol g setting.
3.12 Setting Up VLANs
VLANs offer a method of dividing one physical network into multiple broadcast domains. In
enterprise networks, these broadcast domains usually match with IP subnet boundaries, so that
each subnet has its own VLAN. The advantages of VLANs include:
• Performance
• Ease of management
• Security
• Trunks
• You don't have to configure any hardware device, when physically moving your server to
another location.
To set up VLANs, consult the following documentation:
• To configure VLANs on SUSE Linux Enterprise Server, see:
http://www.novell.com/support/viewContent.do?externalId=3864609
• To configure tagged VLAN traffic only on Red Hat Enterprise Linux, see:
http://kbase.redhat.com/faq/docs/DOC-8062
• To configure mixed VLAN tagged and untagged traffic on Red Hat Enterprise Linux, see:
http://kbase.redhat.com/faq/docs/DOC-8064
Issue 10
© Solarflare Communications 2013
51
Solarflare Server Adapter
User Guide
3.13 Setting Up Teams
Teaming network adapters (network bonding) allows a number of physical adapters to act as one,
virtual adapter. Teaming network interfaces, from the same adapter or from multiple adapters,
creates a single virtual interface with a single MAC address.
The virtual adapter or virtual interface can assist in load balancing and providing failover in the event
of physical adapter or port failure.
Teaming configuration support provided by the Linux bonding driver includes:
• 802.3ad Dynamic link aggregation
• Static link aggregation
• Fault Tolerant
To set up an adapter team, consult the following documentation:
General:
http://www.kernel.org/doc/Documentation/networking/bonding.txt
RHEL 5:
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5.4/html/Deployment_Guide/s2modules-bonding.html
RHEL6:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s2networkscripts-interfaces-chan.html
SLES:
http://www.novell.com/documentation/sles11/book_sle_admin/data/
sec_basicnet_yast.html#sec_basicnet_yast_netcard_man
Issue 10
© Solarflare Communications 2013
52
Solarflare Server Adapter
User Guide
3.14 Running Adapter Diagnostics
You can use ethtool to run adapter diagnostic tests. Tests can be run offline (default) or online.
Offline runs the full set of tests, which can interrupt normal operation during testing. Online
performs a limited set of tests without affecting normal adapter operation.
As root user, enter the following command:
ethtool --test ethX offline|online
The tests run by the command are as follows:
Table 13: Adapter Diagnostic Tests
Diagnostic Test
Purpose
core.nvram
Verifies the flash memory ‘board configuration’ area by
parsing and examining checksums.
core.registers
Verifies the adapter registers by attempting to modify the
writable bits in a selection of registers.
core.interrupt
Examines the available hardware interrupts by forcing the
controller to generate an interrupt and verifying that the
interrupt has been processed by the network driver.
tx/rx.loopback
Verifies that the network driver is able to pass packets to
and from the network adapter using the MAC and Phy
loopback layers.
core.memory
Verifies SRAM memory by writing various data patterns
(incrementing bytes, all bit on and off, alternating bits on
and off) to each memory location, reading back the data
and comparing it to the written value.
core.mdio
Verifies the MII registers by reading from PHY ID registers
and checking the data is valid (not all zeros or all ones).
Verifies the MMD response bits by checking each of the
MMDs in the Phy is present and responding.
chanX eventq.poll
Verifies the adapter’s event handling capabilities by
posting a software event on each event queue created by
the driver and checking it is delivered correctly.
The driver utilizes multiple event queues to spread the
load over multiple CPU cores (RSS).
phy.bist
Issue 10
Examines the PHY by initializing it and causing any
available built-in self tests to run.
© Solarflare Communications 2013
53
Solarflare Server Adapter
User Guide
3.15 Running Cable Diagnostics
Cable diagnostic data can be gathered from the Solarflare 10GBASE-T adapters physical interface
using the ethtool -t command which runs a comprehensive set of diagnostic tests on the
controller, PHY, and attached cables. To run the cable tests enter the following command:
ethtool -t ethX [online | offline]
Online tests are non-intrusive and will not disturb live traffic.
The following is an extract from the output of the ethtool diagnostic offline tests:
phy
phy
phy
phy
phy
phy
phy
phy
cable.pairA.length
cable.pairB.length
cable.pairC.length
cable.pairD.length
cable.pairA.status
cable.pairB.status
cable.pairC.status
cable.pairD.status
9
9
9
9
1
1
1
1
Cable length is the estimated length in metres. A length value of 65535 indicates length not
estimated due to pair busy or cable diagnostic routine not completed successfully.
The cable status can be one of the following values:
0 - invalid, or cable diagnostic routine did not complete successfully
1 - pair ok, no fault detected
2 - pair open or Rt > 115 ohms
3 - intra pair short or Rt < 85 ohms
4 - inter pair short or Rt < 85 ohms
9 - pair busy or link partner forces 100Base-Tx or 1000Base-T test mode.
Issue 10
© Solarflare Communications 2013
54
Solarflare Server Adapter
User Guide
3.16 Linux Utilities RPM
The Solarflare Utilities RPM for Linux contains:
• A boot ROM utility. Configuring the Boot ROM with sfboot...Page 56
• A flash firmware update utility. Upgrading Adapter Firmware with Sfupdate...Page 70
• A license key install utility. License Install with sfkey...Page 75
The RPM package, is supplied as 64bit and 32bit binaries compiled to be compatible with GLIBC
versions for all supported distributions.
The Solarflare utilities RPM file can be downloaded from the following location:
https://support.solarflare.com/
• SF-104451-LS is a 32bit binary RPM package.
• SF-107601-LS is a 64bit binary RPM package.
Download and copy the zipped binary RPM package to the required directory. Unzip and install
(64bit package example):
1
Unzip the package:
# unzip SF-107601-LS-20_Solarflare_Linux_Utilities_RPM_64bit.zip
2
Install the binary RPM:
# rpm -Uvh sfutils-4.0.1.6605-1.x86_64.rpm
Preparing...
1:sfutils
3
########################################### [100%]
########################################### [100%]
Check that the RPM installed correctly:
# rpm -q sfutils
sfutils-4.0.1.6605-1.x86_64
Directions for the use of the utility programs are explained in the following sections:
Issue 10
© Solarflare Communications 2013
55
Solarflare Server Adapter
User Guide
3.17 Configuring the Boot ROM with sfboot
• Sfboot: Command Usage...Page 56
• Sfboot: Command Line Options...Page 56
• Sfboot: Examples...Page 66
Sfboot is a command line utility for configuring the Solarflare adapter Boot ROM for PXE and iSCSI
booting. Using sfboot is an alternative to using Ctrl + B to access the Boot Rom agent during server
startup.
See Configuring the Solarflare Boot ROM Agent on page 360 for more information on the Boot Rom
agent.
PXE and iSCSI network boot is not supported for Solarflare adapters on IBM System p servers.
Sfboot: SLES 11 Limitation
Due to limitations in SLES 11 using kernel versions prior to 2.6.27.54 it is necessary to reboot the
server after running the sfboot utility.
Sfboot: Command Usage
The general usage for sfboot is as follows (as root):
sfboot [--adapter=eth] [options] [configurable parameters]
Note that without --adapter, the sfboot command applies to all adapters that are present in the
target host.
The format for the parameters are:
=
Sfboot: Command Line Options
Table 14 lists the options for sfboot and Table 15 lists the available options.
Table 14: Sfboot Options
Issue 10
Option
Description
-?-h, --help
Displays command line syntax and provides a
description of each sfboot option.
-V, --version
Shows detailed version information and exits.
-v, --verbose
Shows extended output information for the
command entered.
© Solarflare Communications 2013
56
Solarflare Server Adapter
User Guide
Table 14: Sfboot Options
Option
Description
-s, --quiet
Suppresses all output, except errors; no user
interaction. You should query the completion code to
determine the outcome of commands when
operating silently (see Performance Tuning on
Windows on page 236).
Aliases: --silent
Lists all available Solarflare adapters. This option
shows the ifname and MAC address.
-l --list
Note: this option may not be used in conjunction
with the any other option. If this option is used with
configuration parameters, those parameters will be
silently ignored.
-i, --adapter =
Performs the action on the identified Solarflare
network adapter. The adapter identifier ethX can be
the ifname or MAC address, as output by the --list
option. If --adapter is not included, the action will
apply to all installed Solarflare adapters.
-c --clear
Resets all adapter options except boot-image to
their default values. Note that --clear can also be
used with parameters, allowing you to reset to
default values, and then apply the parameters
specified.
The following parameters in Table 15 are used to control the configurable parameters for the Boot
ROM driver when running prior to the operating system booting.
Table 15: Sfboot Parameters
Issue 10
Parameter
Description
bootimage=
Specifies which boot firmware images are served-up
to the BIOS during start-up. This parameter can not
be used if the --adapter option has been specified.
This option is not reset if --clear is used.
© Solarflare Communications 2013
57
Solarflare Server Adapter
User Guide
Table 15: Sfboot Parameters
Parameter
Description
linkspeed=
Specifies the network link speed of the adapter used
by the Boot ROM - the default is auto. On the
10GBASE-T adapters “auto” instructs the adapter to
negotiate the highest speed supported in common
with it’s link partner. On SFP+ adapters, “auto”
instructs the adapter to use the highest link speed
supported by the inserted SFP+ module. On
10GBASE-T and SFP+ adapters, any other value
specified will fix the link at that speed, regardless of
the capabilities of the link partner, which may result
in an inability to establish the link.
auto Auto-negotiate link speed (default)
10G 10G bit/sec
1G 1G bit/sec
100M 100M bit/sec
linkup-delay=
Specifies the delay (in seconds) the adapter defers its
first connection attempt after booting, allowing time
for the network to come up following a power failure
or other restart. This can be used to wait for
spanning tree protocol on a connected switch to
unblock the switch port after the physical network
link is established. The default is 5 seconds.
banner-delay=
Specifies the wait period for Ctrl-B to be pressed to
enter adapter configuration tool.
seconds = 0-256
bootskip-delay=
Specifies the time allowed for Esc to be pressed to
skip adapter booting.
seconds = 0-256
boottype=
Sets the adapter boot type.
pxe – PXE (Preboot eXecution Environment) booting
iscsi – iSCSI (Internet Small Computer System
Interface) booting
disabled – Disable adapter booting
Issue 10
© Solarflare Communications 2013
58
Solarflare Server Adapter
User Guide
Table 15: Sfboot Parameters
Parameter
Description
initiatordhcp=
Enables or disables DHCP address discovery for the
adapter by the Boot ROM except for the Initiator IQN
(see initiator-iqn-dhcp). This option is only
valid if iSCSI booting is enabled (boottype=iscsi).
If initiator-DHCP is set to disabled, the following
options will need to be specified:
initiator-ip=
netmask=
The following options may also be needed:
gateway=
primary-dns=
initiator-ip=
Specifies the IPv4 address (in standard “.” notation
form) to be used by the adapter when initiatordhcp is disabled.
Note that this option is only valid if iSCSI booting is
enabled (boot-type=iscsi).
Example:
sfboot boot-type=iscsi initiatordhcp=disabled initiatorip=<192.168.1.3>
netmask=
Specifies the IPv4 subnet mask (in standard “.”
notation form) to be used by the adapter when
initiator-dhcp is disabled. Note that this option
is only valid if iSCSI booting is enabled (boottype=iscsi).
Example:
sfboot boot-type=iscsi initiatordhcp=disabled netmask=255.255.255.0
gateway=
Specifies the IPv4 subnet mask (in standard “.”
notation form) to be used by the adapter when
initiator-dhcp is disabled. Note that this option
is only valid if iSCSI booting is enabled (boottype=iscsi).
Example:
sfboot boot-type=iscsi initiatordhcp=disabled gateway=192.168.0.10
Issue 10
© Solarflare Communications 2013
59
Solarflare Server Adapter
User Guide
Table 15: Sfboot Parameters
Parameter
Description
primary-dns=
Specifies the IPv4 address (in standard “.” notation
form) of the Primary DNS to be used by the adapter
when initiator-dhcp is disabled.
This option is only valid if iSCSI booting is enabled
(boot-type=iscsi).
Example:
sfboot boot-type=iscsi initiatordhcp=disabled primary-dns=192.168.0.3
initiator-iqndhcp=
Enables or disables use of DHCP for the initiator IQN
only.
initiator-iqn=
Specifies the IQN (iSCSI Qualified Name) to be used
by the adapter when initiator-iqn-dhcp is
disabled. The IQN is a symbolic name in the “.”
notation form; for example:
iqn.2009.01.com.solarflare, and is a maximum of 223
characters long.
Note that this option is only valid if iSCSI booting is
enabled (boot-type=iscsi).
Example:
sfboot initiator-iqn-dhcp=disabled
initiatoriqn=iqn.2009.01.com.solarflare
adapter=2
lun-retry-count=
Specifies the number of times the adapter attempts
to access and login to the Logical Unit Number (LUN)
on the iSCSI Target before failing. Note that this
option is only valid if iSCSI booting is enabled (boottype=iscsi).
Example:
sfboot lun-retry-count=3
Issue 10
© Solarflare Communications 2013
60
Solarflare Server Adapter
User Guide
Table 15: Sfboot Parameters
Parameter
Description
targetdhcp=
Enables or disables the use of DHCP to discover iSCSI
target parameters on the adapter.
If target-dhcp is disabled, you must specify the
following options:
target-server=
target-iqn=
target-port=
target-lun=
Example - Enable the use of DHCP to configure iSCSI
Target settings:
sfboot boot-type=iscsi targetdhcp=enabled
target-server=
Specifies the iSCSI target’s DNS name or IPv4 address
to be used by the adapter when target-dhcp is
disabled.
Note that this option is only valid if iSCSI booting is
enabled (boot-type=iscsi).
Example:
sfboot boot-type=iscsi targetdhcp=disabled target-server=192.168.2.2
target-port=
Specifies the Port number to be used by the iSCSI
target when target-dhcp is disabled. The default
Port number is Port 3260.
Note that this option is only valid if iSCSI booting is
enabled (boot-type=iscsi).
Example:
sfboot boot-type=iscsi targetdhcp=disabled target-port=3262
This option should only be used if your target is using
a non-standard TCP Port.
target-lun=
Specifies the Logical Unit Number (LUN) to be used
by the iSCSI target when target-dhcp is disabled.
The default LUN is 0.
Note that this option is only valid if iSCSI booting is
enabled (boot-type=iscsi).
Issue 10
© Solarflare Communications 2013
61
Solarflare Server Adapter
User Guide
Table 15: Sfboot Parameters
Parameter
Description
target-iqn=
Specifies the IQN of the iSCSI target when targetdhcp is disabled. Maximum of 223 characters.
Note that this option is only valid if iSCSI booting is
enabled (boot-type=iscsi).
Note that if there are spaces contained in ,
then the IQN must be wrapped in double quotes (“”).
Example:
sfboot target-dhcp=disabled targetiqn=iqn.2009.01.com.solarflare
adapter=2
vendor-id=
Specifies the device vendor ID to be advertised to the
DHCP server. This must match the vendor id
configured at the DHCP server when using DHCP
option 43 to obtain the iSCSI target.
chap=
Enables or disables the use of Challenge Handshake
Protocol (CHAP) to authenticate the iSCSI
connection.
Note that this option is only valid if iSCSI booting is
enabled (boot-type=iscsi).
To be valid, this option also requires the following
sub-options to be specified:
username=
secret=
Example:
sfboot boot-type=iscsi chap=enabled
username=initiatorusername
secret=initiatorsecret
Issue 10
© Solarflare Communications 2013
62
Solarflare Server Adapter
User Guide
Table 15: Sfboot Parameters
Parameter
Description
username=
Specifies the CHAP initiator username (maximum 64
characters).
Note that this option is required if either CHAP or
Mutual CHAP is enabled (chap=enabled,
mutual-chap=enabled).
Note that if there are spaces contained in
, then it must be wrapped in double
quotes (“”).
Example:
sfboot boot-type=iscsi chap=enabled
username=username
Specifies the CHAP initiator secret (minimum 12
characters, maximum 20 characters).
secret=
Note that this option is valid if either CHAP or Mutual
CHAP is enabled (chap=enabled, mutualchap=enabled).
Note that if there are spaces contained in ,
then it must be wrapped in double quotes (“”).
Example:
sfboot boot-type=iscsi chap=enabled
username=username secret=veryverysecret
mutualchap=
Enables/disables Mutual CHAP authentication when
iSCSI booting is enabled.
This option also requires the following sub-options to
be specified:
target-username=
target-secret=
username=
secret=
Example:
sfboot boot-type=iscsi mutualchap=enabled username=username
secret=veryverysecret targetusername=targetusername targetsecret=anothersecret
Issue 10
© Solarflare Communications 2013
63
Solarflare Server Adapter
User Guide
Table 15: Sfboot Parameters
Parameter
Description
target-username=
Specifies the username that has been configured on
the iSCSI target (maximum 64 characters).
Note that this option is necessary if Mutual CHAP is
enabled on the adapter (mutual-chap=enabled).
Note that if there are spaces contained in
, then it must be wrapped in double
quotes (“”).
target-secret=
Specifies the secret that has been configured on the
iSCSi target (minimum 12 characters; maximum 20
characters).
Note: This option is necessary if Mutual CHAP is
enabled on the adapter (mutual-chap=enabled).
Note that if there are spaces contained in ,
then it must be wrapped in double quotes (“”).
mpio-priority=
Specifies the Multipath I/O (MPIO) priority for the
adapter. This option is only valid for iSCSI booting
over multi-port adapters, where it can be used to
establish adapter port priority. The range is 1- 255,
with 1 being the highest priority.
mpio-attempts=
Specifies the number of times MPIO will try and use
each port in turn to login to the iSCSI target before
failing.
msix-limit=
Specifies the maximum number of MSI-X interrupts
the specified adapter will use. The default is 32.
<8|16|32|64|128|256|512|1024>
Note: Using the incorrect setting can impact the
performance of the adapter. Contact Solarflare
technical support before changing this setting.
sriov=
Issue 10
Enable SR-IOV support for operating systems that
support this.
© Solarflare Communications 2013
64
Solarflare Server Adapter
User Guide
Table 15: Sfboot Parameters
Parameter
Description
vf-count=
The number of virtual functions (VF) advertised to
the operating system. The Solarflare SFC9000 family
of controllers support a total limit of 127 virtual
functions per port and a total 1024 interrupts.
Depending on the values of msix-limit and vf-msixlimit, some of these virtual functions may not be
configured.
Enabling all 127 VFs per port with more than one
MSI-X interrupt per VF may not be supported by the
host BIOS - in which case you may get 127 VFs on one
port and none on others. Contact your BIOS vendor
or reduce the VF count.
The sriov parameter is implied if vf-count is greater
than zero.
vf-msix-limit=<1|2|4|8>
The maximum number of interrupts a virtual
function may use.
firmware-variant=
For SFN7000 series adapters only.
The ultra-low-latency variant produces best latency
without support for TX VLAN insertion or RX VLAN
stripping (not currently used features). It is
recommended that Onload customers use the ultralow-latency variant.
Default value = auto - means the driver will select
ultra-low-latency by default.
Issue 10
© Solarflare Communications 2013
65
Solarflare Server Adapter
User Guide
Sfboot: Examples
• Show the current boot configuration for all adapters:
sfboot
Solarflare boot configuration utility [v3.0.3]
Copyright Solarflare Communications 2006-2010, Level 5 Networks 2002-2005
eth1:
Boot image
Link speed
Link-up delay time
Banner delay time
Boot skip delay time
Boot type
MSI-X interrupt limit
Option ROM and UEFI
Negotiated automatically
5 seconds
2 seconds
5 seconds
PXE
32
eth2:
Boot image
Link speed
Link-up delay time
Banner delay time
Boot skip delay time
Boot type
MSI-X interrupt limit
Option ROM and UEFI
Negotiated automatically
5 seconds
2 seconds
5 seconds
PXE
32
• List all Solarflare adapters installed on the localhost:
sfboot --list
Solarflare boot configuration utility [v3.0.2]
Copyright Solarflare Communications 2006-2010, Level 5 Networks 2002-2005
eth4 - 00-0F-53-01-3C-AC, eth5 - 00-0F-53-01-3C-AD, eth6 - 00-0F-53-013C-A4, eth7 - 00-0F-53-01-3C-A5
Issue 10
© Solarflare Communications 2013
66
Solarflare Server Adapter
User Guide
• Enable iSCSI booting on adapter eth4. Implement default iSCSI settings:
sfboot --adapter=eth4 boot-type=iscsi
Solarflare boot configuration utility [v3.0.2]
Copyright Solarflare Communications 2006-2010, Level 5 Networks 2002-2005
eth4:
Boot image
Link speed
Link-up delay time
Banner delay time
Boot skip delay time
Boot type
Use DHCP for Initiator
Use DHCP for Initiator IQN
LUN busy retries
Use DHCP for Target
DHCP Vendor Class ID
CHAP authentication
Mutual CHAP authentication
MPIO priority
MPIO boot attempts
MSI-X interrupt limit
Issue 10
Option ROM and UEFI
Negotiated automatically
5 seconds
2 seconds
5 seconds
iSCSI
Enabled
Enabled
2
Enabled
SFCgPXE
Disabled
Disabled
0
3
32
© Solarflare Communications 2013
67
Solarflare Server Adapter
User Guide
• iSCSI enable adapter eth2. Disable DHCP. Specify adapter IP address and netmask:
sfboot boot-type=iscsi --adapter=eth2 initiator-dhcp=disabled initiatorip=192.168.0.1 netmask=255.255.255.0
Solarflare boot configuration utility [v3.0.2]
Copyright Solarflare Communications 2006-2010, Level 5 Networks 2002-2005
eth2:
Boot image
Link speed
Link-up delay time
Banner delay time
Boot skip delay time
Boot type
Use DHCP for Initiator
Initiator IP address
Initiator netmask
Initiator default gateway
Initiator primary DNS
Use DHCP for Initiator IQN
LUN busy retries
Use DHCP for Target
DHCP Vendor Class ID
CHAP authentication
Mutual CHAP authentication
MPIO priority
MPIO boot attempts
MSI-X interrupt limit
Option ROM and UEFI
Negotiated automatically
5 seconds
2 seconds
5 seconds
iSCSI
Disabled
192.168.0.1
255.255.255.0
0.0.0.0
0.0.0.0
Enabled
2
Enabled
SFCgPXE
Disabled
Disabled
0
3
32
• Enable PXE boot
sfboot boot-image=option rom boot-type=pxe
• Enable SR-IOV
sfboot sriov=enabled vf-count=16 vf-msix-limit=1
Issue 10
© Solarflare Communications 2013
68
Solarflare Server Adapter
User Guide
• SFN7000 Series - Firmware Variant
sfboot firmware-variant=full-feature
Solarflare boot configuration utility [v4.0.0]
Copyright Solarflare Communications 2006-2013, Level 5 Networks 2002-2005
eth4:
Boot image
Link speed
Link-up delay time
Banner delay time
Boot skip delay time
Boot type
MSI-X interrupt limit
Number of Virtual Functions
VF MSI-X interrupt limit
Firmware variant
Issue 10
Option ROM only
Negotiated automatically
7 seconds
3 seconds
6 seconds
PXE
32
0
1
full feature / virtualization
© Solarflare Communications 2013
69
Solarflare Server Adapter
User Guide
3.18 Upgrading Adapter Firmware with Sfupdate
• Sfupdate: Command Usage...Page 70
• Sfupdate: Command Line Options...Page 73
• Sfupdate: Examples...Page 74
Sfupdate is a command line utility to manage and upgrade the Solarflare adapter Boot ROM, Phy and
adapter firmware. Embedded within the sfupdate executable are firmware images for various
Solarflare adapters - the exact updates available via sfupdate depend on the adapter.
See Configuring the Solarflare Boot ROM Agent on page 360 for more information on the Boot Rom
agent.
NOTE: All Applications accelerated with OpenOnload should be terminated before updating the
firmware with sfupdate.
Sfupdate: Command Usage
The general usage for sfupdate is as follows (as root):
sfupdate [--adapter=eth] [options]
where:
ethN is the interface name (ifname) of the Solarflare adapter to be upgraded.
option is one of the command options listed in Table 16.
The format for the options are: