Mellanox Technologies SwitchX User manual

Type
User manual
www.mellanox.com
SwitchX 36-Port QSFP 40Gb/s Ethernet System
Hardware User Manual
P/N:MSX10XX
Rev 1.3
Document Number: 3459
Rev 1.3
Mellanox Technologies
2
Mellanox Technologies
350 Oakmead Parkway Suite 100
Sunnyvale, CA 94085
U.S.A.
www.mellanox.com
Tel: (408) 970-3400
Fax: (408) 970-3403
Mellanox Technologies, Ltd.
Beit Mellanox
PO Box 586 Yokneam 20692
Israel
www.mellanox.com
Tel: +972 (0)74 723 7200
Fax: +972 (0)4 959 3245
Mellanox®, Mellanox logo, BridgeX®, ConnectX®, CORE-Direct®, InfiniBridge®, InfiniHost®, InfiniScale®, PhyX®,
SwitchX®, Virtual Protocol Interconnect® and Voltaire® are registered trademarks of Mellanox Technologies, Ltd.
Connect-IB™, FabricIT™, MLNX-OS™, ScalableHPC™, Unbreakable-Link™, UFM™ and Unified Fabric Manager™ are
trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners.
© Copyright 2012. Mellanox Technologies. All Rights Reserved.
NOTE:
THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED
DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY
KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE
THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT
HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE
PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND
DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST
QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES
FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND
(INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED
DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
3
Table of Contents
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
About this Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Abreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Mellanox Part Numbering Legend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Chapter 1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.1 Features- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 12
1.1.1 Full Feature List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2 Serial Number and Product Version Information - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 12
Chapter 2 40 Gb/s Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Chapter 3 Basic Operation and Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1 Switch Platform Hardware Overview - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 15
3.1.1 Status LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.2 Reset Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 Air Flow - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 22
3.3 QSFP Cable Power Budget Classification - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 22
3.4 Interfaces - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 23
3.4.1 Port Connector Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.2 Management and Firmware Updating Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.5 Switch Platform Installation and Operation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 24
3.5.1 Installation Safety Warnings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.5.2 Mechanical Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.6 Package Contents and Installation - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 28
3.6.1 Installing the Switch in the Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.7 Grounding the Switch - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 33
3.8 Power Connections and Initial Power On - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 33
3.9 Extracting and Inserting the Power Supply Unit - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 35
3.10 Supported Approved Cables - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 36
3.11 Splitting the Data Stream Using Breakout (Fanout) Cables- - - - - - - - - - - - - - - - - - - - - - - - - 36
3.12 Cable Installation- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 39
3.13 Extracting and Inserting the Fan Unit- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 40
3.14 Switch Shut Down Procedure - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 41
3.15 Disassembly of the Switch from the Rack - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 42
3.16 Disposal - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 42
3.17 Resetting the Switch- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 42
Chapter 4 Management and Tools Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1 Chassis Management Using the MLNX-OS™ Software - - - - - - - - - - - - - - - - - - - - - - - - - - - 44
4.2 Configuring the Switch for the First Time - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 44
4.2.1 Starting a Remote Connection to the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2.2 Upgrading Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Chapter 5 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Appendix A Specification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
A.1 Approved Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
A.2 EMC Certifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
A.3 China CCC Warning Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Rev 1.3
Mellanox Technologies
4
Appendix B Transferring the Power Cord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Appendix C QSFP Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Appendix D RJ-45 CONSOLE and Ethernet interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58
Appendix E Replacement Parts Ordering Numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Appendix F Avertissements de sécurité d’installation (French) . . . . . . . . . . . . . . . . . . . . . . . 60
Appendix G Installation - Sicherheitshinweise (German) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
Appendix H Advertencias de seguridad para la instalación (Spanish) . . . . . . . . . . . . . . . . . . .68
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
5
List of Figures
Figure 1: Connector Side View of the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Figure 2: Pull Out Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Figure 3: QSFP Switch System Power and Connector Side Panels . . . . . . . . . . . . . . . . . . . . . . . .15
Figure 4: System Status Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
Figure 5: Power, Fan, and System LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Figure 6: Fan Status LED Connector Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Figure 7: Fan Status LED Power side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Figure 8: Power Side Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Figure 9: PS Unit Status LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Figure 10: Bad Port LED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Figure 11: Reset Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Figure 12: Port LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Figure 13: Port Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Figure 14: Top and Bottom Port Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Figure 15: Management Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Figure 16: Rack Rail Kit Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Figure 17: Placement of Switch in Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Figure 18: Mounting Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Figure 19: Screwing on the Rail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Figure 20: Inserting the Caged Nuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Figure 21: Slide the Rail into the Rail Slide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Figure 22: Status LEDs 5 Minutes After Power On . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Figure 23: Two Power Inlets - Electric Caution Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Figure 24: Fan and PS unit Air Flow Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
Figure 25: Power Supply Unit Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
Figure 26: PS Unit Pulled Out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Figure 27: Breakout or Fanout Cable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37
Figure 28: Port Splitting Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37
Figure 29: 3 Examples of Port Mapping Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39
Figure 30: Top and Bottom Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40
Figure 31: Air Flow Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41
Figure 32: Fan Module Latches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41
Figure 33: Reset Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
Figure 34: Host Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
Figure 35: Transfer Power Cord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
Figure 36: Put the Cord Through the Switch Slide Before Screwing it to the Switch . . . . . . . . . . . .54
Rev 1.3
Mellanox Technologies
6
Figure 37: Screw the Switch Slide to the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Figure 38: Transfer Power Cord Finished . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Figure 39: QSFP Connector Male and Female Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
7
List of Tables
Table 1: Revision History Table 8
Table 2: Reference Documents and Web Sites 9
Table 3: System Status LED Configurations 17
Table 4: Fan Status LED Configurations 18
Table 5: PS Unit Status LED Configurations 19
Table 6: Bad Port LED Configurations 20
Table 7: Port Connector Physical and Logical Link Indications for Ethernet 21
Table 8: Air Flow Direction 22
Table 9: Installation Kit According to Rack Size 27
Table 10: Port Splitting Options 38
Table 11: SX103[56]Specification Data 49
Table 12: InfiniBand QSFP Connector Pinout 56
Table 13: RJ-45 Pinout 58
Table 14: Replacement Parts Ordering Numbers 59
Rev 1.3
Mellanox Technologies
8
Revision History
Table 1 - Revision History Table
Date Revision Description
October 2012 1.3 Added caution not to insert cables with excessive force.
September 2012 1.2 Added examples of Port mapping
Replaced Visio Graphics with Mechanical drawings.
Added Figure 17.
Added China CCC warning statement
February 2012 1.1 Added one OPN
Added China RoHS to the parts list
Added Appendix for Thermal Threshold Definitions
Removed reference to spring clip in Appendix “Transferring the Power Cord”
Changed Specifications table formatting 4 column to 2 column
Fixed port LED assignment
Added Cable Splitting section
July 2011 1.0 Initial Release
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
9
About this Manual
This manual describes the installation and basic use of the Mellanox switch which is based on the
SwitchX switch device.
Intended Audience
This manual is intended for users and system administrators responsible for designing and test-
ing switch platforms.
Related Documentation
Additional Documentation available from Mellanox:
All of these documents can be found on the Mellanox Website. They are available either through
the product pages or through the support page with a login and password.
Conventions
Throughout this manual, the name SX10XX and the term switch are used to describe the 36-port
QSFP40Gb/s Ethernet switch, unless explicitly indicated otherwise.
The following icons are used throughout this document to indicate information that is important
to the user.
Table 2 - Reference Documents and Web Sites
Document Name Description
Mellanox SwitchX® Switch Products
Release Notes
For possible hardware issues see the switch support product page. This
requires a customer support login.
Mellanox MLNX-OS SwitchX Soft-
ware User Manual
This document contains information regarding configuring and manag-
ing Mellanox Technologies' SwitchX Switch Platforms.
MLNX-OS Software Command Refer-
ence Guide
Command Reference Guide for MLNX-OS listing all of the commands
available through MLNX-OS with explanations and examples.
MLNX-OS Software Configuration
Guide
Configuration Guide for MLNX-OS displaying different configuration
scenarios.
This icon makes recommendations to the user.
This icon indicates information that is helpful to the user.
Rev 1.3
Mellanox Technologies
10
Abreviations
FDR — Fourteen Data Rate
Mellanox Part Numbering Legend
This icon indicates a situation that can potentially cause damage to hardware or soft-
ware.
BEWARE! This icon indicates a situation that can potentially cause personal injury or
damage to hardware or software.
Place Field Decoder
M Mellanox Technologies
SX System Type SwitchX Switch
P Data Transfer Protocol 1 = Ethernet
R Size of box 0 = 1U
1 = 1.5U
2 = 2U
40 Gb/s equivalent
throughput
16 = 640Gb/s
24 = 960Gb/s
35 = 2880 Gb/s Light version of the Management software
36 = 2880 Gb/s
C Data Rate B = 40Gb/s Ethernet
- Separator
P # Power Supplies 0=0, 1=1, 2=2....
M Depth of the Unit S = standard depth, B = short depth
Y Air Flow direction R= Connector side to Power side airflow
F= Power Side side to Connector side airflow
R Chip Generation R – SwitchX
S – SwitchX-2
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
11
1 Overview
The SX10xx Ethernet system switch family provides the highest-performing fabric solution in a
1U form factor by delivering up to 2.88Tb/s of non-blocking throughput to High-Performance
Computing (HPC), high frequency trading and Enterprise Data Centers (EDC), with ultra-low-
latency. It has 36 40GbE ports that when connected to Mellanox NICs with Mellanox cables and
adapters gives you the optimal end-to-end solution for Ethernet Data-Centers.
Figure 1: Connector Side View of the Switch
OverviewRev 1.3
Mellanox Technologies
12
1.1 Features
1.1.1 Full Feature List
1.2 Serial Number and Product Version Information
The Serial number and the MAC for the switch are found on the pull out tab below the USB con-
nector. The MAC for the Management PC is also on this tab. This MAC is for the mgmt0 man-
agement interface (the top Ethernet port). The second interface mgmt1 has the next consecutive
MAC.
•DHCP
•Telnet
SSHv2
•CLI
FTP
TFTP
•SCP
Upload/download configuration
Static MAC
•Syslog
System alarms
Auto temperature control
Simple Network Time Protocol SNTP
•RADIUS
•TACACS+
LDAP
802.1w RSTP
802.3x Flow Control
802.1Qab PFC
•802.1Qaz - ETS
•Dual SW Image
Familiar industry standard CLI
Forward Error Correction (FEC) *
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
13
Figure 2: Pull Out Tab
S/N: MT1117X00014
P/N: MSX1035B-1BFR
MAC: 0002C9020040
Made in IL
MGT :
00:02:C9:11:A2:01
Rev:A1
36
35
MGT
1
2
SX1035
CONSOLE
34
33
Pull out tab
40 Gb/s EthernetRev 1.3
Mellanox Technologies
14
2 40 Gb/s Ethernet
Each QSFP port is capable of up to 40Gb/s as well as 10Gb/s with a QSA QSFP to SFP+ adaptor
or QSFP to SFP+ Mellanox cables. The switch contains 36 ports of 40GbE. Each port can be
connected with QSFP cables and connectors for 40Gb/s speed, or 10Gb/s speed when connecting
through QSA Mellanox adaptors.
Specific ports can support breakout cables. See “Splitting the Data Stream Using Breakout
(Fanout) Cables” on page 36 for mor information.
40 GbE is only guaranteed to work with approved Mellanox Cables.
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
15
3 Basic Operation and Installation
3.1 Switch Platform Hardware Overview
Figure 3 shows the power side panel and connector side panel views of the switch. The figure
shows port configurations for the switch systems, the hot-swap power supply location, the hot-
swap fan module, 2 – Ethernet RJ-45 connectors for management, 1 – RJ-45 connector for con
-
necting to a host PC, 1 – USB connector, and various status LEDs.
Figure 3: QSFP Switch System Power and Connector Side Panels
All connectivity except for power cords is via the connector side panel. All connectors can sup-
port active cables.
3.1.1 Status LEDs
3.1.1.1 System Status Indicators
The System Status Indicators are located to the left of the connectors on the connector side panel,
and on the power side at the far right. Both of these LEDs give identical information.
Figure 4: System Status Indicators
OK
!
!
OK
OK
!
!
OK
12C
SX1036
Power Side Panel
Connector Side Panel
Power side
status LED
Connector side
status LED
Green – OK
Yellow – a fault in the system
Red – Major Fault or Fatal Error
LED color shows the
switch status
These three symbols show the
possible colors for the LED
below.
Both of these LEDs show the
switch status. One is on the
Power side and one is on the
Connector side
Basic Operation and InstallationRev 1.3
Mellanox Technologies
16
The system status indicators should display as follows:
When the switch is plugged in, within five minutes the STATUS LED should light up
green.
The PS1 LED for the plugged in PS unit should light up green
The PS2 LED for the second PS unit should light up green only if a second PS unit is
installed in the switch for redundancy and Hot-Swap ability and it is connected to a
power source.
If two PS units are installed and only one PS unit is connected to a power supply the PS2
LED will be red.
If only one PS unit is installed in the switch, the PS2 LED will be off.
The FAN LED should light up green.
If the FAN LED shows red, troubleshoot the fan module. See “Troubleshooting” on page 46.
If the PS LEDs are not green, this indicates a problem with the power supplies. The switch is
operational only if at least one of the PS LEDs is green.
As long as there is power to the switch (one PS unit is connected), and the switch is
booted up and running, the status LED should be green.
If the STATUS LED shows red after five minutes, unplug the switch and call your
Mellanox representative for assistance.
If the switch shuts down due to over temperature, unplug the switch, wait 5 minutes
and replug in the switch. For more information
See “Troubleshooting” on page 46.
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
17
Figure 5: Power, Fan, and System LEDs
3.1.1.2 Fan Status Indicators
The indicator is located to the left of the connectors on the connector side panel.
Figure 6: Fan Status LED Connector Side
The LED indicator on the Fan Module is on the right side of the module.
Table 3 - System Status LED Configurations
LED Configuration STATUS/ System Health LED
Solid Green OK – The system is up and running.
Flashing Green The system is booting up. This assignment is valid on managed systems only.
Solid Red Major Error – Possible damage can result to the switch. Turn off immediately.
e.g. bad firmware, can’t boot, overheated
Off Off – The system has no power.
!
PS1
PS2
!
UID
RST
1
2
This is a two color LED
Green — OK
Red — Major fault
Off — No power to the fans
Basic Operation and InstallationRev 1.3
Mellanox Technologies
18
Figure 7: Fan Status LED Power side
The following fan status conditions are possible:
3.1.1.3 Power Supply Status Indicators
TheSX10XX switch is available with one factory installed Power Supply Unit. A second Power
Supply Unit can be added to support hot-swap ability and to add redundancy. See
Section F,
“Replacement Parts Ordering Numbers,” on page 59 for ordering part numbers.
Table 4 - Fan Status LED Configurations
LED Configuration FAN LED
Solid Green OK – All fans are up and running
Solid Red Error – One or more fans is not operating properly. The system should be powered
down and troubleshoot the fan module.
Off Off – The fan unit is not receiving any power. Check that the fan unit is properly
and completely inserted.
All fans must be operating while the power supply is plugged in.
If the switch shuts down due to over temperature, unplug the switch, wait 5 minutes
and replug in the switch. For more information
See “Troubleshooting” on page 46.
!
This is a two color LED
Green — OK
Red — Major fault
Off — No power to the fans
Fan air flow indicator
SwitchX 36-Port QSFP 40Gb/s Ethernet System Hardware User Manual Rev 1.3
Mellanox Technologies
19
Figure 8: Power Side Panel
The primary power supply unit (PS1) is located on the right side of the power side panel, with
PS2 on the left side. Each PS unit has a single 2 color LED on the right side of the PS unit, that
indicates the internal status of the unit.
Figure 9: PS Unit Status LEDs
Table 5 - PS Unit Status LED Configurations
LED Color Status
Solid Green OK – The Power supply is delivering the correct voltage – 12VDC
Solid Red Error – The PS unit is not operational
Off Off – There is no power to the system (neither PS unit is receiving power). If one
PS unit is showing green and the second PS unit is unplugged it will show a red
indication.
OK
!
!
OK
OK
!
!
OK
12C
Primary Power Supply Unit
Fan Unit
PS unit 1 status LED
Fan status LED
This is a two color LED
Green — OK
Red — Major fault
Off — No power to the system
Power supply fan air flow
indicator
Basic Operation and InstallationRev 1.3
Mellanox Technologies
20
3.1.1.4 Bad Port LED
Figure 10: Bad Port LED
The Bad Port indicator is located on the left side of the connector side panel of the unit. The fol-
lowing Bad Port conditions are possible.
This LED shows symbol errors. Possible causes for this are:
Bad cable
Bad connection
Bad connector
This LED lights up when one or more ports is receiving a symbol error. The LED immediately
goes off until the next symbol error is received.
3.1.1.5 UID LED Switch Identifier
The UID LED is a debug feature that lights a blue LED on the switch connector side panel for
ease in finding a particular switch.
3.1.2 Reset Button
On the connector side panel under the system LEDs is a reset button. This reset button requires a
tool to be pressed.
Table 6 - Bad Port LED Configurations
LED Configuration Description
Off OK – No symbol errors have been received in last few seconds
Flashing Orange Error – One or possibly more ports have just received a symbol error
DO NOT use a sharp pointed object such as needle or push pin for pressing the Reset
button. Sharp objects can cause damage, use a flat object to push the reset button.
!
PS1
PS2
!
UID
RST
1
2
This is a one color LED
Flashing Orange — received symbol
error
UID
  • Page 1 1
  • Page 2 2
  • Page 3 3
  • Page 4 4
  • Page 5 5
  • Page 6 6
  • Page 7 7
  • Page 8 8
  • Page 9 9
  • Page 10 10
  • Page 11 11
  • Page 12 12
  • Page 13 13
  • Page 14 14
  • Page 15 15
  • Page 16 16
  • Page 17 17
  • Page 18 18
  • Page 19 19
  • Page 20 20
  • Page 21 21
  • Page 22 22
  • Page 23 23
  • Page 24 24
  • Page 25 25
  • Page 26 26
  • Page 27 27
  • Page 28 28
  • Page 29 29
  • Page 30 30
  • Page 31 31
  • Page 32 32
  • Page 33 33
  • Page 34 34
  • Page 35 35
  • Page 36 36
  • Page 37 37
  • Page 38 38
  • Page 39 39
  • Page 40 40
  • Page 41 41
  • Page 42 42
  • Page 43 43
  • Page 44 44
  • Page 45 45
  • Page 46 46
  • Page 47 47
  • Page 48 48
  • Page 49 49
  • Page 50 50
  • Page 51 51
  • Page 52 52
  • Page 53 53
  • Page 54 54
  • Page 55 55
  • Page 56 56
  • Page 57 57
  • Page 58 58
  • Page 59 59
  • Page 60 60
  • Page 61 61
  • Page 62 62
  • Page 63 63
  • Page 64 64
  • Page 65 65
  • Page 66 66
  • Page 67 67
  • Page 68 68
  • Page 69 69
  • Page 70 70

Mellanox Technologies SwitchX User manual

Type
User manual

Ask a question and I''ll find the answer in the document

Finding information in a document is now easier with AI

in other languages