Mellanox Technologies SwitchX-2 MSX1024B-2BFS User manual

Category
Network switches
Type
User manual
www.mellanox.com
SwitchX®-2 60 Port Ethernet Switch System
Hardware User Manual
P/N: MSX1024B-1BFS, MSX1024B-1BRS, MSX1024B-2BFS, MSX1024B-2BRS
Rev 1.1
Document Number: 4083
Rev 1.0
Mellanox Technologies
2
Mellanox Technologies
350 Oakmead Parkway Suite 100
Sunnyvale, CA 94085
U.S.A.
www.mellanox.com
Tel: (408) 970-3400
Fax: (408) 970-3403
Mellanox Technologies, Ltd.
Beit Mellanox
PO Box 586 Yokneam 20692
Israel
www.mellanox.com
Tel: +972 (0)74 723 7200
Fax: +972 (0)4 959 3245
© Copyright 2013. Mellanox Technologies. All Rights Reserved.
Mellanox®, Mellanox logo, BridgeX®, ConnectX®, CORE-Direct®, InfiniBridge®, InfiniHost®, InfiniScale®,
MLNX-OS®, PhyX®, SwitchX®, UFM®, Virtual Protocol Interconnect® and Voltaire® are registered trademarks of
Mellanox Technologies, Ltd.
Connect-IB™, FabricIT™, Mellanox Open Ethernet™, MetroX™, MetroDX™, ScalableHPC™, Unbreakable-Link™ are
trademarks of Mellanox Technologies, Ltd.
All other trademarks are property of their respective owners.
NOTE:
THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED
DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY
KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE
THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT
HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE
PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND
DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST
QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
ARE DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES
FOR ANY DIRECT, INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND
(INCLUDING, BUT NOT LIMITED TO, PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND RELATED
DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
3
Table of Contents
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
About this Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Chapter 1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.1 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2 Serial Number and Product Version Information . . . . . . . . . . . . . . . . . . . . . . . . 11
Chapter 2 Ethernet Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Chapter 3 Basic Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1 Switch Platform Hardware Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 Status LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2.1 System Status Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2.2 Fan Status Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.3 Power Supply Status Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.3 Air Flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.4 SFP+ Cable Power Budget Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.5 Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.5.1 Port Connector Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.5.2 Port Speed Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.5.3 Management and Firmware Updating Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.5.4 Reset Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Chapter 4 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.1 Installation Safety Warnings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2 Installation Kits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.3 Package Contents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4 Mechanical Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.5 Grounding the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.6 Power Connections and Initial Power On . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.7 Battery Replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.8 Configuring the Switch the First Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.9 Managed (Internally Managed) Switches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.10 Rerunning the Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.11 Connecting to the Switch Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.11.1 Starting a Remote Connection to the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.12 Starting an SSH Connection to the Switch (CLI) . . . . . . . . . . . . . . . . . . . . . . . . 42
Chapter 5 Extracting and Inserting FRUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.1 Extracting and Inserting the Power Supply Unit . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.2 Extracting and Inserting the Fan Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Rev 1.1
Mellanox Technologies
4
Chapter 6 Cabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.1 Splitting the Data Stream Using Breakout (Fanout) Cables . . . . . . . . . . . . . . . . 47
6.2 Cable Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Chapter 7 Switch Shut Down Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Chapter 8 Disassembly of the Switch from the Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8.1 Disposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
8.2 Upgrading Software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Chapter 9 Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Appendix A Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
A.1 Approved Cables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
A.2 EMC Certifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
A.3 China CCC Warning Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Appendix B SFP+ Removal Tool Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Appendix C QSFP Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Appendix D SFP+ Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Appendix E RJ-45 CONSOLE and I2C Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Appendix F Accessory and Replacement Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Appendix G Avertissements de sécurité d’installation (French) . . . . . . . . . . . . . . . . . . 65
Appendix H Installation - Sicherheitshinweise (German) . . . . . . . . . . . . . . . . . . . . . . . 69
Appendix I Advertencias de seguridad para la instalación (Spanish) . . . . . . . . . . . . 72
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
5
List of Figures
Figure 1: Connector Side View of the Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Figure 2: Pull Out Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Figure 3: QSFP Switch System Power and Connector Side Panels . . . . . . . . . . . . . . . . . . . . . . .14
Figure 4: System Status Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Figure 5: Fan Status LED Connector Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Figure 6: Fan Status LED Power side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Figure 7: Power Status LED Connector Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Figure 8: Power Side Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Figure 9: PS Unit Status LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Figure 10: Bad Port LED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Figure 11: Port LED to Port Location Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Figure 12: Port LEDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Figure 13: Component Airflow Mismatch (Wrong) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Figure 14: Port Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Figure 15: Top and Bottom QSFP Port Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Figure 16: Top and Bottom SFP+ Port Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Figure 17: Reset Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Figure 18: Rack Rail Kit Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Figure 19: Placement of Switch in Rack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Figure 20: Mounting Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Figure 21: Screwing on the Rail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Figure 22: Inserting the Caged Nuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Figure 23: Slide the Rail into the Rail Slide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Figure 24: Status LEDs 5 Minutes After Power On . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Figure 25: Two Power Inlets - Electric Caution Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
Figure 26: Console Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Figure 27: Fan and PS unit Air Flow Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
Figure 28: Power Supply Unit Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
Figure 29: PS Unit Pulled Out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
Figure 30: Air Flow Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
Figure 31: Fan Module Latches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
Figure 32: Breakout or Fanout Cable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
Figure 33: Port Splitting Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
Figure 34: SFP+ Cable Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Figure 35: QSFP Connector Male and Female Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
Rev 1.1
Mellanox Technologies
6
Figure 36: Rear View of Module With Pin Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
7
List of Tables
Table 1: Revision History Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
Table 2: Reference Documents and Web Sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Table 3: System Status LED Assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
Table 4: Fan Status LED Assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Table 5: PS Unit Status LED Assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Table 6: Bad Port LED Assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Table 7: Port Connector Physical and Logical Link Assignments for Ethernet . . . . . . . . . . . . . . .20
Table 8: Air Flow Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
Table 9: Installation Kit According to Rack Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Table 10: Serial Terminal Program Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37
Table 11: Configuration Wizard Session - IP Configuration by DHCP . . . . . . . . . . . . . . . . . . . . . .37
Table 12: Configuration Wizard Session - IP Zeroconf Configuration . . . . . . . . . . . . . . . . . . . . . .39
Table 13: Configuration Wizard Session - Static IP Configuration . . . . . . . . . . . . . . . . . . . . . . . . .40
Table 14: SX1024 Specification Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Table 15: RJ-45 CONSOLE Pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
Table 16: RJ-45 I2C Pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63
Table 17: OPNs for Replacement Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
Rev 1.1
Mellanox Technologies
8
Revision History
Table 1 - Revision History Table
Date Revision Description
March 2013 1.1 Added SFP+ cable and module removal tool to contents.
Added figures to appendix.
Added note use only FCC compliant Ethernet cables.
Added note concerning USB insertion
Replaced wizard tables from MLNX-OS doc
added section opening the CLI
January 2013 1.0 Initial Release
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
9
About this Manual
This manual describes the installation and basic use of the Mellanox switch which is based on the
SwitchX®-2 switch device.
Intended Audience
This manual is intended for users and system administrators responsible for installing, running,
and managing switch platforms.
The manual assumes familiarity with the Ethernet specification.
Related Documentation
Additional Documentation available from Mellanox:
All of these documents can be found on the Mellanox Website at www.mellanox.com. They are
available either through the product pages www.mellanox.com Products InfiniBand/VPI
Switch Systems or through the support page with a login and password.
Conventions
Throughout this manual, the name SX1024 and the term switch are used to describe the 60 port
SFP+/QSFP switch unless explicitly indicated otherwise.
The following icons are used throughout this document to indicate information that is important
to the user.
Table 2 - Reference Documents and Web Sites
Document Name Description
SwitchX®-2 Switch Hardware Installation
Guide
The Installation Guide contains instructions for Installation in a rack and initial-
ization of the switch platform. This document can be found on the product web
page.
SwitchX®-2 Switch Product Hardware
Release Notes
For possible hardware issues see the switch support product page. This
document can be found on the support web page for this product.
This requires a customer support login. Look up the relevant SwitchX®-
based switch system/series release note file.
Mellanox MLNX-OS® Software User
Manual
This document contains information regarding configuring and manag-
ing Mellanox Technologies Switch Platforms.
MLNX-OS® Software Command Ref-
erence Guide
Command Reference Guide for MLNX-OS® listing all of the commands
available through MLNX-OS® with explanations and examples.
This icon makes recommendations to the user.
This icon indicates information that is helpful to the user.
Rev 1.1
Mellanox Technologies
10
Abbreviations
– Used to indicate a 4X port running at 56Gb/s
Mellanox Part Numbering Legend
This icon indicates a situation that can potentially cause damage to hardware or soft-
ware.
BEWARE! This icon indicates a situation that can potentially cause personal injury or
damage to hardware or software.
Place Field Decoder
M Mellanox Technologies
SX System Type SwitchX® switch Family
P Data Transfer Protocol 1 = Ethernet
R Size of box 0 = 1U
1 = 1.5U
2 = 2U
FF 40 Gb/s equivalent
throughput
16 = 640Gb/s
24 = 960Gb/s
35 = 2880 Gb/s Light version of the Management software
36 = 2880 Gb/s
C Data Rate X = 10Gb/s Ethernet B = 40Gb/s Ethernet
- Separator
P # Power Supplies 0=0, 1=1, 2=2....
M Depth of the Unit S = standard depth, B = short depth
Y Air Flow direction R= Connector side to Power side airflow
F= Power Side side to Connector side airflow
R Chip Generation R – SwitchX
S – SwitchX-2
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
11
1 Overview
The SX1024 Ethernet switch family provides the highest-performing fabric solution in a 1U form
factor by delivering up to 1.152 Gb/s of non-blocking throughput to High-Performance Comput
-
ing (HPC), high frequency trading and Enterprise Data Centers (EDC), with ultra-low latency. It
has 48 10GbE ports that when connected to 10GbE based servers, provides a unique non-block
-
ing solution for Ethernet Data-Centers.
Figure 1: Connector Side View of the Switch
1.1 Features
Based on Mellanox SwitchX-2
Supports 48 SFP+ ports of 10 GbE
Supports 12 QSFP ports of 40 GbE Upgradable with a license to 56 GbE
2x 10/100/1000BASE-T RJ-45 Ethernet port
1x RS232 RJ-45 Console connector
1x USB connector
1 + 1 redundant 300W AC/DC hot swappable Power supplies
3 + 1 redundant fans ordered in two fan units (Support Air in/out)
On board Temperature monitoring
Reset Push button on front panel
Various status LEDs
19” rack mountable chassis, 1U with redundant power supplies and Fan units
1.2 Serial Number and Product Version Information
The serial number and the MAC for the switch are found on the pull out tab below the Mellanox
logo on the right side of the Connector side panel. The MAC for the Management PC is also on
this tab. This MAC is for the mgmt0 management interface (the top Ethernet port). The second
interface mgmt1 has the next consecutive MAC. On this switch the mgmt1 interface is found on
the power side on the left side.
RST
UID
S
X
1
0
2
4
M
G
T
C
O
N
S
O
L
M
e
l
l
a
n
o
x
OverviewRev 1.1
Mellanox Technologies
12
Figure 2: Pull Out Tab
SX1024
Mellanox
6059
S/N:M T1244X03494
P/N:M SX1024B1BFS
MAC:0002C9020040
MGTMAC:
00:02:C9:11:A2:01
Rev:A1
244X6511240006
Pull out tab
mgmt0
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
13
2 Ethernet Ports
This Ethernet switch system supports 48 SFP+ ports, 12 QSFP ports and maintains a 4:1 ratio.
Each SFP+ port is capable of 10GbE/1GbE port speeds. Up to 24 SFP+ ports are capable of 1/2/
4/8G FC. Ports 1-12 are grouped as FC ports as well as ports 38-48.
All Ethernet ports can be manually configured through the CLI management interface.
Each of the 12 QSFP ports is capable of up to 40GbE, upgradable to 56 GbE with a license, as
well as 10GbE with a QSA QSFP to SFP+ adapter or QSFP to SFP+ Mellanox cables.
Each of the 48 SFP+ ports is capable of up to 10GbE.
40 GbE and 56 GbE are only guaranteed to work with approved cables and modules.
Basic OperationRev 1.1
Mellanox Technologies
14
3 Basic Operation
3.1 Switch Platform Hardware Overview
Figure 3 shows the power side panel view of the switch. The figure shows the hot-swap power
supply locations and the hot-swap fan modules, and an I2C banana connector.
The figure also shows the connector side panel views of the internally managed switch. The fig-
ure shows port configurations for the switch systems. Managed systems have 2 – Ethernet RJ-45
connectors for management, 1 – RJ-45 connector for connecting to a host PC, 1 – USB connec
-
tor. All switches have various status LEDs for on site switch status information.
Figure 3: QSFP Switch System Power and Connector Side Panels
3.2 Status LEDs
3.2.1 System Status Indicators
The System Status Indicators are located to the left of the connectors on the connector side panel,
and on the power side at the far right. Both of these LEDs give identical information.
Figure 4: System Status Indicators
!
OK
!
OK
!
OK
!
OK
Mellanox
SX1024
RST
UID
Mellanox
605958575655545352515049
PS
Power Side Panel
Connector Side Panel
Power side
status LED
Connector side
status LED
Green – OK
Yellow – a fault in the
system
Red – Major Fault or
Fatal Error
Led Color shows the
switch status
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
15
The system status indicators should display as follows:
when the switch is plugged in, within five minutes the STATUS LED should light up
green
the FAN LED should light up green. If the FAN LED shows red, troubleshoot the fan
module.
See “Troubleshooting” on page 52.
Table 3 shows the system status LED assignment.
As long as there is power to the switch (one PS unit is connected), and the switch is
booted up and running, the status LED should be green.
If the STATUS LED shows red after five minutes, unplug the switch and call your
Mellanox representative for assistance.
If the switch shuts down due to over temperature, unplug the switch, wait 5 minutes
and replug in the switch. For more information See “Troubleshooting” on page 52.
Table 3 - System Status LED Assignments
LED Configuration STATUS/ System Health LED
Solid Green OK – The system is up and running.
Flashing Green The system is booting up. This assignment is valid on managed systems only.
Solid Yellow To be implemented in a future SW release.
Solid Red Major Error – Possible damage can result to the switch. Turn off immediately.
e.g. bad firmware, can’t boot, overheated. If the system is booting up, it can take
up to 5 minutes for the status led to change to green.
Off Off – The system has no power.
RST
UID
PS
Basic OperationRev 1.1
Mellanox Technologies
16
3.2.2 Fan Status Indicators
The indicator is located to the left of the connectors on the connector side panel.
Figure 5: Fan Status LED Connector Side
The LED indicator on the Fan Module is on the right side of the module.
Figure 6: Fan Status LED Power side
Table 4 shows the fan status LED assignment.
Table 4 - Fan Status LED Assignments
LED Configuration FAN LED
Solid Green OK – All fans are up and running.
Solid Red Error – One or more fans is not operating properly. The system should be powered
down and troubleshoot the fan module.
Off Off – The fan unit is not receiving any power. Check that the fan unit is properly
and completely inserted.
RST
UID
PS
!
This is a two color LED
Fan air flow direction
indicator
UID
RST
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
17
3.2.3 Power Supply Status Indicators
The indicator is located to the left of the connectors on the connector side panel.
Figure 7: Power Status LED Connector Side
Figure 8: Power Side Panel
Each PS unit has a single 2 color LED on the right side of the PS unit, that indicates the internal
status of the unit.
All fans must be operating while the power supply is plugged in.
If the switch shuts down due to over temperature, unplug the switch, wait 5 minutes
and replug in the switch. For more information See “Troubleshooting” on page 52.
RST
UID
PS
These are two color LEDs
Green — OK
Red — Major fault
Off — No power
!
OK
!
OK
!
OK
!
OK
Mellanox
Primary Power Supply Unit
Fan Units
PS unit 1 status LED
Fan status LEDs
Basic OperationRev 1.1
Mellanox Technologies
18
Figure 9: PS Unit Status LEDs
Table 5 shows the power supply unit status LED assignment.
3.2.3.1 Bad Port LED
Figure 10: Bad Port LED
The Bad Port indicator is located on the left side of the connector side panel of the unit. Table 6
shows the fan status LED assignment.
Table 5 - PS Unit Status LED Assignments
LED Color Status
Solid Green OK – The Power supply is delivering the correct voltage – 12VDC.
Solid Red Error – The PS unit is not operational.
Off Off – There is no power to the system (neither PS unit is receiving power). If one
PS unit is showing green and the second PS unit is unplugged it will show a red
indication.
This is a two color LED
Power supply fan air flow
direction indicator
RST
UID
PS
This is a one color LED
Flashing Orange — received symbol error
SwitchX®-2 60 Port Ethernet Switch System Hardware User Manual Rev 1.1
Mellanox Technologies
19
This LED lights up when one or more ports is receiving a symbol error. The LED immediately
goes off until the next symbol error is received.
3.2.3.2 UID LED Switch Identifier
The UID LED is a debug feature that lights a blue LED on the switch connector side panel for
ease in finding a particular switch.
3.2.3.3 Port Connector LED Indicators
The SFP+ ports come three rows to a column. The LEDs refer to the ports according to the color
coding in
Figure 11.
Figure 11: Port LED to Port Location Assignment
Table 6 - Bad Port LED Assignments
LED Configuration Description
Off OK – No symbol errors have been received in last few seconds.
Flashing Orange Error – One or possibly more ports have just received a symbol error
Possible causes are:
Bad cable
Bad connection
Bad connector
43
44
45
46
47
48
49
50
50
49
43 44 45 47 4846
Basic OperationRev 1.1
Mellanox Technologies
20
Figure 12: Port LEDs
3.3 Air Flow
These switches can come with two air flow patterns. The two patterns are:
Connector side inlet to power side outlet
Power side inlet to connector side outlet
The air flow is specified in the product model number. See “Mellanox Part Numbering Legend”
on page 10.On the PS unit and fan modules the air flow direction can be seen on the power side
panel. All components need to have the same air flow. A mismatch in the air flow will affect the
heat dissipation.
Figure 13 shows a mismatch in component airflow.
Table 7 - Port Connector Physical and Logical Link Assignments for Ethernet
LED Configuration LED Description
Off Physical link is down / Default.
Solid Green Physical link is up with no traffic.
Flashing Green Physical link is up with traffic.
Flashing Orange Physical errors.
All PS units must have the same air flow as the fan modules.
5352515049
Port LEDs
  • Page 1 1
  • Page 2 2
  • Page 3 3
  • Page 4 4
  • Page 5 5
  • Page 6 6
  • Page 7 7
  • Page 8 8
  • Page 9 9
  • Page 10 10
  • Page 11 11
  • Page 12 12
  • Page 13 13
  • Page 14 14
  • Page 15 15
  • Page 16 16
  • Page 17 17
  • Page 18 18
  • Page 19 19
  • Page 20 20
  • Page 21 21
  • Page 22 22
  • Page 23 23
  • Page 24 24
  • Page 25 25
  • Page 26 26
  • Page 27 27
  • Page 28 28
  • Page 29 29
  • Page 30 30
  • Page 31 31
  • Page 32 32
  • Page 33 33
  • Page 34 34
  • Page 35 35
  • Page 36 36
  • Page 37 37
  • Page 38 38
  • Page 39 39
  • Page 40 40
  • Page 41 41
  • Page 42 42
  • Page 43 43
  • Page 44 44
  • Page 45 45
  • Page 46 46
  • Page 47 47
  • Page 48 48
  • Page 49 49
  • Page 50 50
  • Page 51 51
  • Page 52 52
  • Page 53 53
  • Page 54 54
  • Page 55 55
  • Page 56 56
  • Page 57 57
  • Page 58 58
  • Page 59 59
  • Page 60 60
  • Page 61 61
  • Page 62 62
  • Page 63 63
  • Page 64 64
  • Page 65 65
  • Page 66 66
  • Page 67 67
  • Page 68 68
  • Page 69 69
  • Page 70 70
  • Page 71 71
  • Page 72 72
  • Page 73 73
  • Page 74 74

Mellanox Technologies SwitchX-2 MSX1024B-2BFS User manual

Category
Network switches
Type
User manual

Ask a question and I''ll find the answer in the document

Finding information in a document is now easier with AI