Broadcom Brocade Fabric OS Extension Configuration 8.1.0 User guide

Type
User guide
Supporting Fabric OS 8.1.0
CONFIGURATION GUIDE
Brocade Fabric OS Extension
Conguration Guide, 8.1.0
53-1004390-03
28 April 2017
©
2017, Brocade Communications Systems, Inc. All Rights Reserved.
Brocade, the B-wing symbol, and MyBrocade are registered trademarks of Brocade Communications Systems, Inc., in the United States and in other
countries. Other brands, product names, or service names mentioned of Brocade Communications Systems, Inc. are listed at www.brocade.com/en/legal/
brocade-Legal-intellectual-property/brocade-legal-trademarks.html. Other marks may belong to third parties.
Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment,
equipment feature, or service oered or to be oered by Brocade. Brocade reserves the right to make changes to this document at any time, without
notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade
sales oce for information on feature and product availability. Export of technical data contained in this document may require an export license from the
United States government.
The authors and Brocade Communications Systems, Inc. assume no liability or responsibility to any person or entity with respect to the accuracy of this
document or any loss, cost, liability, or damages arising from the information contained herein or the computer programs that accompany it.
The product described by this document may contain open source software covered by the GNU General Public License or other open source license
agreements. To nd out which open source software is included in Brocade products, view the licensing terms applicable to the open source software, and
obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.
Brocade Fabric OS Extension Conguration Guide, 8.1.0
2 53-1004390-03
Contents
Preface...................................................................................................................................................................................................................................9
Document conventions............................................................................................................................................................................................................................9
Notes, cautions, and warnings.....................................................................................................................................................................................................9
Text formatting conventions.........................................................................................................................................................................................................9
Command syntax conventions.................................................................................................................................................................................................10
Brocade resources..................................................................................................................................................................................................................................10
Document feedback.............................................................................................................................................................................................................................. 10
Contacting Brocade Technical Support......................................................................................................................................................................................... 11
Brocade customers.......................................................................................................................................................................................................................11
Brocade OEM customers.......................................................................................................................................................................................................... 11
About this Document.......................................................................................................................................................................................................13
Supported hardware and software...................................................................................................................................................................................................13
What's new in this document............................................................................................................................................................................................................. 13
Changes made for the initial release ( 53-1004390-01)...........................................................................................................................................13
Changes made for this release ( 53-1004390-02)..................................................................................................................................................... 14
Changes made for this release ( 53-1004390-03)..................................................................................................................................................... 14
Extension Concepts and Features................................................................................................................................................................................ 15
Brocade Extension concepts..............................................................................................................................................................................................................15
Extension trunks, tunnels, circuits, and interfaces......................................................................................................................................................................16
VE_Ports and VEX_Ports................................................................................................................................................................................................................... 18
Gigabit Ethernet interfaces..................................................................................................................................................................................................................19
Ethernet interfaces..................................................................................................................................................................................................................................19
Virtual circuits and tunnels...................................................................................................................................................................................................................19
FCIP Extension........................................................................................................................................................................................................................................ 20
IP Extension...............................................................................................................................................................................................................................................21
IP Extension and IP networking...............................................................................................................................................................................................21
Extension Trunking................................................................................................................................................................................................................................. 22
Redundancy and fault tolerance..............................................................................................................................................................................................23
Considerations for multiple tunnel use with protocol optimization ..........................................................................................................................24
IP WAN network considerations....................................................................................................................................................................................................... 24
IP LAN network considerations........................................................................................................................................................................................................ 25
Extension Hot Code Load................................................................................................................................................................................................................... 25
Extension HCL operation...........................................................................................................................................................................................................25
Extension HCL limitations and considerations..................................................................................................................................................................27
Fibre Channel Storage Area Network considerations..............................................................................................................................................................28
Adaptive Rate Limiting..........................................................................................................................................................................................................................29
Brocade 7840 switch and Brocade SX6 blade support for ARL.............................................................................................................................29
Brocade FX8-24 support for ARL........................................................................................................................................................................................ 30
FSPF link cost calculation when ARL is used................................................................................................................................................................... 30
ARL considerations.......................................................................................................................................................................................................................30
Compression options............................................................................................................................................................................................................................ 31
Compression options for the Brocade 7840 Extension Switch and Brocade SX6 Extension Blade....................................................... 31
Compression options for the Brocade FX8-24 Extension Blade............................................................................................................................ 32
FastWrite and Open Systems Tape Pipelining............................................................................................................................................................................32
FastWrite and OSTP network congurations.....................................................................................................................................................................32
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 3
FICON Acceleration...............................................................................................................................................................................................................................34
VM Insight..................................................................................................................................................................................................................................................34
IP security...................................................................................................................................................................................................................................................35
IPsec for the extension switches and blades......................................................................................................................................................................36
Limitations using IPsec over tunnels.....................................................................................................................................................................................36
IPv6 addressing.......................................................................................................................................................................................................................................37
IPv6 with embedded IPv4 addresses...................................................................................................................................................................................37
Memory use limitations for large-device tunnel congurations.......................................................................................................................................... 38
Control blocks created during FCP trac ow..................................................................................................................................................................40
Control blocks created during FICON trac ow............................................................................................................................................................ 40
Considerations for tunnel control block memory and device conguration..........................................................................................................41
Firmware downloads..............................................................................................................................................................................................................................42
Extension Platforms and Features................................................................................................................................................................................43
Extension platforms and features overview................................................................................................................................................................................. 43
Brocade 7840 Extension Switch and Brocade SX6 Extension Blade overview......................................................................................................... 45
Brocade 7840 Extension Switch ports................................................................................................................................................................................46
Brocade SX6 Extension Blade ports.....................................................................................................................................................................................47
Ethernet port groups.................................................................................................................................................................................................................... 47
Fibre Channel port groups.........................................................................................................................................................................................................48
Network DP components...........................................................................................................................................................................................................49
10VE and 20VE port distribution.......................................................................................................................................................................................... 52
10 GbE and 40 GbE port and circuit considerations ....................................................................................................................................................53
Brocade 7840 license options................................................................................................................................................................................................53
Brocade SX6 license options....................................................................................................................................................................................................53
Brocade FX8-24 Extension Blade overview.............................................................................................................................................................................. 54
Brocade FX8-24 operating modes.......................................................................................................................................................................................54
Brocade FX8-24 data processor complexes....................................................................................................................................................................54
Removing the Brocade FX8-24 Extension Blade.......................................................................................................................................................... 55
Brocade FX8-24 License options......................................................................................................................................................................................... 55
Brocade FX8-24 multigigabit circuits.................................................................................................................................................................................. 56
Crossports and failover............................................................................................................................................................................................................... 56
Bandwidth allocation and restrictions....................................................................................................................................................................................56
Tunnel and circuit requirements for Brocade Extension platforms.................................................................................................................................... 59
Brocade 7840 and SX6 requirements.................................................................................................................................................................................60
Brocade FX8-24 requirements...............................................................................................................................................................................................61
Brocade IP Extension............................................................................................................................................................................................................................62
Tunnels and hybrid mode...........................................................................................................................................................................................................63
Out-of-order delivery on a tunnel...........................................................................................................................................................................................63
IP Extension and IP on the LAN..............................................................................................................................................................................................63
IP Extension and Trac Control Lists .................................................................................................................................................................................. 63
IP Extension and QoS................................................................................................................................................................................................................. 68
IP Extension and compression................................................................................................................................................................................................ 68
IP Extension and IP LAN deployment..................................................................................................................................................................................69
IP Extension limitations and considerations....................................................................................................................................................................... 71
Extension Hot Code Load for Brocade 7840 and Brocade SX6...................................................................................................................................... 71
Path Maximum Transmission Unit discovery..............................................................................................................................................................................71
Circuit failover........................................................................................................................................................................................................................................... 72
Circuit failover grouping.............................................................................................................................................................................................................. 73
Failover in TI zones....................................................................................................................................................................................................................... 74
Bandwidth calculation during failover....................................................................................................................................................................................74
Brocade Fabric OS Extension Conguration Guide, 8.1.0
4 53-1004390-03
10 GbE lossless link loss (FX8-24 blade)..........................................................................................................................................................................75
Circuit spillover.........................................................................................................................................................................................................................................76
Understanding circuit spillover utilization.............................................................................................................................................................................76
Circuit spillover considerations.................................................................................................................................................................................................79
Service Level Agreement ....................................................................................................................................................................................................................79
Conguring Extension Features....................................................................................................................................................................................81
Conguration overview......................................................................................................................................................................................................................... 81
Conguration prerequisites.................................................................................................................................................................................................................82
Conguring platform modes .............................................................................................................................................................................................................83
Conguring FCIP or hybrid mode..........................................................................................................................................................................................83
Conguring VE mode.................................................................................................................................................................................................................. 84
Clearing SX6 blade conguration ..........................................................................................................................................................................................85
Conguring GbE mode on Brocade FX8-24 blade.......................................................................................................................................................85
Conguring VEX_Ports on the FX8-24..............................................................................................................................................................................86
Conguring ports.................................................................................................................................................................................................................................... 87
Conguring port speeds.............................................................................................................................................................................................................87
Conguring IP...........................................................................................................................................................................................................................................88
Conguring IPIF............................................................................................................................................................................................................................. 88
Conguring IP route......................................................................................................................................................................................................................90
Conguring VLAN..................................................................................................................................................................................................................................91
Verifying IP connectivity.......................................................................................................................................................................................................................92
Conguring Service Level Agreement .......................................................................................................................................................................................... 93
Conguring IPsec....................................................................................................................................................................................................................................95
Conguring IPsec on the Brocade 7840 and Brocade SX6...................................................................................................................................... 96
IPsec IKE authentication failures............................................................................................................................................................................................. 99
Conguring IPsec on the FX8-24...................................................................................................................................................................................... 100
Conguring Extension tunnels for FCIP.....................................................................................................................................................................................101
Conguring VE_Ports to persistently disabled.............................................................................................................................................................. 102
Conguring tunnels....................................................................................................................................................................................................................104
Conguring emulation features on tunnels......................................................................................................................................................................107
Conguring compression options....................................................................................................................................................................................... 107
Conguring WAN on tunnels.................................................................................................................................................................................................109
Conguring failover....................................................................................................................................................................................................................117
Conguring failover groups.................................................................................................................................................................................................... 118
Conguring spillover..................................................................................................................................................................................................................121
Conguring VE_Ports to persistently enabled...............................................................................................................................................................122
Verifying tunnel conguration................................................................................................................................................................................................124
Conguring Extension Hot Code Load.......................................................................................................................................................................................127
Conguring DP complexes and HCL tunnels................................................................................................................................................................ 130
Conguring IP Extension.................................................................................................................................................................................................................. 131
Conguration steps for IP Extension features................................................................................................................................................................ 132
Conguring hybrid mode for IP Extension features.....................................................................................................................................................134
Conguring IPIF for IP Extension........................................................................................................................................................................................ 134
Conguring WAN IP route for IP Extension.................................................................................................................................................................... 135
Conguring a tunnel to support IP Extension.................................................................................................................................................................137
Conguring bandwidth distribution..................................................................................................................................................................................... 138
Conguring LAN IP route for IP Extension and PBR..................................................................................................................................................139
Conguring tunnel compression..........................................................................................................................................................................................142
Conguring a tunnel and circuits for IP Extension........................................................................................................................................................143
Conguring Ethernet interfaces (GbE port) for IP Extension LAN features....................................................................................................... 144
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 5
Conguring LAG.........................................................................................................................................................................................................................145
Conguring a LAN gateway (SVI) for IP Extension...................................................................................................................................................... 148
Conguring Trac Control Lists for IP Extension ........................................................................................................................................................150
Example IP Extension Conguration..................................................................................................................................................................................156
Conguring Brocade FX8-24 crossport features..................................................................................................................................................................161
Conguring crossports on the Brocade FX8-24..........................................................................................................................................................161
Crossports and failover............................................................................................................................................................................................................ 162
Conguring IP routes with crossports................................................................................................................................................................................164
Conguring VLAN tags with crossports........................................................................................................................................................................... 165
Display VLAN tag conguration using the portshow vlantag command............................................................................................................165
Using ping with crossports..................................................................................................................................................................................................... 165
Using traceroute with crossports..........................................................................................................................................................................................165
Using logical switches........................................................................................................................................................................................................................166
Logical switch overview .......................................................................................................................................................................................................... 166
Considerations for logical switches.....................................................................................................................................................................................168
Trac Isolation Zoning.......................................................................................................................................................................................................................177
Zoning.......................................................................................................................................................................................................................................................177
Troubleshooting Tools.................................................................................................................................................................................................. 179
In-band management........................................................................................................................................................................................................................ 179
IP routing........................................................................................................................................................................................................................................179
Conguring IP addresses and routes.................................................................................................................................................................................180
VLAN tagging support.............................................................................................................................................................................................................184
IP forwarding support............................................................................................................................................................................................................... 184
WAN analysis tools..............................................................................................................................................................................................................................186
The tperf option...........................................................................................................................................................................................................................186
Using ping to test a connection............................................................................................................................................................................................ 187
Using traceroute..........................................................................................................................................................................................................................188
Using WAN Tool...................................................................................................................................................................................................................................188
WAN Tool commands.............................................................................................................................................................................................................. 189
Conguring a WAN Tool session and displaying results............................................................................................................................................190
Resolving test session problems......................................................................................................................................................................................... 196
Using the portshow command.......................................................................................................................................................................................................196
Displaying IP interfaces............................................................................................................................................................................................................196
Displaying IP routes...................................................................................................................................................................................................................196
Displaying switch mode information with the extncfg command.......................................................................................................................... 197
Displaying GbE port information with the portcfgge command.............................................................................................................................197
Listing the MAC addresses of LAN and GE ports....................................................................................................................................................... 197
Displaying LAG information...................................................................................................................................................................................................198
Displaying tunnel HCL information.....................................................................................................................................................................................198
Displaying TCL information....................................................................................................................................................................................................199
Displaying IP Extension LAN statistics..............................................................................................................................................................................199
Displaying performance statistics........................................................................................................................................................................................201
Displaying QoS statistics.........................................................................................................................................................................................................201
Displaying details........................................................................................................................................................................................................................201
Filtering portshow display output ....................................................................................................................................................................................... 201
Displaying tunnel status...........................................................................................................................................................................................................202
Displaying tunnel information................................................................................................................................................................................................203
Displaying a tunnel with circuit information .................................................................................................................................................................... 203
Displaying tunnel performance ............................................................................................................................................................................................204
Displaying tunnel TCP statistics ..........................................................................................................................................................................................205
Brocade Fabric OS Extension Conguration Guide, 8.1.0
6 53-1004390-03
Displaying circuits.......................................................................................................................................................................................................................205
Displaying a single circuit........................................................................................................................................................................................................ 205
Displaying TCP statistics for circuits...................................................................................................................................................................................205
Displaying circuit performance ............................................................................................................................................................................................ 205
Displaying GbE port performance.......................................................................................................................................................................................206
Displaying QoS prioritization for a circuit......................................................................................................................................................................... 206
Displaying tunnel information (FX8-24 blade)...............................................................................................................................................................208
Tunnel issues ........................................................................................................................................................................................................................................ 208
Tunnel does not come online................................................................................................................................................................................................ 208
Tunnel goes online and oine...............................................................................................................................................................................................210
Troubleshooting Extension links....................................................................................................................................................................................................211
Gathering additional information..........................................................................................................................................................................................211
Using FTRACE..................................................................................................................................................................................................................................... 212
FTRACE conguration.............................................................................................................................................................................................................212
Changing conguration settings...........................................................................................................................................................................................214
Displaying FTRACE status on a DP complex................................................................................................................................................................ 216
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 7
Brocade Fabric OS Extension Conguration Guide, 8.1.0
8 53-1004390-03
Preface
Document conventions......................................................................................................................................................................................9
Brocade resources............................................................................................................................................................................................ 10
Document feedback.........................................................................................................................................................................................10
Contacting Brocade Technical Support....................................................................................................................................................11
Document conventions
The document conventions describe text formatting conventions, command syntax conventions, and important notice formats used in
Brocade technical documentation.
Notes, cautions, and warnings
Notes, cautions, and warning statements may be used in this document. They are listed in the order of increasing severity of potential
hazards.
NOTE
A Note provides a tip, guidance, or advice, emphasizes important information, or provides a reference to related information.
ATTENTION
An Attention statement indicates a stronger note, for example, to alert you when trac might be interrupted or the device might
reboot.
CAUTION
A Caution statement alerts you to situations that can be potentially hazardous to you or cause damage to hardware,
rmware, software, or data.
DANGER
A Danger statement indicates conditions or situations that can be potentially lethal or extremely hazardous to you. Safety
labels are also attached directly to products to warn of these conditions or situations.
Text formatting conventions
Text formatting conventions such as boldface, italic, or Courier font may be used to highlight specic words or phrases.
Format Description
bold text Identies command names.
Identies keywords and operands.
Identies the names of GUI elements.
Identies text to enter in the GUI.
italic text Identies emphasis.
Identies variables.
Identies document titles.
Courier font
Identies CLI output.
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 9
Format Description
Identies command syntax examples.
Command syntax conventions
Bold and italic text identify command syntax components. Delimiters and operators
dene groupings of parameters and their logical
relationships.
Convention Description
bold text Identies command names, keywords, and command options.
italic text Identies a variable.
value In Fibre Channel products, a xed value provided as input to a command option is printed in plain text, for
example, --show WWN.
[ ] Syntax components displayed within square brackets are optional.
Default responses to system prompts are enclosed in square brackets.
{ x | y | z } A choice of required parameters is enclosed in curly brackets separated by vertical bars. You must select
one of the options.
In Fibre Channel products, square brackets may be used instead for this purpose.
x | y A vertical bar separates mutually exclusive elements.
< > Nonprinting characters, for example, passwords, are enclosed in angle brackets.
... Repeat the previous element, for example, member[member...].
\ Indicates a “soft” line break in command examples. If a backslash separates two lines of a command
input, enter the entire command at the prompt without the backslash.
Brocade resources
Visit the Brocade website to locate related documentation for your product and additional Brocade resources.
White papers, data sheets, and the most recent versions of Brocade software and hardware manuals are available at www.brocade.com.
Product documentation for all supported releases is available to registered users at MyBrocade.
Click the Support tab and select Document Library to access product documentation on MyBrocade or www.brocade.com. You can
locate documentation by product or by operating system.
Release notes are bundled with software downloads on MyBrocade. Links to software downloads are available on the MyBrocade landing
page and in the Document Library.
Document feedback
Quality is our
rst concern at Brocade, and we have made every eort to ensure the accuracy and completeness of this document.
However, if you nd an error or an omission, or you think that a topic needs further development, we want to hear from you. You can
provide feedback in two ways:
Through the online feedback form in the HTML documents posted on www.brocade.com
By sending your feedback to documentation@brocade.com
Provide the publication title, part number, and as much detail as possible, including the topic heading and page number if applicable, as
well as your suggestions for improvement.
Brocade resources
Brocade Fabric OS Extension Conguration Guide, 8.1.0
10 53-1004390-03
Contacting Brocade Technical Support
As a Brocade customer, you can contact Brocade Technical Support 24x7 online or by telephone. Brocade OEM customers should
contact their OEM/solution provider.
Brocade customers
For product support information and the latest information on contacting the Technical Assistance Center, go to www.brocade.com and
select Support.
If you have purchased Brocade product support directly from Brocade, use one of the following methods to contact the Brocade
Technical Assistance Center 24x7.
Online Telephone
Preferred method of contact for non-urgent issues:
Case management through the MyBrocade portal.
Quick Access links to Knowledge Base, Community, Document
Library, Software Downloads and Licensing tools
Required for Sev 1-Critical and Sev 2-High issues:
Continental US: 1-800-752-8061
Europe, Middle East, Africa, and Asia Pacic: +800-AT FIBREE
(+800 28 34 27 33)
Toll-free numbers are available in many countries.
For areas unable to access a toll-free number:
+1-408-333-6061
Brocade OEM customers
If you have purchased Brocade product support from a Brocade OEM/solution provider, contact your OEM/solution provider for all of
your product support needs.
OEM/solution providers are trained and
certied by Brocade to support Brocade
®
products.
Brocade provides backline support for issues that cannot be resolved by the OEM/solution provider.
Brocade Supplemental Support augments your existing OEM support contract, providing direct access to Brocade expertise.
For more information, contact Brocade or your OEM.
For questions regarding service levels and response times, contact your OEM/solution provider.
Contacting Brocade Technical Support
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 11
Brocade Fabric OS Extension Conguration Guide, 8.1.0
12 53-1004390-03
About this Document
Supported hardware and software..............................................................................................................................................................13
What's new in this document........................................................................................................................................................................13
Supported hardware and software
The following hardware platforms support Brocade Extension (Fibre Channel over IP features and IP Extension features) as described in
this manual:
Brocade DCX 8510-4, and DCX 8510-8 with one or more FX8-24 blades
Brocade 7840 switch
Brocade X6-4 Director and Brocade X6-8 Director with one or more SX6 Extension blades
The following software is supported as described in this manual:
Brocade Fabric OS 8.1.0 software
What's new in this document
Changes made for the initial release ( 53-1004390-01)
New additions or deletions in this document support Fabric OS 8.1.0.
Major sections of this publication aected by additions and corrections include the following:
Extension Concepts and Features
The chapter has been modied for additional clarity and presentation of the Brocade Extension features.
The chapter contains additional information about extension interfaces, circuits, and tunnels.
FCIP Extension on page 20 was added.
IP LAN network considerations on page 25 was added.
Extension Hot Code Load on page 25 was expanded.
Fibre Channel Storage Area Network considerations on page 28FC SAN considerations was added.
FICON Acceleration on page 34 was added.
VM Insight on page 34 was added.
IP security on page 35 was expanded and information about Suite B Cryptographic Suites was added.
Extension Features on Brocade Extension Switches and Blades
The chapter has been modied for additional clarity and presentation of the Brocade Extension platforms and features.
Brocade 7840 Extension Switch and Brocade SX6 Extension Blade overview on page 45 provides information about
these two related platforms.
Fibre Channel port groups on page 48 was added.
Tunnel and circuit requirements for Brocade Extension platforms on page 59 was updated and reorganized.
Conguring Extension Features
This chapter has been extensively reorganized and updated so that the presentation of information more closely matches
conguration work-ow. Additional examples are provided and command results are upated.
Conguring IPsec on page 95 was updated.
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 13
Conguring IP Extension on page 131 contains information that previously existed in a separate chapter.
Conguring app-type for TCL on page 156 was added.
Conguring Brocade FX8-24 crossport features on page 161 was added. It consolidates information about crossport
conguration.
Troubleshooting Tools
This chapter has been updated for Fabric OS 8.1.0 with modications to command examples and outputs.
Conguring a WAN Tool session and displaying results on page 190 was updated.
Filtering portshow display output on page 201 was updated.
Changes made for this release ( 53-1004390-02)
Additions and corrections to the document include the following:
Minor editorial corrections have been made throughout this document.
In VE_Ports and VEX_Ports on page 18, the table header was corrected.
In VM Insight on page 34, instances of VMOSAN were changed to VM Insight.
In Fibre Channel port groups on page 48, corrected information about Brocade 7840 port groups.
Changes made for this release ( 53-1004390-03)
Additions and corrections to the document include the following:
In Using WAN Tool on page 188, WAN Tool commands on page 189, and Conguring a WAN Tool session and displaying
results on page 190, several updates were made for CLI operation and command output. The number of congured WAN Tool
sessions per DP was changed from 8 to 10 and the maximum number of sessions per slot is 16.
What's new in this document
Brocade Fabric OS Extension Conguration Guide, 8.1.0
14 53-1004390-03
Extension Concepts and Features
Brocade Extension concepts........................................................................................................................................................................15
Extension trunks, tunnels, circuits, and interfaces................................................................................................................................16
VE_Ports and VEX_Ports..............................................................................................................................................................................18
Gigabit Ethernet interfaces.............................................................................................................................................................................19
Ethernet interfaces.............................................................................................................................................................................................19
Virtual circuits and tunnels............................................................................................................................................................................. 19
FCIP Extension...................................................................................................................................................................................................20
IP Extension......................................................................................................................................................................................................... 21
Extension Trunking............................................................................................................................................................................................22
IP WAN network considerations..................................................................................................................................................................24
IP LAN network considerations...................................................................................................................................................................25
Extension Hot Code Load..............................................................................................................................................................................25
Fibre Channel Storage Area Network considerations.........................................................................................................................28
Adaptive Rate Limiting.................................................................................................................................................................................... 29
Compression options.......................................................................................................................................................................................31
FastWrite and Open Systems Tape Pipelining...................................................................................................................................... 32
FICON Acceleration..........................................................................................................................................................................................34
VM Insight.............................................................................................................................................................................................................34
IP security............................................................................................................................................................................................................. 35
IPv6 addressing................................................................................................................................................................................................. 37
Memory use limitations for large-device tunnel
congurations.....................................................................................................38
Firmware downloads........................................................................................................................................................................................42
Brocade Extension concepts
Brocade extension switches and extension blades for Brocade director families provide fast and reliable network infrastructure to address
the requirements of Storage Area Networks (SANs) that are extended beyond the traditional reaches of Fibre Channel (FC)
communications. Without some form of distance extension, the distance between the source and the destination in a SAN is limited to a
few kilometers.
Brocade extension products support both FC/FICON-based data
ows and IP-based storage data ows. Brocade extension solutions
maximize replication and backup throughput over distance, using data compression, disk and tape protocol acceleration, and WAN-
optimized TCP. Brocade extension supports applications such as remote data replication (RDR), centralized backup, and data migration.
Brocade extension uses the existing IP wide area network (WAN) infrastructure to connect Fibre Channel and IP fabrics between distant
endpoints that are either impractical or costly using native Fibre Channel or IP connections. The basis of the connection is the extension
tunnel, built on a physical connection between two extension switches or blades. Extension tunnels allow Fibre Channel and IP trac to
pass through the IP WAN. The extension tunnel connections ensure lossless transmission, and that FC and IP frames are delivered in the
correct order. The Fibre Channel fabric and all targets and initiators, whether FC or IP, are unaware of the presence of the IP WAN.
The extension tunnel provides load balancing across separate network paths, optimization for extended links, rate-limiting to ensure
optimal performance, and lossless link loss (LLL) recovery.
The two Brocade protocol technologies implemented in Brocade extension products are FCIP and IP Extension.
FCIP, or Fiber Channel over IP, is a tunneling protocol to link Fibre Channel over distance on standard IP networks. Used
primarily for remote replication, backup, and storage access, FCIP provides Fibre Channel connectivity over IP networks
between Fibre Channel devices or fabrics. The FCIP link is an inter-switch link (ISL) that transports FC control and data frames
between switches. The following table outlines FCIP.
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 15
Network WAN/MAN
Transport FCIP/TCP/IP/Ethernet
Encapsulation Brocade encapsulates Fibre Channel data sequences into compressed
batches. Those batches ll TCP segments to their maximum size and then
form IP datagrams.
IP-routable Yes
Brocade IP Extension is used primarily for IP storage applications, such as remote host-based or database-based replication,
NAS replication, IP backups, and tape grids. IP Extension uses the same VE_Ports and circuits that FCIP uses, or it can use its
own. The following table outlines IP Extension.
Network WAN/MAN
Transport IP Extension/TCP/IP/Ethernet
Encapsulation Brocade encapsulates IP ow (also called “streams”) data sequences into
compressed batches. Those batches ll TCP segments to their maximum
size and then form IP datagrams.
IP-routable Yes
For additional information about implementation and technical details of Brocade extension technology, refer to the Brocade white paper,
Extension Trunking.
Extension trunks, tunnels, circuits, and interfaces
An extension tunnel is a conduit that contains one or more circuits. When a tunnel contains multiple circuits, it is also called an extension
trunk because multiple circuits are trunked together. An extension tunnel, or extension trunk, is a single Inter-Switch Link (ISL). Circuits are
individual extension connections within the trunk, each with its own unique source and destination IP interface (IPIF) address.
To understand tunnels, you must understand the relationship between tunnels and VE_Ports. Because an extension tunnel is an ISL,
each end of the tunnel requires its own Virtual E_Port (VE_Port). For example, the Brocade SX6 extension blade supports a number of
VE_Ports. The available VE_Ports on the Brocade SX6 are numbered 16 to 35. When you create a tunnel, that tunnel can be numbered
any of 16 to 35. Tunnels are more complicated than this, but the point is that each end of the tunnel is
identied by number, and that
number is directly associated with a VE_Port on the extension platform. Tunnels are frequently created between dierent VE_Ports, so
the tunnel number can be dierent at each end, from a conguration point of view. Each extension platform, such as Brocade SX6
extension blade, Brocade 7840 extension switch, and Brocade FX8-24 extension blade, has a dierent numbering scheme for its
VE_Ports.
Circuits exist within tunnels. A circuit is a connection between a pair of IP addresses that are dened within an extension tunnel. Circuits
provide the links for trac ow between source and destination interfaces that are located on either end of the tunnel. Each tunnel can
contain a single circuit or multiple circuits.
NOTE
In this publication, the "source" or "local" is the switch you are conguring, while the "destination" or "remote" is the switch on the
other end of the tunnel or circuit. Local and remote will depend on which side of the of the tunnel you are conguring.
You must congure unique IPIFs as the local and remote endpoints of each circuit. On the local side, a circuit is congured with a source
IPIF address and a destination address. On the remote side of the circuit, its source IPIF address is the same as the local-side destination
address. Similarly, on the remote side of the circuit its IPIF destination address points to the local-side source address. Multiple IPIFs can
be congured on each physical Ethernet interface.
Extension trunks, tunnels, circuits, and interfaces
Brocade Fabric OS Extension Conguration Guide, 8.1.0
16 53-1004390-03
NOTE
On the Brocade FX8-24, the IP address (or IPIF) for each local and remote address must be individually unique. However, on
the Brocade 7840 and Brocade SX6, each address pair must be unique. For example, the following address pairs use the
same source address in each pair but the destination addresses are dierent. For the FX8-24, these addresses are not unique.
For the SX6 and 7840, the address pairs are unique.
--local-ip 10.0.1.10 --remote-ip 10.1.1.10
--local-ip 10.0.1.10 --remote-ip 10.1.1.11
The circuit conguration parameters on the local and remote sides of the circuits and tunnel must match in addition to the source and
destination IPIF addresses pointing to each other. For example, if you congure IPsec on a tunnel, each end of the tunnel must be
congured to use the same IPsec parameters. Other parameters for each circuit must match, such as MTU size, bandwidth allocations,
QoS, VLAN ID, and keepalive values.
You must congure a gateway IP route for the circuit to the destination network when the remote IPIF is not on the same subnet as the
local IPIF. You can dene a specic number of routes per IPIF based on the extension platform. Refer to Tunnel and circuit requirements
for Brocade Extension platforms on page 59 for specications.
ATTENTION
When using Brocade IP Extension, the local and remote LAN subnet addresses must be
dierent.
The following gure shows an example of two extension tunnels. The rst tunnel is a trunk of four circuits, and the second tunnel is a
trunk of two circuits. Each circuit is assigned a unique IPIF address. Those IPIFs are, in turn, assigned to Ethernet interfaces. In the gure,
each IPIF is assigned to a dierent Ethernet interface. This is not required. Ethernet interface assignment is exible depending on the
environment's needs and assignments can be made as desired. For instance, multiple IP interfaces can be assigned to a single Ethernet
interface. The circuit ows from IP interface to IP interface through the assigned Ethernet interfaces.
FIGURE 1 Extension tunnel and circuits
Extension trunks, tunnels, circuits, and interfaces
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 17
For specications and restrictions on tunnels, circuits, and trunks for the 7840 Extension Switch, the SX6 Extension Blade, and the
FX8-24 Extension Blade, refer to the Extension Platforms and Features chapter of this publication.
VE_Ports and VEX_Ports
All Fibre Channel communications between switches that include fabric services must communicate across an Inter-Switch Link (ISL).
Because an extension tunnel is an ISL, each end requires its own E_Port. There are various types of E_Ports:
E_Port
EX_Port
VE_Port (virtual E_Port)
VEX_Port (virtual EX_Port)
VE_Ports and VEX_ports are virtual because they face the extension tunnel and thus enable communication across an extension tunnel.
Extension trunking — multiple circuits in a tunnel — operates the same whether the virtual port is a VE_Port or a VEX_Port.
NOTE
VEX_Ports are supported only on the Brocade FX8-24 extension blade. The Brocade 7840 switch and the Brocade SX6
blade do not support VEX_Ports.
EX and VEX ports are FC Routed (FCR) ports. Router ports are the demarcation point of fabric services for a fabric. Fabric services do
not extend beyond an EX or VEX port. Remote edge fabrics are edge fabrics connected through a WAN connection and VEX_Port.
TABLE 3 VE_Ports and VEX_Ports
E port types No FCR FCR
Native FC E_Port EX_Port
Extended over tunnel VE_Port VEX_Port (on FX8-24 only)
Because an extension trunk is logically a single tunnel, only a single VE_Port or VEX_Port is used for each end of the tunnel, even
though more than one circuit can be contained within the tunnel.
Once the tunnels are congured and the WAN-optimized TCP (WO-TCP) connections are made for a circuit, a logical Inter-Switch Link is
established between the switches. VE_Ports operate like E_Ports for all fabric services and Fabric OS operations, except that VE_Ports
use TCP/IP and Ethernet as the transport instead of FC.
A VEX_Port exposed by the extension tunnel to form an ISL connection allows you to congure a VEX_Port to support FCR
demarcation. From the point of view of a switch in an edge fabric, a VEX_Port appears as a normal E_Port. It follows the same Fibre
Channel protocol as other E_Ports. However, VEX_Ports terminate the attached fabric at the port and do not allow fabrics to merge by
propagating fabric services or routing topology information beyond that edge fabric. This provides edge fabric or remote edge fabric
isolation outward from the EX_Port or VEX_Port.
NOTE
VE_Ports or VEX_Ports cannot connect in parallel to the same domain at the same time as Fibre Channel E_Ports or
EX_Ports.
An extension tunnel is assigned to a VE_Port or VEX_Port on the switch or blade at each end of the tunnel. Because multiple VE_Ports
are supported on the extension switch or blade, you can create multiple tunnels on a switch or blade.
Fibre Channel frames enter an extension tunnel through the VE_Ports (or VEX_Ports) and are encapsulated and passed to TCP layer
connections. A Data Processing (DP) complex on the switch or blade handles the FC frame encapsulation, de-encapsulation, and
transmission to the TCP link.
VE_Ports and VEX_Ports
Brocade Fabric OS Extension Conguration Guide, 8.1.0
18 53-1004390-03
Gigabit Ethernet interfaces
On the Brocade extension platforms, the GbE interfaces are abstracted from the VE/VEX_Ports, IP interfaces, and extension tunnels and
trunks. This means that the GbE interfaces are not xed to the VE/VEX_Ports, IP interfaces, or extension tunnels and trunks. Depending
on the platform, you have access to a combination of 1 GbE interfaces, 10 GbE interfaces, and 40 GbE interfaces.
A GbE interface can be congured for WAN operations or LAN operations on platforms that support Brocade IP Extension. A GbE
interface is used for WAN operation when it functions as the endpoint of a circuit. The interface is WAN-facing, and you are creating the
tunnels and circuits. WAN is the only allowed GbE interface mode when a platform is congured for FCIP-only operation.
When a platform is congured for hybrid mode, which supports both FCIP and IP Extension, you congure the LAN-facing GbE ports
for LAN mode operations. When GbE ports are in LAN mode, they can be members of a static Link Aggregation Group (LAG).
Ethernet interfaces
Ethernet interfaces are assigned by associating them with IP interfaces (IPIFs). IP interfaces on the extension platforms are virtual entities
within the platform. The IPIFs, via their IP addresses, are assigned to circuits by designating the source IP address. The circuit is grouped
into an extension trunk by designating the VE_Port (or VEX_Port) to which it belongs. Tunnels/trunks are
identied by their VE_Port
number. If an extension trunk already exists, you can create an additional circuit and include it in the tunnel.
Each Ethernet interface on the Brocade SX6, Brocade 7840, or Brocade FX8-24 can be used by circuits from multiple VE/VEX_Ports.
Each VE/VEX_Port can use multiple Ethernet ports. Each VE/VEX_Port can have multiple IP interfaces, or circuits. Each IP interface or
circuit in a VE/VEX_Port is associated with an IP address. In turn, those IP addresses are assigned to
specic Ethernet interfaces.
For Brocade Extension Trunking, the data processor (DP) that owns the VE_Port controls all the member circuits. There is no distributed
processing, load sharing or LLL across DPs. Failover between DPs is done at the FC level by the Brocade FC switching ASIC, provided
the conguration permits it. The only components that can be shared are the Ethernet interfaces.
The IP network routes the circuits over separate network paths and WAN links based on the destination IP addresses and possibly other
Layer 2 and Layer 3 header attributes. Ethernet interfaces on the Brocade SX6, Brocade 7840, and Brocade FX8-24 provide a single
convenient connection to the data center LAN for one to many circuits.
Virtual circuits and tunnels
When a tunnel is created, a base virtual tunnel is created for that tunnel. A virtual tunnel is the actual object that carries the data ows. An
FCIP virtual tunnel itself consists of four tunnels:
Control tunnel, or F-Class. The control tunnel is higher priority and has no guaranteed minimum bandwidth. It can use all
available bandwidth up to the maximum congured bandwidth rate.
High data tunnel. The high data tunnel is for high-priority QoS data.
Medium data tunnel. The medium data tunnel is for medium-priority QoS data.
Low data tunnel. The low data tunnel is for low-priority QoS data.
Data tunnels adhere to minimum bandwidth congurations and compete with other data tunnels up to the maximum rate.
A virtual tunnel includes all the circuits that are congured within the tunnel. The circuits in the virtual tunnel are called virtual circuits.
When an extension platform is congured to operate in hybrid mode, which supports both FCIP extension and IP extension, the high,
medium, and low data tunnels are created for both protocols. This means a total of seven virtual tunnels are created: the F-Class tunnel,
and three QoS data tunnels for each protocol.
Virtual circuits and tunnels
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 19
NOTE
The Brocade FX8-24 does not support IP extension. It creates the four FCIP virtual tunnels (1 control and 3 data).
An additional tunnel group is created for high availability (HA) when the hot code load (HCL) feature is congured on the Brocade SX6 or
Brocade 7840. That tunnel group contains a main tunnel, a local backup tunnel, and a remote backup tunnel. When HCL is activated, the
high, medium, and low QoS data tunnels are collapsed into a single QoS data tunnel for the duration of the HCL update. For additional
information, refer to Extension Hot Code Load on page 25.
FCIP Extension
FCIP, or Fiber Channel over IP, is a tunneling protocol to link Fibre Channel over distance on standard IP networks. Used primarily for
remote replication, backup, and storage access, FCIP extension provides Fibre Channel connectivity over IP networks between Fibre
Channel devices or fabrics. The FCIP link is an inter-switch link (ISL) that transports FC control and data frames between switches. FCIP
extension is supported on all Brocade extension platforms.
FCIP extension provides the following:
Network resiliency using Extension Trunking, which is a single logical tunnel comprised of one or more individual circuits
Ecient use of VE_Ports
High performance for high speed WAN links (one or more 10 Gbps and 40 Gbps links)
TCP Acceleration with WAN Optimized TCP (WO-TCP)
Aggregation of circuit bandwidth
Failover/failback
Failover groups and metrics
Spillover
Use of disparate characteristic WAN paths
Non-disruptive link loss
Lossless Link Loss (LLL)
Adaptive Rate Limiting (ARL)
In-Order Delivery (IOD)
Deterministic path for protocol acceleration
WAN bandwidth pooling — pool bandwidth from multiple links/providers
High speed compression using Deate
High speed IPsec (AES 256)
Diagnostic and troubleshooting tools — SAN Health and WAN Tool
QoS for FCIP with DSCP and/or 802.1P marking and enforcement
9216-byte Jumbo Frames support on the Brocade SX6 blade and Brocade 7840 switch
Within the architecture of the Brocade 7840, SX6, and FX8-24, there are Brocade FC Application-Specic Integrated Circuits (ASICs),
which know only FC protocol. The VE/VEX_Ports are not part of the ASICs and are logical representations of actual FC ports. In
actuality, multiple FC ports feed a VE/VEX_Port, permitting high data rates well above 8 gigabits per second (Gbps), 16 Gbps, and 32
Gbps, which is necessary for feeding the compression engine at high data rates and for high-speed trunks. On the WAN side, the
Brocade 7840 and SX6 features 10 Gigabit Ethernet (GbE) and 40 GbE interfaces. The Brocade FX8-24 features 10 GbE and 1 GbE
interfaces. Think of VE and VEX_Ports as the transition point from the FC world to the TCP/IP world inside the extension devices.
FCIP Extension
Brocade Fabric OS Extension Conguration Guide, 8.1.0
20 53-1004390-03
  • Page 1 1
  • Page 2 2
  • Page 3 3
  • Page 4 4
  • Page 5 5
  • Page 6 6
  • Page 7 7
  • Page 8 8
  • Page 9 9
  • Page 10 10
  • Page 11 11
  • Page 12 12
  • Page 13 13
  • Page 14 14
  • Page 15 15
  • Page 16 16
  • Page 17 17
  • Page 18 18
  • Page 19 19
  • Page 20 20
  • Page 21 21
  • Page 22 22
  • Page 23 23
  • Page 24 24
  • Page 25 25
  • Page 26 26
  • Page 27 27
  • Page 28 28
  • Page 29 29
  • Page 30 30
  • Page 31 31
  • Page 32 32
  • Page 33 33
  • Page 34 34
  • Page 35 35
  • Page 36 36
  • Page 37 37
  • Page 38 38
  • Page 39 39
  • Page 40 40
  • Page 41 41
  • Page 42 42
  • Page 43 43
  • Page 44 44
  • Page 45 45
  • Page 46 46
  • Page 47 47
  • Page 48 48
  • Page 49 49
  • Page 50 50
  • Page 51 51
  • Page 52 52
  • Page 53 53
  • Page 54 54
  • Page 55 55
  • Page 56 56
  • Page 57 57
  • Page 58 58
  • Page 59 59
  • Page 60 60
  • Page 61 61
  • Page 62 62
  • Page 63 63
  • Page 64 64
  • Page 65 65
  • Page 66 66
  • Page 67 67
  • Page 68 68
  • Page 69 69
  • Page 70 70
  • Page 71 71
  • Page 72 72
  • Page 73 73
  • Page 74 74
  • Page 75 75
  • Page 76 76
  • Page 77 77
  • Page 78 78
  • Page 79 79
  • Page 80 80
  • Page 81 81
  • Page 82 82
  • Page 83 83
  • Page 84 84
  • Page 85 85
  • Page 86 86
  • Page 87 87
  • Page 88 88
  • Page 89 89
  • Page 90 90
  • Page 91 91
  • Page 92 92
  • Page 93 93
  • Page 94 94
  • Page 95 95
  • Page 96 96
  • Page 97 97
  • Page 98 98
  • Page 99 99
  • Page 100 100
  • Page 101 101
  • Page 102 102
  • Page 103 103
  • Page 104 104
  • Page 105 105
  • Page 106 106
  • Page 107 107
  • Page 108 108
  • Page 109 109
  • Page 110 110
  • Page 111 111
  • Page 112 112
  • Page 113 113
  • Page 114 114
  • Page 115 115
  • Page 116 116
  • Page 117 117
  • Page 118 118
  • Page 119 119
  • Page 120 120
  • Page 121 121
  • Page 122 122
  • Page 123 123
  • Page 124 124
  • Page 125 125
  • Page 126 126
  • Page 127 127
  • Page 128 128
  • Page 129 129
  • Page 130 130
  • Page 131 131
  • Page 132 132
  • Page 133 133
  • Page 134 134
  • Page 135 135
  • Page 136 136
  • Page 137 137
  • Page 138 138
  • Page 139 139
  • Page 140 140
  • Page 141 141
  • Page 142 142
  • Page 143 143
  • Page 144 144
  • Page 145 145
  • Page 146 146
  • Page 147 147
  • Page 148 148
  • Page 149 149
  • Page 150 150
  • Page 151 151
  • Page 152 152
  • Page 153 153
  • Page 154 154
  • Page 155 155
  • Page 156 156
  • Page 157 157
  • Page 158 158
  • Page 159 159
  • Page 160 160
  • Page 161 161
  • Page 162 162
  • Page 163 163
  • Page 164 164
  • Page 165 165
  • Page 166 166
  • Page 167 167
  • Page 168 168
  • Page 169 169
  • Page 170 170
  • Page 171 171
  • Page 172 172
  • Page 173 173
  • Page 174 174
  • Page 175 175
  • Page 176 176
  • Page 177 177
  • Page 178 178
  • Page 179 179
  • Page 180 180
  • Page 181 181
  • Page 182 182
  • Page 183 183
  • Page 184 184
  • Page 185 185
  • Page 186 186
  • Page 187 187
  • Page 188 188
  • Page 189 189
  • Page 190 190
  • Page 191 191
  • Page 192 192
  • Page 193 193
  • Page 194 194
  • Page 195 195
  • Page 196 196
  • Page 197 197
  • Page 198 198
  • Page 199 199
  • Page 200 200
  • Page 201 201
  • Page 202 202
  • Page 203 203
  • Page 204 204
  • Page 205 205
  • Page 206 206
  • Page 207 207
  • Page 208 208
  • Page 209 209
  • Page 210 210
  • Page 211 211
  • Page 212 212
  • Page 213 213
  • Page 214 214
  • Page 215 215
  • Page 216 216
  • Page 217 217
  • Page 218 218

Broadcom Brocade Fabric OS Extension Configuration 8.1.0 User guide

Type
User guide

Ask a question and I''ll find the answer in the document

Finding information in a document is now easier with AI