Page is loading ...
Supporting Fabric OS 8.1.0
CONFIGURATION GUIDE
Brocade Fabric OS Extension
Conguration Guide, 8.1.0
53-1004390-03
28 April 2017
©
2017, Brocade Communications Systems, Inc. All Rights Reserved.
Brocade, the B-wing symbol, and MyBrocade are registered trademarks of Brocade Communications Systems, Inc., in the United States and in other
countries. Other brands, product names, or service names mentioned of Brocade Communications Systems, Inc. are listed at www.brocade.com/en/legal/
brocade-Legal-intellectual-property/brocade-legal-trademarks.html. Other marks may belong to third parties.
Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment,
equipment feature, or service oered or to be oered by Brocade. Brocade reserves the right to make changes to this document at any time, without
notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade
sales oce for information on feature and product availability. Export of technical data contained in this document may require an export license from the
United States government.
The authors and Brocade Communications Systems, Inc. assume no liability or responsibility to any person or entity with respect to the accuracy of this
document or any loss, cost, liability, or damages arising from the information contained herein or the computer programs that accompany it.
The product described by this document may contain open source software covered by the GNU General Public License or other open source license
agreements. To nd out which open source software is included in Brocade products, view the licensing terms applicable to the open source software, and
obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.
Brocade Fabric OS Extension Conguration Guide, 8.1.0
2 53-1004390-03
Contents
Preface...................................................................................................................................................................................................................................9
Document conventions............................................................................................................................................................................................................................9
Notes, cautions, and warnings.....................................................................................................................................................................................................9
Text formatting conventions.........................................................................................................................................................................................................9
Command syntax conventions.................................................................................................................................................................................................10
Brocade resources..................................................................................................................................................................................................................................10
Document feedback.............................................................................................................................................................................................................................. 10
Contacting Brocade Technical Support......................................................................................................................................................................................... 11
Brocade customers.......................................................................................................................................................................................................................11
Brocade OEM customers.......................................................................................................................................................................................................... 11
About this Document.......................................................................................................................................................................................................13
Supported hardware and software...................................................................................................................................................................................................13
What's new in this document............................................................................................................................................................................................................. 13
Changes made for the initial release ( 53-1004390-01)...........................................................................................................................................13
Changes made for this release ( 53-1004390-02)..................................................................................................................................................... 14
Changes made for this release ( 53-1004390-03)..................................................................................................................................................... 14
Extension Concepts and Features................................................................................................................................................................................ 15
Brocade Extension concepts..............................................................................................................................................................................................................15
Extension trunks, tunnels, circuits, and interfaces......................................................................................................................................................................16
VE_Ports and VEX_Ports................................................................................................................................................................................................................... 18
Gigabit Ethernet interfaces..................................................................................................................................................................................................................19
Ethernet interfaces..................................................................................................................................................................................................................................19
Virtual circuits and tunnels...................................................................................................................................................................................................................19
FCIP Extension........................................................................................................................................................................................................................................ 20
IP Extension...............................................................................................................................................................................................................................................21
IP Extension and IP networking...............................................................................................................................................................................................21
Extension Trunking................................................................................................................................................................................................................................. 22
Redundancy and fault tolerance..............................................................................................................................................................................................23
Considerations for multiple tunnel use with protocol optimization ..........................................................................................................................24
IP WAN network considerations....................................................................................................................................................................................................... 24
IP LAN network considerations........................................................................................................................................................................................................ 25
Extension Hot Code Load................................................................................................................................................................................................................... 25
Extension HCL operation...........................................................................................................................................................................................................25
Extension HCL limitations and considerations..................................................................................................................................................................27
Fibre Channel Storage Area Network considerations..............................................................................................................................................................28
Adaptive Rate Limiting..........................................................................................................................................................................................................................29
Brocade 7840 switch and Brocade SX6 blade support for ARL.............................................................................................................................29
Brocade FX8-24 support for ARL........................................................................................................................................................................................ 30
FSPF link cost calculation when ARL is used................................................................................................................................................................... 30
ARL considerations.......................................................................................................................................................................................................................30
Compression options............................................................................................................................................................................................................................ 31
Compression options for the Brocade 7840 Extension Switch and Brocade SX6 Extension Blade....................................................... 31
Compression options for the Brocade FX8-24 Extension Blade............................................................................................................................ 32
FastWrite and Open Systems Tape Pipelining............................................................................................................................................................................32
FastWrite and OSTP network congurations.....................................................................................................................................................................32
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 3
FICON Acceleration...............................................................................................................................................................................................................................34
VM Insight..................................................................................................................................................................................................................................................34
IP security...................................................................................................................................................................................................................................................35
IPsec for the extension switches and blades......................................................................................................................................................................36
Limitations using IPsec over tunnels.....................................................................................................................................................................................36
IPv6 addressing.......................................................................................................................................................................................................................................37
IPv6 with embedded IPv4 addresses...................................................................................................................................................................................37
Memory use limitations for large-device tunnel congurations.......................................................................................................................................... 38
Control blocks created during FCP trac ow..................................................................................................................................................................40
Control blocks created during FICON trac ow............................................................................................................................................................ 40
Considerations for tunnel control block memory and device conguration..........................................................................................................41
Firmware downloads..............................................................................................................................................................................................................................42
Extension Platforms and Features................................................................................................................................................................................43
Extension platforms and features overview................................................................................................................................................................................. 43
Brocade 7840 Extension Switch and Brocade SX6 Extension Blade overview......................................................................................................... 45
Brocade 7840 Extension Switch ports................................................................................................................................................................................46
Brocade SX6 Extension Blade ports.....................................................................................................................................................................................47
Ethernet port groups.................................................................................................................................................................................................................... 47
Fibre Channel port groups.........................................................................................................................................................................................................48
Network DP components...........................................................................................................................................................................................................49
10VE and 20VE port distribution.......................................................................................................................................................................................... 52
10 GbE and 40 GbE port and circuit considerations ....................................................................................................................................................53
Brocade 7840 license options................................................................................................................................................................................................53
Brocade SX6 license options....................................................................................................................................................................................................53
Brocade FX8-24 Extension Blade overview.............................................................................................................................................................................. 54
Brocade FX8-24 operating modes.......................................................................................................................................................................................54
Brocade FX8-24 data processor complexes....................................................................................................................................................................54
Removing the Brocade FX8-24 Extension Blade.......................................................................................................................................................... 55
Brocade FX8-24 License options......................................................................................................................................................................................... 55
Brocade FX8-24 multigigabit circuits.................................................................................................................................................................................. 56
Crossports and failover............................................................................................................................................................................................................... 56
Bandwidth allocation and restrictions....................................................................................................................................................................................56
Tunnel and circuit requirements for Brocade Extension platforms.................................................................................................................................... 59
Brocade 7840 and SX6 requirements.................................................................................................................................................................................60
Brocade FX8-24 requirements...............................................................................................................................................................................................61
Brocade IP Extension............................................................................................................................................................................................................................62
Tunnels and hybrid mode...........................................................................................................................................................................................................63
Out-of-order delivery on a tunnel...........................................................................................................................................................................................63
IP Extension and IP on the LAN..............................................................................................................................................................................................63
IP Extension and Trac Control Lists .................................................................................................................................................................................. 63
IP Extension and QoS................................................................................................................................................................................................................. 68
IP Extension and compression................................................................................................................................................................................................ 68
IP Extension and IP LAN deployment..................................................................................................................................................................................69
IP Extension limitations and considerations....................................................................................................................................................................... 71
Extension Hot Code Load for Brocade 7840 and Brocade SX6...................................................................................................................................... 71
Path Maximum Transmission Unit discovery..............................................................................................................................................................................71
Circuit failover........................................................................................................................................................................................................................................... 72
Circuit failover grouping.............................................................................................................................................................................................................. 73
Failover in TI zones....................................................................................................................................................................................................................... 74
Bandwidth calculation during failover....................................................................................................................................................................................74
Brocade Fabric OS Extension Conguration Guide, 8.1.0
4 53-1004390-03
10 GbE lossless link loss (FX8-24 blade)..........................................................................................................................................................................75
Circuit spillover.........................................................................................................................................................................................................................................76
Understanding circuit spillover utilization.............................................................................................................................................................................76
Circuit spillover considerations.................................................................................................................................................................................................79
Service Level Agreement ....................................................................................................................................................................................................................79
Conguring Extension Features....................................................................................................................................................................................81
Conguration overview......................................................................................................................................................................................................................... 81
Conguration prerequisites.................................................................................................................................................................................................................82
Conguring platform modes .............................................................................................................................................................................................................83
Conguring FCIP or hybrid mode..........................................................................................................................................................................................83
Conguring VE mode.................................................................................................................................................................................................................. 84
Clearing SX6 blade conguration ..........................................................................................................................................................................................85
Conguring GbE mode on Brocade FX8-24 blade.......................................................................................................................................................85
Conguring VEX_Ports on the FX8-24..............................................................................................................................................................................86
Conguring ports.................................................................................................................................................................................................................................... 87
Conguring port speeds.............................................................................................................................................................................................................87
Conguring IP...........................................................................................................................................................................................................................................88
Conguring IPIF............................................................................................................................................................................................................................. 88
Conguring IP route......................................................................................................................................................................................................................90
Conguring VLAN..................................................................................................................................................................................................................................91
Verifying IP connectivity.......................................................................................................................................................................................................................92
Conguring Service Level Agreement .......................................................................................................................................................................................... 93
Conguring IPsec....................................................................................................................................................................................................................................95
Conguring IPsec on the Brocade 7840 and Brocade SX6...................................................................................................................................... 96
IPsec IKE authentication failures............................................................................................................................................................................................. 99
Conguring IPsec on the FX8-24...................................................................................................................................................................................... 100
Conguring Extension tunnels for FCIP.....................................................................................................................................................................................101
Conguring VE_Ports to persistently disabled.............................................................................................................................................................. 102
Conguring tunnels....................................................................................................................................................................................................................104
Conguring emulation features on tunnels......................................................................................................................................................................107
Conguring compression options....................................................................................................................................................................................... 107
Conguring WAN on tunnels.................................................................................................................................................................................................109
Conguring failover....................................................................................................................................................................................................................117
Conguring failover groups.................................................................................................................................................................................................... 118
Conguring spillover..................................................................................................................................................................................................................121
Conguring VE_Ports to persistently enabled...............................................................................................................................................................122
Verifying tunnel conguration................................................................................................................................................................................................124
Conguring Extension Hot Code Load.......................................................................................................................................................................................127
Conguring DP complexes and HCL tunnels................................................................................................................................................................ 130
Conguring IP Extension.................................................................................................................................................................................................................. 131
Conguration steps for IP Extension features................................................................................................................................................................ 132
Conguring hybrid mode for IP Extension features.....................................................................................................................................................134
Conguring IPIF for IP Extension........................................................................................................................................................................................ 134
Conguring WAN IP route for IP Extension.................................................................................................................................................................... 135
Conguring a tunnel to support IP Extension.................................................................................................................................................................137
Conguring bandwidth distribution..................................................................................................................................................................................... 138
Conguring LAN IP route for IP Extension and PBR..................................................................................................................................................139
Conguring tunnel compression..........................................................................................................................................................................................142
Conguring a tunnel and circuits for IP Extension........................................................................................................................................................143
Conguring Ethernet interfaces (GbE port) for IP Extension LAN features....................................................................................................... 144
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 5
Conguring LAG.........................................................................................................................................................................................................................145
Conguring a LAN gateway (SVI) for IP Extension...................................................................................................................................................... 148
Conguring Trac Control Lists for IP Extension ........................................................................................................................................................150
Example IP Extension Conguration..................................................................................................................................................................................156
Conguring Brocade FX8-24 crossport features..................................................................................................................................................................161
Conguring crossports on the Brocade FX8-24..........................................................................................................................................................161
Crossports and failover............................................................................................................................................................................................................ 162
Conguring IP routes with crossports................................................................................................................................................................................164
Conguring VLAN tags with crossports........................................................................................................................................................................... 165
Display VLAN tag conguration using the portshow vlantag command............................................................................................................165
Using ping with crossports..................................................................................................................................................................................................... 165
Using traceroute with crossports..........................................................................................................................................................................................165
Using logical switches........................................................................................................................................................................................................................166
Logical switch overview .......................................................................................................................................................................................................... 166
Considerations for logical switches.....................................................................................................................................................................................168
Trac Isolation Zoning.......................................................................................................................................................................................................................177
Zoning.......................................................................................................................................................................................................................................................177
Troubleshooting Tools.................................................................................................................................................................................................. 179
In-band management........................................................................................................................................................................................................................ 179
IP routing........................................................................................................................................................................................................................................179
Conguring IP addresses and routes.................................................................................................................................................................................180
VLAN tagging support.............................................................................................................................................................................................................184
IP forwarding support............................................................................................................................................................................................................... 184
WAN analysis tools..............................................................................................................................................................................................................................186
The tperf option...........................................................................................................................................................................................................................186
Using ping to test a connection............................................................................................................................................................................................ 187
Using traceroute..........................................................................................................................................................................................................................188
Using WAN Tool...................................................................................................................................................................................................................................188
WAN Tool commands.............................................................................................................................................................................................................. 189
Conguring a WAN Tool session and displaying results............................................................................................................................................190
Resolving test session problems......................................................................................................................................................................................... 196
Using the portshow command.......................................................................................................................................................................................................196
Displaying IP interfaces............................................................................................................................................................................................................196
Displaying IP routes...................................................................................................................................................................................................................196
Displaying switch mode information with the extncfg command.......................................................................................................................... 197
Displaying GbE port information with the portcfgge command.............................................................................................................................197
Listing the MAC addresses of LAN and GE ports....................................................................................................................................................... 197
Displaying LAG information...................................................................................................................................................................................................198
Displaying tunnel HCL information.....................................................................................................................................................................................198
Displaying TCL information....................................................................................................................................................................................................199
Displaying IP Extension LAN statistics..............................................................................................................................................................................199
Displaying performance statistics........................................................................................................................................................................................201
Displaying QoS statistics.........................................................................................................................................................................................................201
Displaying details........................................................................................................................................................................................................................201
Filtering portshow display output ....................................................................................................................................................................................... 201
Displaying tunnel status...........................................................................................................................................................................................................202
Displaying tunnel information................................................................................................................................................................................................203
Displaying a tunnel with circuit information .................................................................................................................................................................... 203
Displaying tunnel performance ............................................................................................................................................................................................204
Displaying tunnel TCP statistics ..........................................................................................................................................................................................205
Brocade Fabric OS Extension Conguration Guide, 8.1.0
6 53-1004390-03
Displaying circuits.......................................................................................................................................................................................................................205
Displaying a single circuit........................................................................................................................................................................................................ 205
Displaying TCP statistics for circuits...................................................................................................................................................................................205
Displaying circuit performance ............................................................................................................................................................................................ 205
Displaying GbE port performance.......................................................................................................................................................................................206
Displaying QoS prioritization for a circuit......................................................................................................................................................................... 206
Displaying tunnel information (FX8-24 blade)...............................................................................................................................................................208
Tunnel issues ........................................................................................................................................................................................................................................ 208
Tunnel does not come online................................................................................................................................................................................................ 208
Tunnel goes online and oine...............................................................................................................................................................................................210
Troubleshooting Extension links....................................................................................................................................................................................................211
Gathering additional information..........................................................................................................................................................................................211
Using FTRACE..................................................................................................................................................................................................................................... 212
FTRACE conguration.............................................................................................................................................................................................................212
Changing conguration settings...........................................................................................................................................................................................214
Displaying FTRACE status on a DP complex................................................................................................................................................................ 216
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 7
Brocade Fabric OS Extension Conguration Guide, 8.1.0
8 53-1004390-03
Preface
• Document conventions......................................................................................................................................................................................9
• Brocade resources............................................................................................................................................................................................ 10
• Document feedback.........................................................................................................................................................................................10
• Contacting Brocade Technical Support....................................................................................................................................................11
Document conventions
The document conventions describe text formatting conventions, command syntax conventions, and important notice formats used in
Brocade technical documentation.
Notes, cautions, and warnings
Notes, cautions, and warning statements may be used in this document. They are listed in the order of increasing severity of potential
hazards.
NOTE
A Note provides a tip, guidance, or advice, emphasizes important information, or provides a reference to related information.
ATTENTION
An Attention statement indicates a stronger note, for example, to alert you when trac might be interrupted or the device might
reboot.
CAUTION
A Caution statement alerts you to situations that can be potentially hazardous to you or cause damage to hardware,
rmware, software, or data.
DANGER
A Danger statement indicates conditions or situations that can be potentially lethal or extremely hazardous to you. Safety
labels are also attached directly to products to warn of these conditions or situations.
Text formatting conventions
Text formatting conventions such as boldface, italic, or Courier font may be used to highlight specic words or phrases.
Format Description
bold text Identies command names.
Identies keywords and operands.
Identies the names of GUI elements.
Identies text to enter in the GUI.
italic text Identies emphasis.
Identies variables.
Identies document titles.
Courier font
Identies CLI output.
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 9
Format Description
Identies command syntax examples.
Command syntax conventions
Bold and italic text identify command syntax components. Delimiters and operators
dene groupings of parameters and their logical
relationships.
Convention Description
bold text Identies command names, keywords, and command options.
italic text Identies a variable.
value In Fibre Channel products, a xed value provided as input to a command option is printed in plain text, for
example, --show WWN.
[ ] Syntax components displayed within square brackets are optional.
Default responses to system prompts are enclosed in square brackets.
{ x | y | z } A choice of required parameters is enclosed in curly brackets separated by vertical bars. You must select
one of the options.
In Fibre Channel products, square brackets may be used instead for this purpose.
x | y A vertical bar separates mutually exclusive elements.
< > Nonprinting characters, for example, passwords, are enclosed in angle brackets.
... Repeat the previous element, for example, member[member...].
\ Indicates a “soft” line break in command examples. If a backslash separates two lines of a command
input, enter the entire command at the prompt without the backslash.
Brocade resources
Visit the Brocade website to locate related documentation for your product and additional Brocade resources.
White papers, data sheets, and the most recent versions of Brocade software and hardware manuals are available at www.brocade.com.
Product documentation for all supported releases is available to registered users at MyBrocade.
Click the Support tab and select Document Library to access product documentation on MyBrocade or www.brocade.com. You can
locate documentation by product or by operating system.
Release notes are bundled with software downloads on MyBrocade. Links to software downloads are available on the MyBrocade landing
page and in the Document Library.
Document feedback
Quality is our
rst concern at Brocade, and we have made every eort to ensure the accuracy and completeness of this document.
However, if you nd an error or an omission, or you think that a topic needs further development, we want to hear from you. You can
provide feedback in two ways:
• Through the online feedback form in the HTML documents posted on www.brocade.com
• By sending your feedback to documentation@brocade.com
Provide the publication title, part number, and as much detail as possible, including the topic heading and page number if applicable, as
well as your suggestions for improvement.
Brocade resources
Brocade Fabric OS Extension Conguration Guide, 8.1.0
10 53-1004390-03
Contacting Brocade Technical Support
As a Brocade customer, you can contact Brocade Technical Support 24x7 online or by telephone. Brocade OEM customers should
contact their OEM/solution provider.
Brocade customers
For product support information and the latest information on contacting the Technical Assistance Center, go to www.brocade.com and
select Support.
If you have purchased Brocade product support directly from Brocade, use one of the following methods to contact the Brocade
Technical Assistance Center 24x7.
Online Telephone
Preferred method of contact for non-urgent issues:
• Case management through the MyBrocade portal.
• Quick Access links to Knowledge Base, Community, Document
Library, Software Downloads and Licensing tools
Required for Sev 1-Critical and Sev 2-High issues:
• Continental US: 1-800-752-8061
• Europe, Middle East, Africa, and Asia Pacic: +800-AT FIBREE
(+800 28 34 27 33)
• Toll-free numbers are available in many countries.
• For areas unable to access a toll-free number:
+1-408-333-6061
Brocade OEM customers
If you have purchased Brocade product support from a Brocade OEM/solution provider, contact your OEM/solution provider for all of
your product support needs.
• OEM/solution providers are trained and
certied by Brocade to support Brocade
®
products.
• Brocade provides backline support for issues that cannot be resolved by the OEM/solution provider.
• Brocade Supplemental Support augments your existing OEM support contract, providing direct access to Brocade expertise.
For more information, contact Brocade or your OEM.
• For questions regarding service levels and response times, contact your OEM/solution provider.
Contacting Brocade Technical Support
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 11
Brocade Fabric OS Extension Conguration Guide, 8.1.0
12 53-1004390-03
About this Document
• Supported hardware and software..............................................................................................................................................................13
• What's new in this document........................................................................................................................................................................13
Supported hardware and software
The following hardware platforms support Brocade Extension (Fibre Channel over IP features and IP Extension features) as described in
this manual:
• Brocade DCX 8510-4, and DCX 8510-8 with one or more FX8-24 blades
• Brocade 7840 switch
• Brocade X6-4 Director and Brocade X6-8 Director with one or more SX6 Extension blades
The following software is supported as described in this manual:
• Brocade Fabric OS 8.1.0 software
What's new in this document
Changes made for the initial release ( 53-1004390-01)
New additions or deletions in this document support Fabric OS 8.1.0.
Major sections of this publication aected by additions and corrections include the following:
• Extension Concepts and Features
– The chapter has been modied for additional clarity and presentation of the Brocade Extension features.
– The chapter contains additional information about extension interfaces, circuits, and tunnels.
– FCIP Extension on page 20 was added.
– IP LAN network considerations on page 25 was added.
– Extension Hot Code Load on page 25 was expanded.
– Fibre Channel Storage Area Network considerations on page 28FC SAN considerations was added.
– FICON Acceleration on page 34 was added.
– VM Insight on page 34 was added.
– IP security on page 35 was expanded and information about Suite B Cryptographic Suites was added.
• Extension Features on Brocade Extension Switches and Blades
– The chapter has been modied for additional clarity and presentation of the Brocade Extension platforms and features.
– Brocade 7840 Extension Switch and Brocade SX6 Extension Blade overview on page 45 provides information about
these two related platforms.
– Fibre Channel port groups on page 48 was added.
– Tunnel and circuit requirements for Brocade Extension platforms on page 59 was updated and reorganized.
• Conguring Extension Features
– This chapter has been extensively reorganized and updated so that the presentation of information more closely matches
conguration work-ow. Additional examples are provided and command results are upated.
– Conguring IPsec on page 95 was updated.
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 13
– Conguring IP Extension on page 131 contains information that previously existed in a separate chapter.
– Conguring app-type for TCL on page 156 was added.
– Conguring Brocade FX8-24 crossport features on page 161 was added. It consolidates information about crossport
conguration.
• Troubleshooting Tools
– This chapter has been updated for Fabric OS 8.1.0 with modications to command examples and outputs.
– Conguring a WAN Tool session and displaying results on page 190 was updated.
– Filtering portshow display output on page 201 was updated.
Changes made for this release ( 53-1004390-02)
Additions and corrections to the document include the following:
• Minor editorial corrections have been made throughout this document.
• In VE_Ports and VEX_Ports on page 18, the table header was corrected.
• In VM Insight on page 34, instances of VMOSAN were changed to VM Insight.
• In Fibre Channel port groups on page 48, corrected information about Brocade 7840 port groups.
Changes made for this release ( 53-1004390-03)
Additions and corrections to the document include the following:
• In Using WAN Tool on page 188, WAN Tool commands on page 189, and Conguring a WAN Tool session and displaying
results on page 190, several updates were made for CLI operation and command output. The number of congured WAN Tool
sessions per DP was changed from 8 to 10 and the maximum number of sessions per slot is 16.
What's new in this document
Brocade Fabric OS Extension Conguration Guide, 8.1.0
14 53-1004390-03
Extension Concepts and Features
• Brocade Extension concepts........................................................................................................................................................................15
• Extension trunks, tunnels, circuits, and interfaces................................................................................................................................16
• VE_Ports and VEX_Ports..............................................................................................................................................................................18
• Gigabit Ethernet interfaces.............................................................................................................................................................................19
• Ethernet interfaces.............................................................................................................................................................................................19
• Virtual circuits and tunnels............................................................................................................................................................................. 19
• FCIP Extension...................................................................................................................................................................................................20
• IP Extension......................................................................................................................................................................................................... 21
• Extension Trunking............................................................................................................................................................................................22
• IP WAN network considerations..................................................................................................................................................................24
• IP LAN network considerations...................................................................................................................................................................25
• Extension Hot Code Load..............................................................................................................................................................................25
• Fibre Channel Storage Area Network considerations.........................................................................................................................28
• Adaptive Rate Limiting.................................................................................................................................................................................... 29
• Compression options.......................................................................................................................................................................................31
• FastWrite and Open Systems Tape Pipelining...................................................................................................................................... 32
• FICON Acceleration..........................................................................................................................................................................................34
• VM Insight.............................................................................................................................................................................................................34
• IP security............................................................................................................................................................................................................. 35
• IPv6 addressing................................................................................................................................................................................................. 37
• Memory use limitations for large-device tunnel
congurations.....................................................................................................38
• Firmware downloads........................................................................................................................................................................................42
Brocade Extension concepts
Brocade extension switches and extension blades for Brocade director families provide fast and reliable network infrastructure to address
the requirements of Storage Area Networks (SANs) that are extended beyond the traditional reaches of Fibre Channel (FC)
communications. Without some form of distance extension, the distance between the source and the destination in a SAN is limited to a
few kilometers.
Brocade extension products support both FC/FICON-based data
ows and IP-based storage data ows. Brocade extension solutions
maximize replication and backup throughput over distance, using data compression, disk and tape protocol acceleration, and WAN-
optimized TCP. Brocade extension supports applications such as remote data replication (RDR), centralized backup, and data migration.
Brocade extension uses the existing IP wide area network (WAN) infrastructure to connect Fibre Channel and IP fabrics between distant
endpoints that are either impractical or costly using native Fibre Channel or IP connections. The basis of the connection is the extension
tunnel, built on a physical connection between two extension switches or blades. Extension tunnels allow Fibre Channel and IP trac to
pass through the IP WAN. The extension tunnel connections ensure lossless transmission, and that FC and IP frames are delivered in the
correct order. The Fibre Channel fabric and all targets and initiators, whether FC or IP, are unaware of the presence of the IP WAN.
The extension tunnel provides load balancing across separate network paths, optimization for extended links, rate-limiting to ensure
optimal performance, and lossless link loss (LLL) recovery.
The two Brocade protocol technologies implemented in Brocade extension products are FCIP and IP Extension.
• FCIP, or Fiber Channel over IP, is a tunneling protocol to link Fibre Channel over distance on standard IP networks. Used
primarily for remote replication, backup, and storage access, FCIP provides Fibre Channel connectivity over IP networks
between Fibre Channel devices or fabrics. The FCIP link is an inter-switch link (ISL) that transports FC control and data frames
between switches. The following table outlines FCIP.
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 15
Network WAN/MAN
Transport FCIP/TCP/IP/Ethernet
Encapsulation Brocade encapsulates Fibre Channel data sequences into compressed
batches. Those batches ll TCP segments to their maximum size and then
form IP datagrams.
IP-routable Yes
• Brocade IP Extension is used primarily for IP storage applications, such as remote host-based or database-based replication,
NAS replication, IP backups, and tape grids. IP Extension uses the same VE_Ports and circuits that FCIP uses, or it can use its
own. The following table outlines IP Extension.
Network WAN/MAN
Transport IP Extension/TCP/IP/Ethernet
Encapsulation Brocade encapsulates IP ow (also called “streams”) data sequences into
compressed batches. Those batches ll TCP segments to their maximum
size and then form IP datagrams.
IP-routable Yes
For additional information about implementation and technical details of Brocade extension technology, refer to the Brocade white paper,
Extension Trunking.
Extension trunks, tunnels, circuits, and interfaces
An extension tunnel is a conduit that contains one or more circuits. When a tunnel contains multiple circuits, it is also called an extension
trunk because multiple circuits are trunked together. An extension tunnel, or extension trunk, is a single Inter-Switch Link (ISL). Circuits are
individual extension connections within the trunk, each with its own unique source and destination IP interface (IPIF) address.
To understand tunnels, you must understand the relationship between tunnels and VE_Ports. Because an extension tunnel is an ISL,
each end of the tunnel requires its own Virtual E_Port (VE_Port). For example, the Brocade SX6 extension blade supports a number of
VE_Ports. The available VE_Ports on the Brocade SX6 are numbered 16 to 35. When you create a tunnel, that tunnel can be numbered
any of 16 to 35. Tunnels are more complicated than this, but the point is that each end of the tunnel is
identied by number, and that
number is directly associated with a VE_Port on the extension platform. Tunnels are frequently created between dierent VE_Ports, so
the tunnel number can be dierent at each end, from a conguration point of view. Each extension platform, such as Brocade SX6
extension blade, Brocade 7840 extension switch, and Brocade FX8-24 extension blade, has a dierent numbering scheme for its
VE_Ports.
Circuits exist within tunnels. A circuit is a connection between a pair of IP addresses that are dened within an extension tunnel. Circuits
provide the links for trac ow between source and destination interfaces that are located on either end of the tunnel. Each tunnel can
contain a single circuit or multiple circuits.
NOTE
In this publication, the "source" or "local" is the switch you are conguring, while the "destination" or "remote" is the switch on the
other end of the tunnel or circuit. Local and remote will depend on which side of the of the tunnel you are conguring.
You must congure unique IPIFs as the local and remote endpoints of each circuit. On the local side, a circuit is congured with a source
IPIF address and a destination address. On the remote side of the circuit, its source IPIF address is the same as the local-side destination
address. Similarly, on the remote side of the circuit its IPIF destination address points to the local-side source address. Multiple IPIFs can
be congured on each physical Ethernet interface.
Extension trunks, tunnels, circuits, and interfaces
Brocade Fabric OS Extension Conguration Guide, 8.1.0
16 53-1004390-03
NOTE
On the Brocade FX8-24, the IP address (or IPIF) for each local and remote address must be individually unique. However, on
the Brocade 7840 and Brocade SX6, each address pair must be unique. For example, the following address pairs use the
same source address in each pair but the destination addresses are dierent. For the FX8-24, these addresses are not unique.
For the SX6 and 7840, the address pairs are unique.
• --local-ip 10.0.1.10 --remote-ip 10.1.1.10
• --local-ip 10.0.1.10 --remote-ip 10.1.1.11
The circuit conguration parameters on the local and remote sides of the circuits and tunnel must match in addition to the source and
destination IPIF addresses pointing to each other. For example, if you congure IPsec on a tunnel, each end of the tunnel must be
congured to use the same IPsec parameters. Other parameters for each circuit must match, such as MTU size, bandwidth allocations,
QoS, VLAN ID, and keepalive values.
You must congure a gateway IP route for the circuit to the destination network when the remote IPIF is not on the same subnet as the
local IPIF. You can dene a specic number of routes per IPIF based on the extension platform. Refer to Tunnel and circuit requirements
for Brocade Extension platforms on page 59 for specications.
ATTENTION
When using Brocade IP Extension, the local and remote LAN subnet addresses must be
dierent.
The following gure shows an example of two extension tunnels. The rst tunnel is a trunk of four circuits, and the second tunnel is a
trunk of two circuits. Each circuit is assigned a unique IPIF address. Those IPIFs are, in turn, assigned to Ethernet interfaces. In the gure,
each IPIF is assigned to a dierent Ethernet interface. This is not required. Ethernet interface assignment is exible depending on the
environment's needs and assignments can be made as desired. For instance, multiple IP interfaces can be assigned to a single Ethernet
interface. The circuit ows from IP interface to IP interface through the assigned Ethernet interfaces.
FIGURE 1 Extension tunnel and circuits
Extension trunks, tunnels, circuits, and interfaces
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 17
For specications and restrictions on tunnels, circuits, and trunks for the 7840 Extension Switch, the SX6 Extension Blade, and the
FX8-24 Extension Blade, refer to the Extension Platforms and Features chapter of this publication.
VE_Ports and VEX_Ports
All Fibre Channel communications between switches that include fabric services must communicate across an Inter-Switch Link (ISL).
Because an extension tunnel is an ISL, each end requires its own E_Port. There are various types of E_Ports:
• E_Port
• EX_Port
• VE_Port (virtual E_Port)
• VEX_Port (virtual EX_Port)
VE_Ports and VEX_ports are virtual because they face the extension tunnel and thus enable communication across an extension tunnel.
Extension trunking — multiple circuits in a tunnel — operates the same whether the virtual port is a VE_Port or a VEX_Port.
NOTE
VEX_Ports are supported only on the Brocade FX8-24 extension blade. The Brocade 7840 switch and the Brocade SX6
blade do not support VEX_Ports.
EX and VEX ports are FC Routed (FCR) ports. Router ports are the demarcation point of fabric services for a fabric. Fabric services do
not extend beyond an EX or VEX port. Remote edge fabrics are edge fabrics connected through a WAN connection and VEX_Port.
TABLE 3 VE_Ports and VEX_Ports
E port types No FCR FCR
Native FC E_Port EX_Port
Extended over tunnel VE_Port VEX_Port (on FX8-24 only)
Because an extension trunk is logically a single tunnel, only a single VE_Port or VEX_Port is used for each end of the tunnel, even
though more than one circuit can be contained within the tunnel.
Once the tunnels are congured and the WAN-optimized TCP (WO-TCP) connections are made for a circuit, a logical Inter-Switch Link is
established between the switches. VE_Ports operate like E_Ports for all fabric services and Fabric OS operations, except that VE_Ports
use TCP/IP and Ethernet as the transport instead of FC.
A VEX_Port exposed by the extension tunnel to form an ISL connection allows you to congure a VEX_Port to support FCR
demarcation. From the point of view of a switch in an edge fabric, a VEX_Port appears as a normal E_Port. It follows the same Fibre
Channel protocol as other E_Ports. However, VEX_Ports terminate the attached fabric at the port and do not allow fabrics to merge by
propagating fabric services or routing topology information beyond that edge fabric. This provides edge fabric or remote edge fabric
isolation outward from the EX_Port or VEX_Port.
NOTE
VE_Ports or VEX_Ports cannot connect in parallel to the same domain at the same time as Fibre Channel E_Ports or
EX_Ports.
An extension tunnel is assigned to a VE_Port or VEX_Port on the switch or blade at each end of the tunnel. Because multiple VE_Ports
are supported on the extension switch or blade, you can create multiple tunnels on a switch or blade.
Fibre Channel frames enter an extension tunnel through the VE_Ports (or VEX_Ports) and are encapsulated and passed to TCP layer
connections. A Data Processing (DP) complex on the switch or blade handles the FC frame encapsulation, de-encapsulation, and
transmission to the TCP link.
VE_Ports and VEX_Ports
Brocade Fabric OS Extension Conguration Guide, 8.1.0
18 53-1004390-03
Gigabit Ethernet interfaces
On the Brocade extension platforms, the GbE interfaces are abstracted from the VE/VEX_Ports, IP interfaces, and extension tunnels and
trunks. This means that the GbE interfaces are not xed to the VE/VEX_Ports, IP interfaces, or extension tunnels and trunks. Depending
on the platform, you have access to a combination of 1 GbE interfaces, 10 GbE interfaces, and 40 GbE interfaces.
A GbE interface can be congured for WAN operations or LAN operations on platforms that support Brocade IP Extension. A GbE
interface is used for WAN operation when it functions as the endpoint of a circuit. The interface is WAN-facing, and you are creating the
tunnels and circuits. WAN is the only allowed GbE interface mode when a platform is congured for FCIP-only operation.
When a platform is congured for hybrid mode, which supports both FCIP and IP Extension, you congure the LAN-facing GbE ports
for LAN mode operations. When GbE ports are in LAN mode, they can be members of a static Link Aggregation Group (LAG).
Ethernet interfaces
Ethernet interfaces are assigned by associating them with IP interfaces (IPIFs). IP interfaces on the extension platforms are virtual entities
within the platform. The IPIFs, via their IP addresses, are assigned to circuits by designating the source IP address. The circuit is grouped
into an extension trunk by designating the VE_Port (or VEX_Port) to which it belongs. Tunnels/trunks are
identied by their VE_Port
number. If an extension trunk already exists, you can create an additional circuit and include it in the tunnel.
Each Ethernet interface on the Brocade SX6, Brocade 7840, or Brocade FX8-24 can be used by circuits from multiple VE/VEX_Ports.
Each VE/VEX_Port can use multiple Ethernet ports. Each VE/VEX_Port can have multiple IP interfaces, or circuits. Each IP interface or
circuit in a VE/VEX_Port is associated with an IP address. In turn, those IP addresses are assigned to
specic Ethernet interfaces.
For Brocade Extension Trunking, the data processor (DP) that owns the VE_Port controls all the member circuits. There is no distributed
processing, load sharing or LLL across DPs. Failover between DPs is done at the FC level by the Brocade FC switching ASIC, provided
the conguration permits it. The only components that can be shared are the Ethernet interfaces.
The IP network routes the circuits over separate network paths and WAN links based on the destination IP addresses and possibly other
Layer 2 and Layer 3 header attributes. Ethernet interfaces on the Brocade SX6, Brocade 7840, and Brocade FX8-24 provide a single
convenient connection to the data center LAN for one to many circuits.
Virtual circuits and tunnels
When a tunnel is created, a base virtual tunnel is created for that tunnel. A virtual tunnel is the actual object that carries the data ows. An
FCIP virtual tunnel itself consists of four tunnels:
• Control tunnel, or F-Class. The control tunnel is higher priority and has no guaranteed minimum bandwidth. It can use all
available bandwidth up to the maximum congured bandwidth rate.
• High data tunnel. The high data tunnel is for high-priority QoS data.
• Medium data tunnel. The medium data tunnel is for medium-priority QoS data.
• Low data tunnel. The low data tunnel is for low-priority QoS data.
Data tunnels adhere to minimum bandwidth congurations and compete with other data tunnels up to the maximum rate.
A virtual tunnel includes all the circuits that are congured within the tunnel. The circuits in the virtual tunnel are called virtual circuits.
When an extension platform is congured to operate in hybrid mode, which supports both FCIP extension and IP extension, the high,
medium, and low data tunnels are created for both protocols. This means a total of seven virtual tunnels are created: the F-Class tunnel,
and three QoS data tunnels for each protocol.
Virtual circuits and tunnels
Brocade Fabric OS Extension Conguration Guide, 8.1.0
53-1004390-03 19
NOTE
The Brocade FX8-24 does not support IP extension. It creates the four FCIP virtual tunnels (1 control and 3 data).
An additional tunnel group is created for high availability (HA) when the hot code load (HCL) feature is congured on the Brocade SX6 or
Brocade 7840. That tunnel group contains a main tunnel, a local backup tunnel, and a remote backup tunnel. When HCL is activated, the
high, medium, and low QoS data tunnels are collapsed into a single QoS data tunnel for the duration of the HCL update. For additional
information, refer to Extension Hot Code Load on page 25.
FCIP Extension
FCIP, or Fiber Channel over IP, is a tunneling protocol to link Fibre Channel over distance on standard IP networks. Used primarily for
remote replication, backup, and storage access, FCIP extension provides Fibre Channel connectivity over IP networks between Fibre
Channel devices or fabrics. The FCIP link is an inter-switch link (ISL) that transports FC control and data frames between switches. FCIP
extension is supported on all Brocade extension platforms.
FCIP extension provides the following:
• Network resiliency using Extension Trunking, which is a single logical tunnel comprised of one or more individual circuits
• Ecient use of VE_Ports
• High performance for high speed WAN links (one or more 10 Gbps and 40 Gbps links)
• TCP Acceleration with WAN Optimized TCP (WO-TCP)
• Aggregation of circuit bandwidth
• Failover/failback
• Failover groups and metrics
• Spillover
• Use of disparate characteristic WAN paths
• Non-disruptive link loss
• Lossless Link Loss (LLL)
• Adaptive Rate Limiting (ARL)
• In-Order Delivery (IOD)
• Deterministic path for protocol acceleration
• WAN bandwidth pooling — pool bandwidth from multiple links/providers
• High speed compression using Deate
• High speed IPsec (AES 256)
• Diagnostic and troubleshooting tools — SAN Health and WAN Tool
• QoS for FCIP with DSCP and/or 802.1P marking and enforcement
• 9216-byte Jumbo Frames support on the Brocade SX6 blade and Brocade 7840 switch
Within the architecture of the Brocade 7840, SX6, and FX8-24, there are Brocade FC Application-Specic Integrated Circuits (ASICs),
which know only FC protocol. The VE/VEX_Ports are not part of the ASICs and are logical representations of actual FC ports. In
actuality, multiple FC ports feed a VE/VEX_Port, permitting high data rates well above 8 gigabits per second (Gbps), 16 Gbps, and 32
Gbps, which is necessary for feeding the compression engine at high data rates and for high-speed trunks. On the WAN side, the
Brocade 7840 and SX6 features 10 Gigabit Ethernet (GbE) and 40 GbE interfaces. The Brocade FX8-24 features 10 GbE and 1 GbE
interfaces. Think of VE and VEX_Ports as the transition point from the FC world to the TCP/IP world inside the extension devices.
FCIP Extension
Brocade Fabric OS Extension Conguration Guide, 8.1.0
20 53-1004390-03
/
