Broadcom BCM56780 Switch Programming User guide

Brand: Broadcom

Model: BCM56780 Switch Programming

Type: User guide

Broadcom 56780-PG101-PUB

August 13, 2020

BCM56780

Switch Programming Guide

Programming Guide

Broadcom, the pulse logo, Connecting everything, Avago Technologies, Avago, the A logo, BroadSync, Flexport, HiGig,

HiGig2, HiGig3, StrataXGS, and XGS are among the trademarks of Broadcom and/or its affiliates in the United States,

certain other countries, and/or the EU.

The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. For more information, please visit www.broadcom.com.

Broadcom reserves the right to make changes without further notice to any products or data herein to improve reliability,

function, or design. Information furnished by Broadcom is believed to be accurate and reliable. However, Broadcom does

not assume any liability arising out of the application or use of this information, nor the application or use of any product or

circuit described herein, neither does it convey any license under its patent rights nor the rights of others.

BCM56780 Programming Guide Switch Programming Guide

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

Table of Contents

Chapter 1: Introduction ......................................................................................................................7

1.1 Purpose and Audience .............................................................................................................................................7

1.2 Device Nomenclature................................................................................................................................................7

1.3 BCM56780 Family......................................................................................................................................................7

Chapter 2: Device Functional Overview ........................................................................................... 8

2.1 SerDes Cores.............................................................................................................................................................8

2.2 iProc ...........................................................................................................................................................................9

2.3 CMICx.........................................................................................................................................................................9

2.4 Packet Buffer.............................................................................................................................................................9

2.5 EP Redirection.........................................................................................................................................................10

Chapter 3: Packet Processing (PP) Pipeline .................................................................................. 11

3.1 Introduction.............................................................................................................................................................11

3.2 Hardware Components...........................................................................................................................................12

3.2.1 Flexible Components ......................................................................................................................................12

3.2.1.1 Flex Parser ...........................................................................................................................................12

3.2.1.2 Flex Editor.............................................................................................................................................13

3.2.1.3 Lookup Tiles .........................................................................................................................................13

3.2.1.4 Bus Architecture ...................................................................................................................................13

3.2.2 Flexible Control Structures..............................................................................................................................13

3.2.3 Special Functions............................................................................................................................................13

3.3 Network Programming Language..........................................................................................................................14

3.4 Compiler...................................................................................................................................................................15

Chapter 4: Device Physical Interfaces ............................................................................................ 16

4.1 Physical Interfaces..................................................................................................................................................16

4.1.1 SyncE..............................................................................................................................................................16

4.1.2 Blackhawk7 (TSC-BH7) SerDes.....................................................................................................................16

4.1.2.1 Flexport.................................................................................................................................................17

4.1.3 Merlin7 (TSC-M7) SerDes ..............................................................................................................................17

4.1.4 PCIe (PCI Express).........................................................................................................................................18

4.1.5 Out-of-Band Flow Control TX (OOBFC-TX)....................................................................................................18

4.1.6 MIIM/MDIO......................................................................................................................................................18

4.1.6.1 MDIO Mapping......................................................................................................................................20

4.1.7 Broadcom Serial Controller.............................................................................................................................24

4.1.8 BroadSync ......................................................................................................................................................24

4.1.9 JTAG...............................................................................................................................................................25

4.1.10 AVS...............................................................................................................................................................26

4.1.10.1 Static/Open Loop AVS........................................................................................................................26

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

4.2 Port Namespace Mapping......................................................................................................................................27

4.2.1 Physical Port...................................................................................................................................................27

4.2.2 Ingress Data Buffer Port .................................................................................................................................27

4.2.3 Device Port .....................................................................................................................................................27

4.2.4 MMU Port........................................................................................................................................................27

4.2.5 Port Number Relationships.............................................................................................................................28

Chapter 5: Buffer Management ....................................................................................................... 29

5.1 Memory Management Unit Overview....................................................................................................................29

5.1.1 Packet Flows...................................................................................................................................................29

5.1.1.1 Data Path..............................................................................................................................................29

5.1.1.2 Control Path..........................................................................................................................................30

5.2 Ingress Admissions Control (THDI) ......................................................................................................................31

5.2.1 THDI Control Parameters ...............................................................................................................................31

5.3 Egress Admissions Control (THDO) .....................................................................................................................32

5.3.1 THDO and Lossless Traffic.............................................................................................................................32

5.3.2 THDO and Lossy Traffic .................................................................................................................................32

Chapter 6: Traffic Management ....................................................................................................... 33

6.1 Scheduler Overview................................................................................................................................................33

6.1.1 Capabilities .....................................................................................................................................................33

6.1.2 Functional Overview .......................................................................................................................................34

6.1.2.1 Scheduler Overview..............................................................................................................................34

6.2 Front-Panel Port Scheduler....................................................................................................................................35

6.2.1 Scheduling Hierarchy......................................................................................................................................35

6.2.2 Scheduling Disciplines....................................................................................................................................35

6.2.3 Scheduling Precedence..................................................................................................................................36

6.3 CPU Port Scheduler................................................................................................................................................37

6.3.1 Scheduling Hierarchy......................................................................................................................................37

6.3.2 Scheduling Disciplines....................................................................................................................................38

6.3.3 Scheduling Precedence..................................................................................................................................38

Chapter 7: Congestion Management .............................................................................................. 39

7.1 System-Level Congestion Management ...............................................................................................................39

7.1.1 Out-of-Band Flow Control (OoBFC)................................................................................................................39

7.1.1.1 Out-of-Band HCFC Interface ................................................................................................................40

7.1.1.2 General Operation................................................................................................................................40

7.2 Network-Level Congestion Management..............................................................................................................41

7.2.1 MMU Support for ECN....................................................................................................................................41

7.2.1.1 IP WRED-ECN Packet Attributes to MMU............................................................................................41

7.2.1.2 Queue Attributes to Enable WRED and ECN ......................................................................................41

7.2.1.3 MMU-to-EP Packet Marking Signaling .................................................................................................41

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

7.2.1.4 DCTCP Support....................................................................................................................................41

7.3 Link Congestion Management...............................................................................................................................41

7.3.1 PAUSE............................................................................................................................................................41

7.3.1.1 PAUSE Operation.................................................................................................................................42

7.3.2 Priority Flow Control........................................................................................................................................42

7.3.2.1 PFC Packet Format..............................................................................................................................43

7.3.2.2 PFC-Tx .................................................................................................................................................43

7.3.2.3 PFC-Rx.................................................................................................................................................44

7.3.2.4 Deadlock Protection..............................................................................................................................45

7.3.3 Weighted Random Early Detection.................................................................................................................45

Chapter 8: Instrumentation .............................................................................................................. 46

8.1 Headroom Pool Monitor .........................................................................................................................................46

8.2 Visibility into Packet Processing...........................................................................................................................47

8.3 Packet Timestamping.............................................................................................................................................48

8.3.1 Features versus Constraints...........................................................................................................................48

8.3.1.1 Features................................................................................................................................................48

8.3.1.2 Constraints............................................................................................................................................49

8.3.2 Network Flows ................................................................................................................................................49

8.3.2.1 Basic End-to-End Flow .........................................................................................................................49

8.3.2.2 Use Case Scenarios.............................................................................................................................50

8.4 Packetized MMU Statistics.....................................................................................................................................51

8.4.1 Terminology ....................................................................................................................................................51

8.4.2 Introduction .....................................................................................................................................................52

8.4.3 High-Level Description....................................................................................................................................52

8.4.4 Switch/Reporter View......................................................................................................................................53

8.4.5 PktStats Agent (Software)...............................................................................................................................54

8.5 Buffer Statistics Tracking.......................................................................................................................................55

8.6 Latency Distribution Histogram.............................................................................................................................56

8.6.1 Feature Description.........................................................................................................................................56

8.7 Switch Utilization Monitor ......................................................................................................................................57

8.8 Oversubscription Buffer Instrumentation.............................................................................................................57

8.9 Packet Sampling (PSAMP) .....................................................................................................................................58

8.10 Dynamic Load Balancing Flow Monitoring.........................................................................................................58

8.11 CPU Masquerade (CPU Trace).............................................................................................................................60

8.12 Mirror-on-Drop (MoD) ...........................................................................................................................................60

8.13 Trace and Drop Event...........................................................................................................................................61

8.14 Flexible Counters and State.................................................................................................................................61

8.15 PFC Monitor...........................................................................................................................................................63

Related Documents ..........................................................................................................................64

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

Glossary ............................................................................................................................................ 65

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

Chapter 1: Introduction

1.1 Purpose and Audience

This document describes the features and architecture of the Broadcom

BCM56780 family of highly integrated Ethernet

switches. This document does not detail electrical specifications or register information. Refer to the appropriate document

in Related Documents.

NOTE: Throughout this document, all descriptions refer to the BCM56780 device. Regardless of feature variance, the

hardware architecture is the same across devices in each family.

1.2 Device Nomenclature

Refer to the individual device data sheets for the latest package or device features. In case of discrepancies between this

document and the BCM56780 data sheet, the data sheet takes precedence.

1.3 BCM56780 Family

The BCM56780 family includes devices that support different I/O bandwidth, throughput, and port configurations. The

BCM56780 family of devices also supports SKUs for flexible programmability. Refer to the BCM56780 data sheet for more

details.

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

Chapter 2: Device Functional Overview

The BCM56780 is the latest generation in the high-bandwidth, feature-rich Broadcom product line. The device has a

maximum I/O bandwidth of 8.0 Tb/s. The device is built to provide extremely high levels of integration, while also reducing

power consumption. The following figure depicts the BCM56780 device functional blocks.

Figure 1: Device Functional Blocks

2.1 SerDes Cores

For external network port and backplane connectivity, the device includes up to 20 Blackhawk7 cores. Each Blackhawk7

core consists of eight high-speed SerDes lanes, each capable of operating at up to 50Gb/s. One or multiple SerDes lanes

can be used to form a logical port. In addition, the device includes one slower speed core, Merlin7, which consists of four

high speed SerDes lanes, each capable of operating at up to 10 Gb/s. This SerDes core can be configured to support up to

two management ports. Each of the management ports can be configured to operate at up to 10GbE. Traffic from these

management ports passes through the same packet-processing pipeline as other front panel ports. Finally, the device

includes a PCIe SerDes core for the host CPU management interface. This core provides four PCIe Gen3.0 compliant lanes

to provide a theoretical maximum management interface bandwidth of 31.5 Gb/s for the high-performance CPU

Management Interface Controller (CMICx).

NOTE: In this document, the terms Blackhawk7 and TSC-BH7 are interchangeable. Additionally, the terms Merlin7 core

and TSC-M7 are interchangeable.

Blackhawk7 10 Blackhawk7 11 Blackhawk7 12 Blackhawk7 17 Blackhawk7 18 Blackhawk7 19

Blackhawk7 0 Blackhawk7 1 Blackhawk7 2 Blackhawk7 7 Blackhawk7 8 Blackhawk7 9

Arm uCX4 PCIe

10 – 400GbE MAC

Multistage

Field Processor

Engine

Packet Buffer

Programmable

Instrumentation

Programmable

Parser, Editor

Programmable

L2 to L4 Processing

Programmable

Tunnels

10 – 400GbE MAC

CMICx

10GbE 25GbE/50GbE

40GbE

100GbE 200GbE 400GbE

Merlin7

Management Ports

(up to two ports

over four lanes)

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

2.2 iProc

The iProc block is a highly integrated subsystem that provides numerous components that reduce system design cost,

increase management-related system performance, and offload common tasks that are normally handled by the host CPU.

iProc provides two Real-Time Subsystems (RTSs), each with two Arm Cortex R5 processors for a total of four R5

processors. Each RTS has dedicated L1 instruction and data caches, as well as low-latency Tightly Coupled Memory (TCM).

Within the RTS, the two R5 processors share a dedicated SRAM. These elements together comprise the RTS, which is used

to provide resources for tasks that require real-time processing, like the Precision Time Protocol (PTP) stack used for

IEEE 1588 applications. These dedicated resources are used to run Broadcom-provided binary firmware applications that

remove the burden of processing from the host CPU. In addition to the Arm Cortex R5s, the iProc also contains four Arm

Cortex M0 processors. These processors are provided for offloading many common tasks that do not require high

performance but are still a resource burden to the host CPU.

Broadcom functionality offloaded to these M0 processors include gathering and acting on link state changes or gathering

and generation of a serial stream for status LEDs.

The PCIe Gen3 SerDes and associated PCIe controller logic are also integrated in the iProc subsystem. The CMICx is also

integrated in the iProc.

iProc also integrates many ancillary functions that provide low-speed communication with the rest of the system to reduce

system design costs. This includes GPIOs, serial communication controllers, and timing-related functions capable of

generating time stamps on events, as well as synthesizing a clock.

The various blocks in the iProc are connected through a high-performance AXI fabric to provide low-latency delivery of

transactions to the various masters and slaves connected to the fabric.

2.3 CMICx

The device integrates a high-performance CPU Management Interface Controller (CMICx) that provides packet DMA and

low-latency direct access to internal switch counters, registers, and memory data. The CMICx is tightly coupled within the

iProc block. Integrating the CMICx into the iProc provides a low-latency interface path between the switching logic and the

control plane.

CMICx is responsible for providing access to all the registers and memories associated with the forwarding logic through

SBus transactions. Additionally, it allows chunks of control memory to be DMA to and from the host memory space through

SBus DMA operations, which facilitates large configuration changes to be done quickly. Finally, the CMICx also contains

logic for allowing the host CPU to inject packets into the switch’s ingress pipeline, or receive packets from the switch’s egress

pipeline. The CMICx block can also support little-endian or big-endian order.

2.4 Packet Buffer

BCM56780 supports hybrid shared MMU architecture, which is a combination of input buffered and shared buffer

architecture.

See Chapter 5, Buffer Management for more information.

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

2.5 EP Redirection

The BCM56780 implements two Redirection Buffer (RDB) ports to support EP redirection; one per packet processing

pipeline. The EP redirection decision is on a per-packet basis. EP redirection allows a packet to be redirected to or copied

from the egress pipeline. With EP redirection, the chip can support the following features:

 True egress mirror

 Inband Flow Analyzer (IFA) egress node deletion functions entirely in the pipeline

 Egress drop monitoring

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

Chapter 3: Packet Processing (PP) Pipeline

3.1 Introduction

The BCM56780 uses a new tile-based architecture for switching chips. It is an innovative and fully compiler programmable

flexible architecture. The flexible components of this architecture are discussed in Section 3.2, Hardware Components.

Because there is significant flexibility in allocating various hardware resources, the desired packet processing behavior is

defined through a program written in a high-level language known as Network Programming Language (NPL).

The chip hardware capabilities are defined by a Hardware Capability File (HCF), which may change depending on the SKU.

This contains a detailed description of the hardware structure of the pipeline.

A compiler-based toolchain translates NPL programs to microcode known as FlexCode. The FlexCode provides mapping

instructions to program the hardware resources (such as parser, editor, tiles, special functions, and flexible switch logic),

which in turn defines the device personality. This configuration is loaded when the switch is first initialized by the

Broadcom-provided SDK. The compiler also provides logical to physical mapping information that is needed by the SDK.

The following figure depicts the flow diagram.

Figure 2: Flex Flow Generation Diagram

Due to the flexible, compiler-driven nature of the device, a logical forwarding table used by the application may be abstracted

into one or more underlying physical tables in the hardware. This abstraction is handled transparently by Broadcom-provided

Logical Table Software Development Kit (SDKLT).

Hardware Capabilities,

Programming Structures

User Forwarding Intent

FlexCode

Hardware Capability

File (HCF)

Logical Regsfile

Compiler

NPL Program

SDKLT Populate

APIs

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

3.2 Hardware Components

The following figure depicts various hardware components in the BCM56780 architecture.

Figure 3: Packet Processing Pipeline

To support the compiler-based configuration flow, the device architecture contains a number of hardware components that

can be broadly classified as follows:

 Flexible components

 Flexible control structures

 Special functions

3.2.1 Flexible Components

Flexible components are configured by the compiler based on the NPL code. Each component includes some of the

following data and control structures:

 Flex Parser

 Flex Editor

 Lookup Tiles

 Bus Architecture

3.2.1.1 Flex Parser

Any flexible packet processing requires flexibility in the parser as a baseline.

Egress Pipelines

MMU

Packet

Buffer

Scheduler

Flex

Editor

Tiles +

Special

Functions

Tiles +

Special

Functions

Ingress Pipelines

Flex

Parser

Tiles +

Special

Functions

Tiles +

Special

Functions

Tiles +

Special

Functions

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

3.2.1.2 Flex Editor

The flex editor enables editing of the packet in a user-configurable way. The editor has a highly parallel architecture. It uses

a set of engines that implement rewrite, delete, and header construction functions. Each engine operates on a section of the

packet. The activities of these engines, as well as the merging of their outputs are coordinated by a control infrastructure.

3.2.1.3 Lookup Tiles

Tiles are a standard building block for accessing memories in BCM56780 PP architecture-based pipelines. They provide a

flexible means of generating lookups to physical tables and a generalized way of resolving the results used by components

later in the processing pipeline. These tiles are broadly classified into three types based on the type of memory.

 Hash: A key is used to compute a hash value that is then used to index into a bucket in the memory. Each entry in the

bucket is then compared with the original key to determine if there is an exact match. An example of a hash-based

memory is the Layer2 database that is used for destination MAC address lookups.

 TCAM: Each entry provides a programmable match value and mask. A key is provided directly as an input to the

memory. For each entry in the memory, the input key is bitwise ANDed with the entry's mask and compared with the

value in the entry. If there is a match, the entry's index number is returned. An example of a TCAM-based memory is the

Longest Prefix Match database that is used for subnet-based IP address lookups.

 Index: This type of memory is directly indexed using a previously generated control as a direct pointer into the memory.

An example of an index-based memory is the VLAN properties database when the VLAN ID associated with the packet

is used to directly index into a table that has up to 4K entries for the default NPL application.

3.2.1.4 Bus Architecture

The PP bus is the main conduit of information between the various components in the pipeline.

3.2.2 Flexible Control Structures

Flexible control structures are the control structures in the pipeline that help coordinate the activities of the flexible

components, and also serve as interfaces to and from the special functions. For more information, see Section 3.2.3, Special

Functions.

3.2.3 Special Functions

The pipeline has several fixed and semi-fixed functions that implement specific features, such as packet hashing,

aggregation, membership checks, meters, and so on. Although each of these have some control knobs, the objects, data

structures, and functionality of each of these cannot be completely defined by the NPL programmer. Many of these involve

complex arithmetic functions, while others involve control plane and other chip components that limit the degree of flexibility

that can be allowed.

For this reason, these are not configured by the NPL compiler, although the compiler does configure control structures to

map flexible bus objects to and from these functions. They are managed directly by the SDK.

Each special function has a set of inputs and outputs that have some predefined semantics for its operation. During packet

processing, certain NPL-defined objects and commands are provided to a given special function as its inputs. Similarly, the

outputs of each special function are mapped back to the objects or commands bus to be consumed at later levels in the

pipeline.

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

In functional terms, behavior of these special functions is similar to those in other Broadcom devices. For each special

function, there are dedicated internal data structures. Each special function is built as a dedicated component or set of

components. Also, many of these special functions are made more generic so that they could be repurposed for use cases

other than their “classic” use cases.

3.3 Network Programming Language

Software defined networks (SDNs) raised new requirements in terms of network management, telemetry, and evolving

overlay protocols. This required a transition from a fixed function packet processing architecture to a programmable packet

processing architecture. This meant that the underlying hardware resources need to be allocated based on the needs of the

SDN application. It also required to tap on the architectural efficiencies of the underlying hardware in terms of latency, power

usage, and area. To address all of these necessities, the Network Programming Language (NPL) was designed. It is unique

in that it allows for advance programmable hardware architecture along with efficiently programming the packet forwarding

data plane. It also includes constructs to express the networking behavior that takes advantage of advanced features of the

underlying programmable hardware.

The language is sophisticated and supports the following features:

 Customized table pipeline

 Intelligent match-action processing

 Parallelism

 Run-time programmability

 Integrated Instrumentation plane

NPL also provides constructs that provide for the inclusion of component libraries that implement fixed function hardware

blocks discussed in Section 3.2.3, Special Functions.

NPL language constructs promote software reuse that help in building a family of switching solutions ranging from simple to

increasingly complex. For more information, see Related Documents.

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

3.4 Compiler

Like any high-level programming language, NPL requires a set of compilers and associated tools to map the programs

written in NPL to target hardware objects. It consists of two parts:

 The Front-End (FE) compiler – It is responsible for checking the syntax and semantics of the user-written program in

the NPL language, along with generating an Intermediate Representation (IR) and the behavioral model. The IR

contains the application view of network and is device independent.

 The Back-End (BE) compiler – It is responsible for mapping the IRs into specific hardware objects. This is composed of

chip component-specific compilers. It also generates an API that the control plane uses to manage the behavior of the

switch.

The following items are outputs of the BE compiler:

 User-defined logical tables

 Logical-to-physical mapping used by the SDKLT API to populate the logical tables

 FlexCode

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

Chapter 4: Device Physical Interfaces

4.1 Physical Interfaces

4.1.1 SyncE

SyncE provides synchronization over Ethernet between nodes using a recovered RX clock from a selected Ethernet port.

The BCM56780 provides a two-wire interface, consisting of the clock and a valid indication to output the selected RX clock

reference. A second two-wire interface is provided to allow the user to select a second Ethernet port recovered clock as a

backup SyncE clock reference.

The asynchronous nature of Ethernet provides certain transmission challenges. For example, Time Division Multiplexing

(TDM) services such as T1/E1 and SONET/SDH require synchronized clocks at both the source and destination nodes.

Similarly, wireless base stations require synchronization to a common clock to ensure a smooth call hand-off between

adjacent cells. While there are several ways to achieve synchronization over Ethernet, one gaining momentum is

Synchronous Ethernet (SyncE). SyncE uses the physical layer interface to pass timing from node-to-node in the same way

timing is passed in SONET/SDH or T1/E1. This gives telecom and wireless providers confidence that networks based on

SyncE are not only cost-effective, but also as highly reliable as SONET/SDH and T1/E1-based networks.

SyncE was standardized by the ITU-T, in cooperation with IEEE, as two recommendations:

 ITU-T Rec. G.8262 specifies Synchronous Ethernet clocks for SyncE.

 ITU-T Rec. G.8264 describes the specification of Ethernet Synchronization Messaging Channel (ESMC).

ITU-T G.8262 defines Synchronous Ethernet clocks compatible with SDH clocks. Synchronous Ethernet clocks, based on

ITU-T G.813 clocks, are defined in terms of accuracy, noise transfer, holdover performance, noise tolerance, and noise

generation. These clocks are referred to as Ethernet Equipment Slave clocks. G.8262/Y.1362 (published 01-2015 and

amended on 11-2016) is an ITU-T recommendation for Synchronous Ethernet that defines “timing characteristics of

synchronous Ethernet equipment slave clock (EEC)”.

The SyncE recovered clocks from various TSC-BH7 SerDes are multiplexed, and one of them is fed into a post-divider

before being sent to an L1 recovered clock chip output pin.

NOTE: The recovered clock output from the device must be cleaned up by an external DPLL before being sent to other

SyncE devices.

4.1.2 Blackhawk7 (TSC-BH7) SerDes

The TDM-based SerDes Controller-Blackhawk7 (TSC-BH7) is the SerDes IP core integrated into the BCM56780 family of

devices. It is an octal SerDes core with various Physical Coding Sublayer (PCS) functions for Ethernet applications. The

intended application is 10/25/40/50/100/200/400-Gigabit high-speed Ethernet connections for backplanes, optics, cable,

and device-to-device communication.

The TSC-BH7 consists of eight SerDes lanes. There can be up to 20 TSC-BH7 instances, depending on the device variant.

NOTE: The terms Blackhawk7 and TSC-BH7 are interchangeable.

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

4.1.2.1 Flexport

The Flexport

™

feature enables the user to dynamically change the switch’s port configuration with minimal impact to other

ports. Changing the port rate or number of SerDes required for a port are Flexport operations. The ports involved in a

Flexport operation are disabled during the change. Traffic on these ports and SerDes ceases, and no packets can be queued

within the chip to/from these ports during the Flexport operation. Other ports that are not involved in the Flexport operation

continue to receive and transmit traffic (with little impact). In addition, bringing up a new port from a currently unused SerDes

is considered a Flexport operation. It implies that each SerDes lane in a TSC-BH7 can support a different speed and the port

speed configuration can be changed on the fly.

4.1.3 Merlin7 (TSC-M7) SerDes

The Merlin7 SerDes provides the port-based management interface for remote switch management and the PMD for the

data ports that are connected using the Port Macro (PM) with embedded PCS to the device control plane.

The Merlin7 SerDes provides the following features:

 Quad 10Gb/s SERDES optimized for Backplane and Front Panel applications.

 Supports line rates of 8.5 to 12.5 Gb/s lower speeds using Over-Sampling Rates (OSR).

 IBIS-AMI simulation model availability.

 MDIO Management Interface: Clause 45-based MDIO.

 8051 Micro-Sub System Module.

The Merlin7 SerDes core is designed for high density integration in high bandwidth products and is optimized for low-power

consumption and area efficiency. The Merlin7 SerDes supports data rates from 1.25 Gb/s to 12.5 Gb/s. Lower speeds are

supported using Over Sampling Rates (OSR).

The Merlin7 PMD core shares a common transmit clock for all lanes and has a single common transmit phase interpolator

(TXPI) for the transmit clock. The Merlin7 PMD core also supports transmit lane muxing.

The Merlin7 SerDes core (PMD) is connected to the Port Macro (PM) that implements the PCS layer. The PM interfaces to

the control plane of the device.

The Merlin7 SerDes Core data path interface is designed to work well with an IEEE PCS or other coding layers for various

high-speed serial link applications. Each Merlin7 SerDes core is connected using a MDIO port to an internal MDIO bus for

management, configuration, control, and debug support. See Section 4.1.6.1, MDIO Mapping for the MDIO address

mapping of each SerDes core.

The Merlin7 SerDes core has a built-in remote loopback mode, digital loopback mode, fixed-pattern generator, and a PRBS

generator and checker to support testing. The core also supports an embedded 8051 micro subsystem that has an

integrated program RAM. This 8051 subsystem is used to implement link training (CL93, CL72), auto-negotiation (AN), link-

quality analysis, and link debug.

The digital functionality, register address mapping, and test features of the base core are designed to be protocol agnostic.

The core provides control and status interfaces that are used by the upper layer to implement standards compliant registers.

Refer to the BCM56780 data sheet for details about the RX (Analog Front End) AFE (Analog RX) and the TX AFE (Analog

TX) including electrical specifications.

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

Each Merlin7 SerDes requires a high-quality reference clock. Refer to the BCM56780 data sheet for the electrical

specifications of the reference clock and the dedicated analog supplies. Refer to IEEE 802.3 Clause 45 for details about

these MDIO standards. Temperature based compensation is used during PLL calibration. The measured temperature is

used as well for Clause 93/72 to calculate how much TXEQ to request to ensure that a future temperature change can be

handled by the receiver. Merlin7 core supports low-power modes and transmit disable to lower the power statically.

4.1.4 PCIe (PCI Express)

The PCIe interface provides the connectivity to the local host CPU for local configuration, management, and control of the

device, as well as for debugging purposes.

For details on the PCIe Protocol, refer to the PCIe specifications published by the PCI-SIG (http://www.pcisig.com).

For electrical characteristics, strapping pin configurations, and operation modes, refer to the BCM56780 data sheet.

The PCIe interface (which supports only the Endpoint role) is compliant to Gen1, Gen2, and Gen3 of the PCIe interface

standard, has a configurable number of lanes (1, 2, 4) as well as a configurable speed per lane (2.5G, 5G, 8G).

4.1.5 Out-of-Band Flow Control TX (OOBFC-TX)

The out-of-band flow control message mechanism is used to transmit local buffer congestion states through an out-of-band

interface. The OOBFC-TX interface is an output interface from the Memory Management Unit (MMU).

BCM56780 has one OOBFC TX interface composed of the following features:

 Transmit Out-of-Band Flow Control interface clock (TX_OOBFC_CLK)

 2-bit parallel data bus (TX_OOBFC_DATA[1:0])

 Sync pulse signal to indicate start of frame (TX_OOB_SYNC)

Out-of-Band HCFC externalizes internal congestion states using four dedicated pins. Both in-band and out-of-band HCFC

use the same message format, although the out-of-band message is only 15B, which omits the K.SOM character as it is not

needed across the OOBFC interface. The BCM56780 supports out-of-band HCFC only. The OOBFC interface pins are

described in the table below.

NOTE: Refer to the BCM56780 data sheet for the interface timing and electrical characteristics of the OOB-TX interface.

4.1.6 MIIM/MDIO

The twelve independent MDIO Master Mode-Only interfaces (MDIO chains) support Clause (CL45) with different IO voltage

levels (1.2V) as well as adjustable MDIO output delay, which is used for management and control of SerDes and external

PHYs, as well as access of the PHY registers for debugging purposes.

Internally, there are twelve corresponding MDIO chains used to access the internal SerDes cores.

Table 1: OOBFC Interface Pins

Pin Name Pin Description

OOBFC_CLK 125 MHz clock output

OOBFC_DATA0 Data output, least significant bit

OOBFC_DATA1 Data output, most significant bit

OOBFC_SYNC Sync pulse output, asserted at the beginning of each OOB HCFC message

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

In terms of MDIO protocol (MDIO frame formats), all network SerDes cores support CL45 regardless of MDIO electrical

compliance.

For functional purposes, the MDIO interfaces can be configured as the MDIO master interface. Software can access external

PHY devices through these MDIO interfaces using iProc programming registers. MDIO access to internal SerDes can also

be done through iProc register programming without using MDIO chip IO interfaces, but it is strongly not recommended

unless used for DVT/debug purposes.

Broadcom 56780-PG101-PUB

BCM56780 Programming Guide Switch Programming Guide

4.1.6.1 MDIO Mapping

The following table shows the MDIO mapping of the device. The device has twelve MDIO chains (MDIO0 ... MDIO11). Each

chain provides MDIO access to multiple TSC or SerDes cores using iProc.

Table 2: Device MDIO Mapping

SerDes SerDes Core Lane Port Block Internal MDIO Bus # Internal MDIO Addr

(Blackhawk7) TSC0 0 PM0 0 0x1 (1)

100x2 (2)

200x3 (3)

300x4 (4)

400x5 (5)

500x6 (6)

600x7 (7)

700x8 (8)

(Blackhawk7) TSC1 0 PM1 0 0x9 (9)

100xa (10)

2 0 0xb (11)

3 0 0xc (12)

400xd (13)

500xe (14)

6 0 0xf (15)

7 0 0x10 (16)

(Blackhawk7) TSC2 0 PM2 1 0x1 (1)

110x2 (2)

210x3 (3)

310x4 (4)

410x5 (5)

510x6 (6)

610x7 (7)

710x8 (8)

(Blackhawk7) TSC3 0 PM3 1 0x9 (9)

110xa (10)

2 1 0xb (11)

3 1 0xc (12)

410xd (13)

510xe (14)

6 1 0xf (15)

7 1 0x10 (16)

(Blackhawk7) TSC4 0 PM4 2 0x1 (1)

120x2 (2)

220x3 (3)

320x4 (4)

420x5 (5)

520x6 (6)

620x7 (7)

Page is loading ...

Broadcom BCM56780 Switch Programming User guide

Broadcom BCM56780 Switch Programming User guide

Broadcom BCM56780 Switch Programming User guide

Related papers

Other documents