Broadcom BCM88690, BCM88800, BCM88480, and BCM88280 Traffic Manager User guide

Broadcom 88690-PG207-PUB

September 8, 2020

BCM88690

Traffic Manager

Programming Guide

BCM88690

BCM88800

BCM88480

BCM88280

Broadcom, the pulse logo, Connecting everything, Avago Technologies, Avago, the A logo, BroadSync, and StrataDNX are

among the trademarks of Broadcom and/or its affiliates in the United States, certain other countries, and/or the EU.

The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. For more information, please visit www.broadcom.com.

Broadcom reserves the right to make changes without further notice to any products or data herein to improve reliability,

function, or design. Information furnished by Broadcom is believed to be accurate and reliable. However, Broadcom does

not assume any liability arising out of the application or use of this information, nor the application or use of any product or

circuit described herein, neither does it convey any license under its patent rights nor the rights of others.

BCM88690 Programming Guide Traffic Manager

Broadcom 88690-PG207-PUB

2

Broadcom 88690-PG207-PUB

3

BCM88690 Programming Guide Traffic Manager

Table of Contents

Chapter 1: SDK General Concepts .................................................................................................. 17

1.1 BCM APIs.................................................................................................................................................................17

1.1.1 API Return Values ..........................................................................................................................................17

1.1.2 BCM API Objects – GPORTs .........................................................................................................................17

1.2 SOC Properties........................................................................................................................................................19

1.2.1 Default Configuration File ...............................................................................................................................19

1.3 Shell Commands.....................................................................................................................................................20

1.4 Application Reference............................................................................................................................................20

1.4.1 CINT Reference ..............................................................................................................................................20

1.4.2 Default Application Reference ........................................................................................................................20

1.4.3 Tune Reference ..............................................................................................................................................21

1.5 Device-Family Differences .....................................................................................................................................21

Chapter 2: Initialization Template ...................................................................................................22

2.1 Reference.................................................................................................................................................................22

2.2 Driver Reference .....................................................................................................................................................22

2.2.1 APIs ................................................................................................................................................................22

2.2.2 SOC Properties ...............................................................................................................................................28

2.2.3 Shell Commands.............................................................................................................................................28

2.2.4 Configuration Flow ..........................................................................................................................................29

2.2.5 Application Reference.....................................................................................................................................30

2.2.5.1 Enable or Disable Applications .............................................................................................................31

2.2.5.2 Typical Traffic Manager Reference Applications ..................................................................................31

2.2.5.3 Change Application Attributes ..............................................................................................................31

2.3 Device-Family Differences .....................................................................................................................................32

Chapter 3: Hardware Interfaces .......................................................................................................33

3.1 Network Interface....................................................................................................................................................33

3.1.1 Theory of Operation ........................................................................................................................................33

3.1.1.1 Logical Port Mapping ............................................................................................................................33

3.1.1.2 Constraints............................................................................................................................................33

3.1.2 Ethernet Ports .................................................................................................................................................33

3.1.2.1 Port Indexing.........................................................................................................................................33

3.1.3 Driver Reference.............................................................................................................................................41

3.1.3.1 APIs ......................................................................................................................................................41

3.1.3.2 SOC Properties.....................................................................................................................................50

3.1.3.3 Shell Commands ..................................................................................................................................52

3.1.3.4 Configuration Flow................................................................................................................................52

3.1.3.5 Application Reference...........................................................................................................................52

Broadcom 88690-PG207-PUB

4

BCM88690 Programming Guide Traffic Manager

3.1.4 Interlaken ........................................................................................................................................................53

3.1.4.1 ILKN Rates ...........................................................................................................................................53

3.1.4.2 ILKN FEC..............................................................................................................................................53

3.1.4.3 ILKN Lane Assignment and Mapping ...................................................................................................54

3.1.4.4 Driver Reference...................................................................................................................................54

3.1.4.5 Interlaken over Fabric ...........................................................................................................................56

3.1.4.6 External Lookup Interface Ports ...........................................................................................................56

3.1.5 Statistics Counters ..........................................................................................................................................57

3.1.5.1 Driver Reference...................................................................................................................................58

3.1.6 LSS Fault Indications ......................................................................................................................................59

3.1.6.1 Driver Reference...................................................................................................................................59

3.1.7 NIF Priority ......................................................................................................................................................60

3.1.7.1 Driver Reference...................................................................................................................................60

3.1.8 RX Timestamp ................................................................................................................................................61

3.1.8.1 Driver Reference...................................................................................................................................61

3.1.9 FlexE Interface................................................................................................................................................62

3.1.9.1 FlexE Physical Interface .......................................................................................................................62

3.1.9.2 FlexE Group..........................................................................................................................................62

3.1.9.3 FlexE Client ..........................................................................................................................................62

3.1.9.4 FlexE Virtual Client ...............................................................................................................................62

3.1.9.5 FlexE System Modes............................................................................................................................63

3.1.9.6 Driver Reference...................................................................................................................................63

3.1.9.7 APIs ......................................................................................................................................................63

3.1.9.8 SOC Properties.....................................................................................................................................74

3.1.10 Device-Specific Differences ..........................................................................................................................75

3.2 Fabric Interface .......................................................................................................................................................75

3.2.1 Link Indexing...................................................................................................................................................75

3.2.1.1 Driver Reference...................................................................................................................................75

3.2.2 Fabric Rates....................................................................................................................................................77

3.2.2.1 Driver Reference...................................................................................................................................77

3.2.3 PCS Layer Line Coding ..................................................................................................................................77

3.2.3.1 Driver Reference...................................................................................................................................77

3.2.4 Power Consumption........................................................................................................................................79

3.2.4.1 Driver Reference...................................................................................................................................79

3.3 PHY...........................................................................................................................................................................80

3.3.1 PHY Constraints .............................................................................................................................................80

3.3.1.1 Blackhawk Constraints .........................................................................................................................80

3.3.1.2 Falcon Constraints................................................................................................................................80

3.3.2 Driver Reference.............................................................................................................................................80

3.3.2.1 APIs ......................................................................................................................................................80

3.3.2.2 SOC Properties.....................................................................................................................................84

Broadcom 88690-PG207-PUB

5

BCM88690 Programming Guide Traffic Manager

3.3.2.3 Shell Commands ..................................................................................................................................85

3.3.3 Lane Map ........................................................................................................................................................86

3.3.3.1 API ........................................................................................................................................................87

3.3.3.2 ILKN Lane Mapping ..............................................................................................................................89

3.3.3.3 Lane Mapping Usage Examples ...........................................................................................................91

3.3.4 Device Family-Specific Differences ................................................................................................................94

3.4 Linkscan...................................................................................................................................................................95

3.4.1 Driver Reference.............................................................................................................................................95

3.4.1.1 APIs ......................................................................................................................................................95

3.4.1.2 SOC Properties.....................................................................................................................................96

3.4.1.3 Shell Commands ..................................................................................................................................98

3.4.1.4 Application Reference...........................................................................................................................98

3.5 Synchronous Ethernet............................................................................................................................................98

3.5.1 Driver Reference.............................................................................................................................................98

3.5.1.1 APIs ......................................................................................................................................................98

3.5.1.2 SOC Properties.....................................................................................................................................99

3.6 DRAM .....................................................................................................................................................................100

3.6.1 Reference .....................................................................................................................................................100

3.6.2 Theory of Operation ......................................................................................................................................100

3.6.3 Temperature Monitor ....................................................................................................................................100

3.6.4 Driver Reference...........................................................................................................................................101

3.6.4.1 APIs ....................................................................................................................................................101

3.6.4.2 SOC Properties...................................................................................................................................102

3.6.4.3 Configuration Flow..............................................................................................................................104

3.6.4.4 Shell Commands ................................................................................................................................105

3.6.5 Tune..............................................................................................................................................................105

3.6.6 Device-Family Differences ............................................................................................................................106

3.7 Packet Reception and Transmission ..................................................................................................................107

3.7.1 BCM Packet Structure ..................................................................................................................................107

3.7.1.1 Structure bcm_pkt_blk_t .....................................................................................................................107

3.7.1.2 Metadata.............................................................................................................................................107

3.7.1.3 Dune Special Metadata ......................................................................................................................108

3.7.1.4 BCM Packet Flags ..............................................................................................................................109

3.7.2 Packet Receive and Transmit Functions ......................................................................................................110

3.7.2.1 APIs ....................................................................................................................................................110

3.7.2.2 SOC Properties...................................................................................................................................123

3.7.3 Receiving Packets ........................................................................................................................................123

3.7.4 Sending Packets ...........................................................................................................................................124

3.7.5 Device-Family Differences ............................................................................................................................124

Chapter 4: Data Interfaces and Ports ............................................................................................ 125

Broadcom 88690-PG207-PUB

6

BCM88690 Programming Guide Traffic Manager

4.1 Physical Ports .......................................................................................................................................................125

4.2 Physical Interface..................................................................................................................................................125

4.3 Logical Port and Local Port .................................................................................................................................126

4.4 TM Port...................................................................................................................................................................126

4.5 System Port ...........................................................................................................................................................126

4.6 Module Port ...........................................................................................................................................................126

4.7 Egress Replication Port........................................................................................................................................126

4.8 Multiple FAP-IDs per Core....................................................................................................................................127

4.8.1 FAP-ID Allocation..........................................................................................................................................127

4.8.2 Mapping a System Port to Module Port ........................................................................................................127

4.8.3 Mapping a VOQ Connector to VOQ..............................................................................................................127

4.9 Driver Reference ...................................................................................................................................................128

4.9.1 APIs ..............................................................................................................................................................128

4.9.1.1 Port Provisioning.................................................................................................................................128

4.9.1.2 System and Module Ports...................................................................................................................128

4.10 Device-Family Differences .................................................................................................................................128

Chapter 5: Port Provisioning ......................................................................................................... 129

5.1 Driver Reference ...................................................................................................................................................130

5.1.1 APIs ..............................................................................................................................................................130

5.1.1.1 Port Add and Remove ........................................................................................................................130

5.1.2 SOC Properties .............................................................................................................................................133

5.1.2.1 Adding Ports .......................................................................................................................................133

5.1.3 Shell Commands...........................................................................................................................................135

5.1.4 Configuration Flow ........................................................................................................................................136

5.1.5 Application Reference...................................................................................................................................137

Chapter 6: Trunk (Link Aggregation) Ports .................................................................................. 139

6.1 Theory of Operation..............................................................................................................................................139

6.1.1 Trunk LB-Key and PSC.................................................................................................................................139

6.1.2 Trunk PP Ports..............................................................................................................................................139

6.1.3 Trunks as Multicast Group Members ............................................................................................................140

6.1.3.1 Trunk as Ingress MC Members ..........................................................................................................140

6.1.3.2 Trunk as Egress MC Members ...........................................................................................................140

6.1.4 Trunk Member Flags.....................................................................................................................................140

6.1.5 Virtual Trunks................................................................................................................................................141

6.2 Driver Reference ...................................................................................................................................................141

6.2.1 APIs ..............................................................................................................................................................141

6.2.2 SOC Properties .............................................................................................................................................146

6.2.3 Application Reference...................................................................................................................................146

Chapter 7: Ingress Traffic Management ....................................................................................... 147

Broadcom 88690-PG207-PUB

7

BCM88690 Programming Guide Traffic Manager

7.1 Queue Creation .....................................................................................................................................................148

7.1.1 VOQs ............................................................................................................................................................148

7.1.2 FMQs ............................................................................................................................................................148

7.1.3 Driver Reference...........................................................................................................................................148

7.1.3.1 APIs ....................................................................................................................................................148

7.1.3.2 SOC Properties...................................................................................................................................153

7.1.3.3 Shell Commands ................................................................................................................................153

7.1.3.4 Configuration Flow..............................................................................................................................153

7.1.3.5 Application Reference.........................................................................................................................155

7.2 Queue Scheduling.................................................................................................................................................156

7.2.1 Driver Reference...........................................................................................................................................157

7.2.1.1 APIs ....................................................................................................................................................157

7.2.1.2 SOC Properties...................................................................................................................................165

7.2.1.3 Shell Commands ................................................................................................................................165

7.2.1.4 Configuration Flow..............................................................................................................................165

7.2.1.5 Application Reference.........................................................................................................................165

7.3 Queue Statistics Generation................................................................................................................................166

7.3.1 Reference .....................................................................................................................................................166

7.3.2 Driver Reference...........................................................................................................................................166

7.3.2.1 APIs ....................................................................................................................................................166

7.3.2.2 Shell Commands ................................................................................................................................167

7.4 Ingress Congestion Management........................................................................................................................167

7.4.1 Reference .....................................................................................................................................................168

7.5 VOQ Ingress Congestion Management...............................................................................................................168

7.5.1 Reference .....................................................................................................................................................168

7.5.2 Driver Reference...........................................................................................................................................168

7.5.2.1 APIs ....................................................................................................................................................169

7.5.2.2 SOC Properties...................................................................................................................................176

7.5.2.3 Shell Commands ................................................................................................................................176

7.5.2.4 Configuration Flow..............................................................................................................................176

7.5.2.5 Application Reference.........................................................................................................................177

7.6 Virtual Statistics Queues......................................................................................................................................177

7.6.1 Reference .....................................................................................................................................................177

7.6.2 Driver Reference...........................................................................................................................................177

7.6.2.1 APIs ....................................................................................................................................................177

7.6.2.2 SOC Properties...................................................................................................................................188

7.6.2.3 Shell Commands ................................................................................................................................188

7.6.2.4 Application Reference.........................................................................................................................188

7.7 System RED...........................................................................................................................................................189

7.7.1 Reference .....................................................................................................................................................189

Broadcom 88690-PG207-PUB

8

BCM88690 Programming Guide Traffic Manager

7.7.2 Driver Reference...........................................................................................................................................189

7.7.2.1 APIs ....................................................................................................................................................189

7.7.2.2 SOC Properties...................................................................................................................................192

7.7.2.3 Shell Commands ................................................................................................................................192

7.8 Device-Level Ingress Congestion Management.................................................................................................192

7.8.1 Driver Reference...........................................................................................................................................192

7.8.1.1 APIs ....................................................................................................................................................192

7.9 Drop Precedence...................................................................................................................................................194

7.9.1 Driver Reference...........................................................................................................................................194

7.9.1.1 APIs ....................................................................................................................................................194

7.10 Ingress Compensation .......................................................................................................................................195

7.10.1 Reference ...................................................................................................................................................195

7.10.2 Driver Reference.........................................................................................................................................195

7.10.2.1 APIs ..................................................................................................................................................195

7.10.2.2 Application Reference.......................................................................................................................197

7.11 Ingress Shapers ..................................................................................................................................................198

7.11.1 APIs ............................................................................................................................................................198

7.12 Priority Drop ........................................................................................................................................................199

7.12.1 Reference ...................................................................................................................................................200

7.12.2 Driver Reference.........................................................................................................................................200

7.12.2.1 APIs ..................................................................................................................................................200

7.13 Mirror-On-Drop (Dropped Packet Reports).......................................................................................................209

7.13.1 Reference ...................................................................................................................................................209

7.13.2 Driver Reference.........................................................................................................................................210

7.13.2.1 APIs ..................................................................................................................................................210

7.13.3 Configuration Sequence .............................................................................................................................211

7.13.4 Application Reference.................................................................................................................................211

7.14 Device-Family Differences .................................................................................................................................211

Chapter 8: Multicast .......................................................................................................................212

8.1 Theory of Operation..............................................................................................................................................212

8.2 Multicast Mechanisms..........................................................................................................................................212

8.3 Driver Reference ...................................................................................................................................................213

8.3.1 APIs ..............................................................................................................................................................213

8.3.2 SOC Properties .............................................................................................................................................217

8.3.3 Shell Commands...........................................................................................................................................218

8.3.4 Configuration Flow ........................................................................................................................................218

8.3.5 Application Reference...................................................................................................................................220

8.4 Device-Family Differences ...................................................................................................................................220

Chapter 9: Fabric Data Path ...........................................................................................................221

9.1 Fabric Pipes...........................................................................................................................................................221

Broadcom 88690-PG207-PUB

9

BCM88690 Programming Guide Traffic Manager

9.1.1 Driver Reference...........................................................................................................................................221

9.1.1.1 SOC Properties...................................................................................................................................221

9.2 Fabric Priority........................................................................................................................................................223

9.2.1 Driver Reference...........................................................................................................................................224

9.2.1.1 APIs ....................................................................................................................................................224

9.2.1.2 Configuration Flow..............................................................................................................................224

9.3 Mesh Configuration ..............................................................................................................................................225

9.3.1 Driver Reference...........................................................................................................................................225

9.3.1.1 APIs ....................................................................................................................................................225

9.3.1.2 SOC Properties...................................................................................................................................227

9.3.1.3 Configuration Flow..............................................................................................................................227

9.4 Fabric Routing.......................................................................................................................................................228

9.4.1 Driver Reference...........................................................................................................................................228

9.4.1.1 APIs ....................................................................................................................................................228

9.5 Fabric Congestion Management..........................................................................................................................230

9.5.1 Driver Reference...........................................................................................................................................231

9.5.1.1 APIs ....................................................................................................................................................231

9.6 Fabric Pipes Scheduling (WFQs).........................................................................................................................232

9.6.1 Driver Reference...........................................................................................................................................232

9.6.1.1 APIs ....................................................................................................................................................232

9.7 Fabric Device Isolation.........................................................................................................................................234

9.7.1 Driver Reference...........................................................................................................................................234

9.7.1.1 APIs ....................................................................................................................................................234

9.8 0: Isolate device.Device Family-Specific Differences........................................................................................234

Chapter 10: Egress Traffic Management ......................................................................................235

10.1 Egress Transmit Scheduler................................................................................................................................235

10.2 Reference.............................................................................................................................................................235

10.3 Port and Interface Shaping ................................................................................................................................236

10.3.1 Driver Reference .........................................................................................................................................236

10.3.1.1 APIs ..................................................................................................................................................236

10.3.1.2 Shell Commands ..............................................................................................................................237

10.3.1.3 Configuration Flow............................................................................................................................238

10.4 Priority Propagation (Queue Pair Priority)........................................................................................................238

10.4.1 Driver Reference .........................................................................................................................................239

10.4.1.1 APIs ..................................................................................................................................................239

10.4.1.2 Configuration Flow............................................................................................................................239

10.5 Header Compensation........................................................................................................................................240

10.5.1 Driver Reference .........................................................................................................................................240

10.5.1.1 Shell Commands ..............................................................................................................................240

10.6 Egress Port Scheduler........................................................................................................................................240

Broadcom 88690-PG207-PUB

10

BCM88690 Programming Guide Traffic Manager

10.6.1 Driver Reference .........................................................................................................................................241

10.6.1.1 APIs ..................................................................................................................................................241

10.6.1.2 SOC Properties.................................................................................................................................241

10.6.2 8P-Port Scheduler.......................................................................................................................................242

10.6.3 Driver Reference .........................................................................................................................................242

10.6.3.1 APIs ..................................................................................................................................................242

10.6.3.2 Configuration Flow............................................................................................................................245

10.6.4 4P-Port Scheduler.......................................................................................................................................245

10.6.4.1 Driver Reference...............................................................................................................................246

10.6.5 2P-Port Scheduler.......................................................................................................................................246

10.6.6 Driver Reference .........................................................................................................................................247

10.6.6.1 APIs ..................................................................................................................................................247

10.6.6.2 SOC Properties.................................................................................................................................247

10.6.6.3 Shell Commands ..............................................................................................................................247

10.6.6.4 Configuration Flow............................................................................................................................248

10.6.7 1P-Port Scheduler.......................................................................................................................................248

10.6.8 Driver Reference .........................................................................................................................................248

10.6.8.1 APIs ..................................................................................................................................................248

10.6.8.2 SOC Properties.................................................................................................................................248

10.6.8.3 Shell Commands ..............................................................................................................................248

10.6.8.4 Configuration Flow............................................................................................................................249

10.7 Queue-Pair Scheduler.........................................................................................................................................249

10.7.1 Driver Reference .........................................................................................................................................249

10.7.1.1 APIs ..................................................................................................................................................249

10.8 Advanced Egress Traffic Management.............................................................................................................250

10.8.1 Port Enable/Disable ....................................................................................................................................250

10.8.2 Driver Reference .........................................................................................................................................250

10.8.2.1 APIs ..................................................................................................................................................250

10.9 Recycling Interfaces...........................................................................................................................................252

10.9.1 Driver Reference .........................................................................................................................................252

10.9.1.1 APIs ..................................................................................................................................................252

10.10 Egress Congestion Management ....................................................................................................................254

10.10.1 Theory of Operation ..................................................................................................................................254

10.10.1.1 Global Traffic Resources ................................................................................................................254

10.10.1.2 Unicast Traffic Resources...............................................................................................................254

10.10.1.3 Multicast Resource Management ...................................................................................................255

10.10.2 Driver Reference.......................................................................................................................................255

10.10.2.1 API ..................................................................................................................................................256

10.10.2.2 SOC Properties...............................................................................................................................257

10.10.2.3 Shell Commands ............................................................................................................................258

Broadcom 88690-PG207-PUB

11

BCM88690 Programming Guide Traffic Manager

10.10.2.4 Application Reference.....................................................................................................................258

10.11 Egress Port and CoS Mapping.........................................................................................................................263

10.11.1 Driver Reference.......................................................................................................................................263

10.11.1.1 APIs ................................................................................................................................................263

10.12 Mapping Port Priority (Queue Pair) to Multicast PD Service Pools..............................................................265

10.13 Device-Family Differences ...............................................................................................................................266

Chapter 11: Egress (End-to-End) Credit Scheduler .................................................................... 267

11.1 Credit Generator..................................................................................................................................................267

11.1.1 Driver Reference .........................................................................................................................................268

11.1.1.1 APIs ..................................................................................................................................................268

11.2 Core-Port Scheduler ...........................................................................................................................................268

11.2.1 Driver Reference .........................................................................................................................................268

11.2.1.1 APIs ..................................................................................................................................................268

11.2.1.2 Configuration Flow............................................................................................................................269

11.3 Port-TC Scheduler...............................................................................................................................................269

11.3.1 Driver Reference .........................................................................................................................................270

11.3.1.1 APIs ..................................................................................................................................................270

11.3.1.2 SOC Properties.................................................................................................................................272

11.3.1.3 Configuration Flow............................................................................................................................272

11.3.1.4 Shell Commands ..............................................................................................................................272

11.4 Scheduler Flows..................................................................................................................................................273

11.4.1 Aggregate Schedulers ................................................................................................................................273

11.4.2 VOQ Connectors.........................................................................................................................................273

11.4.3 Driver Reference .........................................................................................................................................273

11.4.3.1 APIs ..................................................................................................................................................273

11.4.3.2 SOC Properties.................................................................................................................................284

11.4.3.3 Shell Commands ..............................................................................................................................285

11.4.3.4 Configuration Flow............................................................................................................................285

11.4.3.5 Application Reference.......................................................................................................................288

11.5 Port Priority Propagation ...................................................................................................................................288

11.5.1 Driver Reference .........................................................................................................................................289

11.5.1.1 APIs ..................................................................................................................................................289

11.5.1.2 SOC Properties.................................................................................................................................289

11.6 Egress Replication Port......................................................................................................................................289

11.6.1 Driver Reference .........................................................................................................................................289

11.6.1.1 SOC Properties.................................................................................................................................289

11.6.1.2 Configuration Flow............................................................................................................................290

11.7 Fabric Multicast Scheduling Hierarchy.............................................................................................................291

11.7.1 Driver Reference .........................................................................................................................................291

11.7.1.1 APIs ..................................................................................................................................................291

Broadcom 88690-PG207-PUB

12

BCM88690 Programming Guide Traffic Manager

11.7.1.2 Configuration Flow............................................................................................................................291

11.8 Scheduling Hierarchy Modification During Traffic ..........................................................................................292

11.9 Device-Family Differences .................................................................................................................................292

Chapter 12: TDM and OTN Application ......................................................................................... 293

12.1 Theory of Operation............................................................................................................................................293

12.1.1 Bypass Path................................................................................................................................................293

12.1.2 Packet Path.................................................................................................................................................294

12.1.3 Direct Fabric Routing ..................................................................................................................................294

12.1.4 TDM Mesh MC............................................................................................................................................294

12.2 Driver Reference .................................................................................................................................................294

12.2.1 APIs ............................................................................................................................................................294

12.2.1.1 TDM Cell Size...................................................................................................................................294

12.2.1.2 TDM Interface Configuration.............................................................................................................294

12.2.1.3 TDM Ingress Settings .......................................................................................................................295

12.2.1.4 TDM Egress Settings........................................................................................................................297

12.2.2 SOC Properties...........................................................................................................................................298

12.2.3 Detailed Configuration Flow........................................................................................................................299

Chapter 13: Flow-Control Configuration ...................................................................................... 301

13.1 Theory of Operation............................................................................................................................................301

13.1.1 Flow Control Path Mapping.........................................................................................................................301

13.1.2 PFC Deadlock Detection and Recovery .....................................................................................................302

13.2 Driver Reference .................................................................................................................................................303

13.2.1 APIs ............................................................................................................................................................303

13.2.2 SoC Properties............................................................................................................................................313

13.2.3 Shell Commands.........................................................................................................................................314

13.2.4 Configuration Flow ......................................................................................................................................314

13.2.4.1 FC Path Mapping..............................................................................................................................314

13.2.4.2 PFC Deadlock Detection and Recovery ...........................................................................................315

13.2.5 Application Reference.................................................................................................................................315

Chapter 14: Sniff ............................................................................................................................. 316

14.1 Theory of Operation............................................................................................................................................316

14.2 Driver Reference .................................................................................................................................................317

14.2.1 APIs ............................................................................................................................................................317

14.2.1.1 SOC Properties.................................................................................................................................321

14.2.1.2 Shell Commands ..............................................................................................................................321

14.2.1.3 Configuration Flow............................................................................................................................321

14.2.1.4 Application Reference.......................................................................................................................321

14.3 Device-Family Differences .................................................................................................................................321

Chapter 15: Counter Processor .....................................................................................................322

Broadcom 88690-PG207-PUB

13

BCM88690 Programming Guide Traffic Manager

15.1 Theory of Operation............................................................................................................................................322

15.2 Counter Processor Configuration Steps ..........................................................................................................323

15.2.1 Expansion Configuration.............................................................................................................................323

15.2.1.1 Driver Reference...............................................................................................................................323

15.2.2 Database Configuration ..............................................................................................................................325

15.2.2.1 Attaching Engines.............................................................................................................................325

15.2.2.2 Database Properties .........................................................................................................................325

15.2.3 Driver Reference .........................................................................................................................................325

15.2.3.1 SC Generation Points – References.................................................................................................325

15.2.3.2 Multicast............................................................................................................................................326

15.2.4 Engine Eviction Configuration.....................................................................................................................333

15.2.4.1 Driver Reference...............................................................................................................................333

15.2.4.2 Shell Commands ..............................................................................................................................336

15.2.4.3 Configuration Flow............................................................................................................................336

15.2.4.4 Database Destroy .............................................................................................................................337

15.2.4.5 Reduce Database Range .................................................................................................................337

15.2.4.6 Extend Database Range...................................................................................................................337

15.2.4.7 Application Reference.......................................................................................................................337

15.3 Reading Counters...............................................................................................................................................338

15.3.1 Theory of Operation ....................................................................................................................................338

15.3.2 Driver Reference .........................................................................................................................................339

15.3.2.1 APIs ..................................................................................................................................................340

15.3.2.2 SOC Properties.................................................................................................................................342

15.3.2.3 Shell Commands ..............................................................................................................................342

15.3.2.4 Application Reference.......................................................................................................................343

15.4 Counter Packet Size Compensation..................................................................................................................343

15.4.1 Reference ...................................................................................................................................................343

15.4.2 Theory of Operation ....................................................................................................................................343

15.4.3 Driver Reference .........................................................................................................................................344

15.4.3.1 APIs ..................................................................................................................................................344

15.5 Counter Drop Reason Filtering and Grouping.................................................................................................345

15.5.1 Reference ...................................................................................................................................................345

15.5.2 Theory of Operation ....................................................................................................................................345

15.5.3 Driver Reference .........................................................................................................................................346

15.5.3.1 APIs ..................................................................................................................................................346

15.6 Counter Handling During Warm Boot ...............................................................................................................348

15.6.1 Theory of Operation ....................................................................................................................................348

15.7 Device-Family Differences .................................................................................................................................348

Chapter 16: Metering ...................................................................................................................... 349

16.1 Theory of Operation............................................................................................................................................349

Broadcom 88690-PG207-PUB

14

BCM88690 Programming Guide Traffic Manager

16.1.1 Mapping Object-ID to Specific Generic Metering Index ..............................................................................350

16.2 Ingress Global Meter...........................................................................................................................................351

16.2.1 Theory of Operation ....................................................................................................................................351

16.2.2 Driver Reference .........................................................................................................................................351

16.2.2.1 APIs ..................................................................................................................................................352

16.2.2.2 SOC Properties.................................................................................................................................353

16.2.2.3 Shell Commands ..............................................................................................................................353

16.2.2.4 Configuration Flow............................................................................................................................354

16.2.2.5 Application Reference.......................................................................................................................354

16.3 Generic Meter Database (Ingress and Egress).................................................................................................354

16.3.1 Theory of Operation ....................................................................................................................................354

16.3.2 Driver Reference .........................................................................................................................................356

16.3.2.1 APIs ..................................................................................................................................................356

16.3.2.2 SOC Properties.................................................................................................................................359

16.3.2.3 Shell Commands ..............................................................................................................................359

16.3.2.4 Configuration Flow............................................................................................................................359

16.3.2.5 Application Reference.......................................................................................................................361

16.4 Meter Properties..................................................................................................................................................361

16.4.1 Theory of Operation ....................................................................................................................................361

16.4.1.1 Metering Modes ................................................................................................................................362

16.4.1.2 Rate Dependencies ..........................................................................................................................362

16.4.1.3 Meter Burst Size ...............................................................................................................................362

16.4.1.4 policer_id Settings ............................................................................................................................362

16.4.2 Driver Reference .........................................................................................................................................362

16.4.2.1 APIs ..................................................................................................................................................363

16.4.2.2 SOC Properties.................................................................................................................................365

16.4.2.3 Shell Commands ..............................................................................................................................366

16.4.2.4 Configuration Flow............................................................................................................................366

16.4.2.5 Application Reference.......................................................................................................................366

16.5 Resolution Tables ...............................................................................................................................................366

16.5.1 Theory of Operation ....................................................................................................................................366

16.5.2 Driver Reference .........................................................................................................................................368

16.5.2.1 SOC Properties.................................................................................................................................370

16.5.2.2 Shell Commands ..............................................................................................................................370

16.5.2.3 Configuration Flow............................................................................................................................370

16.5.2.4 Application Reference.......................................................................................................................371

16.6 Meter Packet Size Adjustment...........................................................................................................................371

16.6.1 Theory of Operation ....................................................................................................................................371

16.6.2 Driver Reference .........................................................................................................................................371

16.6.2.1 APIs ..................................................................................................................................................371

16.7 Meter Pointer Generation ...................................................................................................................................372

Broadcom 88690-PG207-PUB

15

BCM88690 Programming Guide Traffic Manager

16.7.1 Theory of Operation ....................................................................................................................................372

16.7.1.1 Application Reference.......................................................................................................................373

16.8 Device Actions Based on Meter.........................................................................................................................373

16.8.1 Theory of Operation ....................................................................................................................................373

16.9 Count Ingress Meters .........................................................................................................................................374

16.9.1 Theory of Operation ....................................................................................................................................374

16.9.2 Driver Reference .........................................................................................................................................374

16.10 Configure Policer Jittering...............................................................................................................................375

16.11 Policer API Structure Initialization ..................................................................................................................375

16.12 Device-Family Differences ...............................................................................................................................375

Chapter 17: Statistics Interface ..................................................................................................... 376

17.1 Theory of Operation............................................................................................................................................376

17.2 Driver Reference .................................................................................................................................................379

17.2.1 APIs ............................................................................................................................................................379

17.2.2 SOC Properties...........................................................................................................................................385

17.2.3 Shell Commands.........................................................................................................................................389

17.2.4 Configuration Flow ......................................................................................................................................389

17.2.5 Application Reference.................................................................................................................................390

17.3 Statistic Interface Compensation......................................................................................................................391

17.3.1 Theory of Operation ....................................................................................................................................391

17.3.2 Driver Reference .........................................................................................................................................392

17.3.2.1 APIs ..................................................................................................................................................392

17.4 Statistics Interface Using the BCM16K (OP2) ..................................................................................................393

17.4.1 Configuration Flow ......................................................................................................................................395

Chapter 18: Interrupts ....................................................................................................................396

18.1 Theory of Operation............................................................................................................................................396

18.1.1 Interrupt Handling Overview .......................................................................................................................396

18.1.2 Interrupt Handler Calling Sequence............................................................................................................396

18.2 Driver Reference .................................................................................................................................................397

18.2.1 APIs ............................................................................................................................................................397

18.2.2 SOC Properties...........................................................................................................................................400

18.2.3 Shell Commands.........................................................................................................................................402

18.2.4 Application Reference.................................................................................................................................402

18.2.4.1 Application Override Interrupt Handler .............................................................................................402

18.2.4.2 Application Initialization Sequence Reference Code ........................................................................403

18.2.4.3 CMIC Interrupts ................................................................................................................................403

18.2.4.4 Forcing Interrupts..............................................................................................................................403

18.2.4.5 Hard Reset Callback Registration.....................................................................................................404

Chapter 19: Shadowing .................................................................................................................. 405

Broadcom 88690-PG207-PUB

16

BCM88690 Programming Guide Traffic Manager

19.1 Theory of Operation............................................................................................................................................405

19.1.1 Caching (Shadowing) Mechanism ..............................................................................................................405

19.1.2 Memory Protection ......................................................................................................................................405

19.2 Driver Reference .................................................................................................................................................406

19.2.1 APIs ............................................................................................................................................................406

19.2.2 SOC Properties...........................................................................................................................................406

19.2.3 Shell Commands.........................................................................................................................................406

19.2.4 Application Reference.................................................................................................................................406

Appendix A: LED Interface ............................................................................................................ 408

A.1 Overview ...............................................................................................................................................................408

A.2 LED Serial Port ....................................................................................................................................................408

A.3 LED Controller .....................................................................................................................................................409

A.3.1 led_accu ......................................................................................................................................................409

A.3.2 led_ram (bank0) ..........................................................................................................................................411

A.3.3 led_send ......................................................................................................................................................411

A.3.3.1 Generation of led_clk ........................................................................................................................412

A.4 Driver Reference ..................................................................................................................................................412

A.4.1 Diagnostic Shell ...........................................................................................................................................412

Appendix B: S-channel Messaging............................................................................................... 414

B.1 Messaging Mechanism .......................................................................................................................................414

B.2 Messaging Registers ...........................................................................................................................................414

B.3 S-channel Messaging Mechanism .....................................................................................................................415

B.3.1 S-channel Message Source and Destination Blocks ...................................................................................416

B.3.2 S-channel Message Format ........................................................................................................................416

B.3.3 Driver Reference .........................................................................................................................................417

Related Documents ........................................................................................................................ 418

Glossary .......................................................................................................................................... 419

Broadcom 88690-PG207-PUB

17

BCM88690 Programming Guide Traffic Manager

Chapter 1: SDK General Concepts

1.1 BCM APIs

A configuration that can be controlled dynamically is exposed to the user application through the BCM API. The BCM

API is

the software library that configures the BCM88XXX. It offers a functional view of the device architecture,

including a use-model

and allocation services.

NOTE: In this document, the term DNX FAP devices refers to the BCM8869X (Jericho2), BCM8880X (J2C), BCM8848X

(Q2A), and BCM8828X (Q2U)devices. Older FAP devices are referred to by name or called legacy FAP devices.

NOTE: In this document and in the SDK, the BCM8828X device (Q2U) is part of the BCM8848X family. All information in

this document that is related to the BCM8848X also applies to BCM8828X unless explicitly stated otherwise.

DNX FAP devices support a subset of APIs in the BCM API library.

Only the APIs mentioned in this document are supported

by DNX FAP devices. If an API is supported by only a subset of DNX FAP devices, or if a description exists per device type,

the relevant device types are explicitly mentioned.

If a single CPU uses multithreading to control multiple devices, create one thread per device. Each thread is responsible for

calling the APIs for the device that is assigned to it. This allows parallel execution of APIs on different devices, while not

attempting to call APIs in parallel on the same device. The driver internally creates threads that perform certain

device

operations in parallel.

The BCM API is independent of the OS and BSP. This is achieved by using software and hardware abstraction

layers,

described in detail in the Core Switch Software Platform Guide Software Development Kit (SDK-PG8xx).

Every BCM API call passes through a dispatch layer that allows controlling local and remote devices, as well as

devices of

different architectures.

Unless specified otherwise, the BCM API prefix is bcm_<module>_xx, which is declared in the SDK_ROOT/include/bcm/

<module>.h header file.

1.1.1 API Return Values

All BCM APIs return success or error values defined by the bcm_error_e enum in include/bcm/error.h. The return

value of

every call to a BCM API must be checked. A return value different from BCM_E_NONE indicates an API failure. Upon

failure, an error message is dispatched according to the logging configuration as defined in the Broadcom System

Log (BSL).

1.1.2 BCM API Objects – GPORTs

The BCM API is a highly polymorphic API offering a sparse set of high-level operations on generic object types.

The most

important object type is gport (also known as generalized port).

A gport is an object that has some of the following attributes:

 It is related to a destination or an intermediate point in the flow of a packet through the system, or its control

path.

 It is related to a hardware object that can be attributed with bandwidth, weight, or priority.

 It is related to a FIFO with configurable admit/discard decision.

 It represents a resource that requires allocation from a resource pool.

Broadcom 88690-PG207-PUB

18

BCM88690 Programming Guide Traffic Manager

Almost any object of interest other than the static translation tables is a gport. A substantial part of a user

application involves

the creation and parameterization of gports, and most of the Driver Reference description

concerns the mapping of user tasks

to gport operations.

The gport object has numerous usages. One subclass of gport objects that is widely used in Traffic

Manager (TM) application

is the family of Class of Service queue (cosq) gports.

Gports are stored in variables of type bcm_gport_t. Values of these variables are gport handles. Gport variables are

usually set in one of two ways:

 By an API that creates the object they refer to or just return a handle to an existing object. For example:

bcm_cosq_gport_add()

 By macros such as:

BCM_GPORT_MODPORT_SET(gport_variable, module_id, port_id)

Broadcom 88690-PG207-PUB

19

BCM88690 Programming Guide Traffic Manager

1.2 SOC Properties

Some device configurations are static, which means they are set only during device initialization. These static configurations,

as well as the

dynamic attributes typically required for device initialization, are provided in a form of configuration properties,

also known as SOC properties. These properties can be provided to the driver in several ways, for example, in

the form of a

configuration file. For details, refer to the Core Switch Software Platform Guide Software Development Kit (SDK-PG8xx).

If several devices use the same configuration file, it is possible to distinguish the configuration by using a special suffix.

 Distinguish by unit ID.

XXX.<unit> is read only by devices with the same unit ID.

For example, fabric_connect_mode.0=FE affects unit ID 0 only.

 Distinguish by chip name.

The chip name represents a tuple of device ID and revision ID.

XXX.<chip name> is read only by devices with the same chip name.

For example, fabric_connect_mode.BCM88690_A0=FE affects Jericho2 revision A0 only.

 Distinguish by device family name.

Each family of devices is distinguished by their family name.

XXX.<family name> is read only by devices with the same family name.

The device family names are as follows:

– The Jericho2 family name is BCM8869X.

– The J2C family name is BCM8880X.

– The Q2A family name is BCM8848X.

For example, fabric_connect_mode.BCM8869X=FE affects the Jericho2 family (the device ID prefix is 8869).

1.2.1 Default Configuration File

All default SOC properties are located at the following path in the SDK, per device family:

 $SDK/rc/config-jr2.bcm

 $SDK/rc/config-q2a.bcm

 $SDK/rc/config-j2c.bcm

Device-specific configuration files are located at the following path at the SDK per device name:

$SDK/rc/dnx_sku/bcm88693.soc

Broadcom 88690-PG207-PUB

20

BCM88690 Programming Guide Traffic Manager

1.3 Shell Commands

The Broadcom shell is an interactive application running on both Broadcom SDK platforms and Broadcom

reference design

platforms. The BCM shell provides access to registers and memories on the switch devices, provides a means of higher-level

configuration, and includes many helpful diagnostic commands. For a detailed description of the BCM shell application, refer

to the

BCM Shell section in the Core Switch Software Platform Guide Software Development Kit (SDK-PG8xx).

Some of the driver reference sections include a list of shell commands relevant to the specific module, along with a short

description.

For a detailed description of a shell command, type the command in the Broadcom shell followed by the usage keyword.

1.4 Application Reference

Driver reference sections across this document might include references to a CINT script, default application, or tuning

configuration.

1.4.1 CINT Reference

For all major features, application-level code is implemented for customer reference. This code demonstrates

a correct API

calling sequence for each feature. The reference code is delivered as part of the SDK release package and verified for each

SDK release. The code is

implemented in the form of C-interpreter (CINT) code. For details on the CINT interpreter, refer to

Network Switching Software Development Kit CINT Interface for Diagnostic Shell (56XX-PG10xx-R).

CINT example code is located at:

 SDK_ROOT/src/examples/sand for CINT files that are supported by both DNX FAP devices and legacy DNX FAP

devices (such as the BCM88670 and BCM88470).

 SDK_ROOT/src/examples/dnx for new CINT files that are supported by the DNX FAP devices.

A README.txt file at each of these locations contains lists

of

available CINT examples per supported feature. A detailed

description of each reference scenario appears in the

header of the relevant CINT file.

1.4.2 Default Application Reference

The DNX FAP device SDK includes a sample application.

This application is a reference targeted at the Broadcom system validation kit (SVK). The SVK is a platform for validating

device capabilities and software maturity. In addition, it may also serve as a reference or a template for customer

applications.

The application reference code is located at SDK_ROOT/src/appl/reference/*.

For a more detailed overview of the default application reference, see Section 2.2.5, Application Reference.

Page is loading ...

Broadcom BCM88690, BCM88800, BCM88480, and BCM88280 Traffic Manager User guide

Broadcom BCM88690, BCM88800, BCM88480, and BCM88280 Traffic Manager User guide

Related papers

Other documents