Cisco IOS XR Software Release 5.2 Configuration Guide

System Monitoring Configuration Guide for Cisco NCS 6000 Series

Routers, Release 5.2.x

First Published: 2014-08-28

Last Modified: 2014-10-06

Americas Headquarters

Cisco Systems, Inc.

170 West Tasman Drive

San Jose, CA 95134-1706

USA

http://www.cisco.com

Tel: 408 526-4000

800 553-NETS (6387)

Fax: 408 527-0883

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,

INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,

EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH

THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,

CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.

CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF

MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT

LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS

HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network

topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional

and coincidental.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://

www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership

relationship between Cisco and any other company. (1110R)

CONTENTS

Preface

Preface xiii

Changes to This Document xiii

Obtaining Documentation and Submitting a Service Request xiii

CHAPTER 1

Implementing and Monitoring Alarms and Alarm Log Correlation 1

Prerequisites for Implementing and Monitoring Alarms and Alarm Log Correlation 2

Information About Implementing Alarms and Alarm Log Correlation 2

Alarm Logging and Debugging Event Management System 2

Correlator 3

System Logging Process 4

Alarm Logger 4

Logging Correlation 4

Correlation Rules 4

Types of Correlation 5

Application of Rules and Rule Sets 5

Root Message and Correlated Messages 5

Alarm Severity Level and Filtering 6

Bistate Alarms 6

Capacity Threshold Setting for Alarms 7

Hierarchical Correlation 7

Context Correlation Flag 8

Duration Timeout Flags 8

Reparent Flag 8

Reissue Nonbistate Flag 9

Internal Rules 9

Alarm Logging Suppression 9

SNMP Alarm Correlation 9

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

iii

How to Implement and Monitor Alarm Management and Logging Correlation 10

Configuring Logging Correlation Rules 10

Configuring Logging Correlation Rule Sets 11

Configuring Root-cause and Non-root-cause Alarms 12

Configuring Hierarchical Correlation Rule Flags 14

Configuring Logging Suppression Rules 15

Applying Logging Correlation Rules 16

Applying Logging Correlation Rule Sets 17

Applying Logging Suppression Rules 19

Modifying Logging Events Buffer Settings 20

Modifying Logging Correlator Buffer Settings 22

Enabling Alarm Source Location Display Field for Bistate Alarms 23

Displaying Alarms by Severity and Severity Range 24

Displaying Alarms According to a Time Stamp Range 25

Displaying Alarms According to a First and Last Range 26

Displaying Alarms by Location 27

Displaying Alarms by Event Record ID 28

Displaying the Logging Correlation Buffer Size, Messages, and Rules 29

Clearing Alarm Event Records and Resetting Bistate Alarms 30

Defining SNMP Correlation Buffer Size 32

Defining SNMP Rulesets 33

Configuring SNMP Correlation Rules 34

Applying SNMP Correlation Rules 34

Applying SNMP Correlation Ruleset 35

Asynchronous Syslog Communication 36

Configuration Examples for Alarm Management and Logging Correlation 36

Increasing the Severity Level for Alarm Filtering to Display Fewer Events and Modifying

the Alarm Buffer Size and Capacity Threshold: Example 36

Configuring a Nonstateful Correlation Rule to Permanently Suppress Node Status Messages:

Example 37

Enabling Alarm Source Location Display Field for Bistate Alarms: Example 38

Additional References 39

CHAPTER 2

Configuring and Managing Embedded Event Manager Policies 41

Prerequisites for Configuring and Managing Embedded Event Manager Policies 42

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

iv

Contents

Information About Configuring and Managing Embedded Event Manager Policies 42

Event Management 42

System Event Detection 42

Policy-Based Event Response 43

Reliability Metrics 43

System Event Processing 43

Embedded Event Manager Management Policies 43

Embedded Event Manager Scripts and the Scripting Interface (Tcl) 44

Script Language 44

Regular Embedded Event Manager Scripts 44

Embedded Event Manager Callback Scripts 45

Embedded Event Manager Policy Tcl Command Extension Categories 45

Cisco File Naming Convention for Embedded Event Manager 46

Embedded Event Manager Built-in Actions 47

Application-specific Embedded Event Management 48

Event Detection and Recovery 48

General Flow of EEM Event Detection and Recovery 49

System Manager Event Detector 49

Timer Services Event Detector 50

Syslog Event Detector 50

None Event Detector 51

Watchdog System Monitor Event Detector 51

Distributed Event Detectors 52

Embedded Event Manager Event Scheduling and Notification 52

Reliability Statistics 53

Hardware Card Reliability Metric Data 53

Process Reliability Metric Data 53

How to Configure and Manage Embedded Event Manager Policies 54

Configuring Environmental Variables 54

Environment Variables 54

Registering Embedded Event Manager Policies 55

Embedded Event Manager Policies 56

How to Write Embedded Event Manager Policies Using Tcl 58

Registering and Defining an EEM Tcl Script 58

Displaying EEM Registered Policies 60

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

v

Contents

Unregistering EEM Policies 60

Suspending EEM Policy Execution 61

Managing EEM Policies 62

Displaying Software Modularity Process Reliability Metrics Using EEM 63

Sample EEM Policies 64

Programming EEM Policies with Tcl 65

Tcl Policy Structure and Requirements 66

EEM Entry Status 67

EEM Exit Status 67

EEM Policies and Cisco Error Number 67

_cerrno: 32-Bit Error Return Values 68

Error Class Encodings for XY 68

Creating an EEM User Tcl Library Index 73

Creating an EEM User Tcl Package Index 76

Configuration Examples for Event Management Policies 79

Environmental Variables Configuration: Example 79

User-Defined Embedded Event Manager Policy Registration: Example 79

Display Available Policies: Example 79

Display Embedded Event Manager Process: Example 79

Configuration Examples for Writing Embedded Event Manager Policies Using Tcl 80

EEM Event Detector Demo: Example 80

EEM Sample Policy Descriptions 80

Event Manager Environment Variables for the Sample Policies 80

Registration of Some EEM Policies 83

Basic Configuration Details for All Sample Policies 83

Using the Sample Policies 83

Running the sl_intf_down.tcl Sample Policy 84

Running the tm_cli_cmd.tcl Sample Policy 84

Running the tm_crash_reporter.tcl Sample Policy 85

Running the tm_fsys_usage.tcl Sample Policy 85

Programming Policies with Tcl: Sample Scripts Example 86

Tracing Tcl set Command Operations: Example 86

Additional References 86

Embedded Event Manager Policy Tcl Command Extension Reference 87

Embedded Event Manager Event Registration Tcl Command Extensions 88

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

vi

Contents

event_register_appl 88

event_register_cli 89

event_register_config 91

event_register_counter 92

event_register_hardware 93

event_register_none 95

event_register_oir 96

event_register_process 97

event_register_snmp 99

event_register_snmp_notification 102

event_register_stat 103

event_register_syslog 106

event_register_timer 108

event_register_timer_subscriber 113

event_register_track 115

event_register_wdsysmon 117

Embedded Event Manager Event Information Tcl Command Extension 123

event_reqinfo 123

event_reqinfo_multi 137

Embedded Event Manager Event Publish Tcl Command Extension 138

event_publish appl 138

Embedded Event Manager Multiple Event Support Tcl Command Extensions 139

Attribute 139

Correlate 139

Trigger 140

Embedded Event Manager Action Tcl Command Extensions 141

action_process 141

action_setnode 143

action_syslog 143

action_track_read 144

Embedded Event Manager Utility Tcl Command Extensions 144

appl_read 144

appl_reqinfo 145

appl_setinfo 146

counter_modify 147

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

vii

Contents

fts_get_stamp 148

register_counter 149

register_timer 150

timer_arm 152

timer_cancel 153

unregister_counter 155

Embedded Event Manager System Information Tcl Command Extensions 156

sys_reqinfo_cpu_all 156

sys_reqinfo_crash_history 157

sys_reqinfo_mem_all 158

sys_reqinfo_proc_version 160

sys_reqinfo_routername 160

sys_reqinfo_syslog_freq 161

sys_reqinfo_syslog_history 162

sys_reqinfo_stat 163

sys_reqinfo_snmp 164

sys_reqinfo_snmp_trap 164

sys_reqinfo_snmp_trapvar 165

SMTP Library Command Extensions 165

smtp_send_email 166

smtp_subst 167

CLI Library Command Extensions 167

cli_close 168

cli_exec 168

cli_get_ttyname 169

cli_open 169

cli_read 170

cli_read_drain 171

cli_read_line 171

cli_read_pattern 172

cli_write 172

Tcl Context Library Command Extensions 175

context_retrieve 175

context_save 179

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

viii

Contents

CHAPTER 3

Implementing IP Service Level Agreements 181

Prerequisites for Implementing IP Service Level Agreements 181

Restrictions for Implementing IP Service Level Agreements 182

Information About Implementing IP Service Level Agreements 182

About IP Service Level Agreements Technology 182

Service Level Agreements 183

Benefits of IP Service Level Agreements 184

Measuring Network Performance with IP Service Level Agreements 185

Operation Types for IP Service Level Agreements 186

IP SLA Responder and IP SLA Control Protocol 186

Response Time Computation for IP SLA 186

IP SLA Operation Scheduling 187

IP SLA—Proactive Threshold Monitoring 187

IP SLA Reaction Configuration 187

IP SLA Threshold Monitoring and Notifications 187

How to Implement IP Service Level Agreements 188

Configuring IP Service Levels Using the UDP Jitter Operation 188

Enabling the IP SLA Responder on the Destination Device 188

Configuring and Scheduling a UDP Jitter Operation on the Source Device 189

Prerequisites for Configuring a UDP Jitter Operation on the Source Device 191

Configuring and Scheduling a Basic UDP Jitter Operation on the Source Device 191

Configuring and Scheduling a UDP Jitter Operation with Additional Characteristics 193

Configuring IP SLA Reactions and Threshold Monitoring 198

Configuring Monitored Elements for IP SLA Reactions 198

Configuring Triggers for Connection-Loss Violations 198

Configuring Triggers for Jitter Violations 199

Configuring Triggers for Packet Loss Violations 200

Configuring Triggers for Round-Trip Violations 201

Configuring Triggers for Timeout Violations 201

Configuring Triggers for Verify Error Violations 202

Configuring Threshold Violation Types for IP SLA Reactions 203

Generating Events for Each Violation 204

Generating Events for Consecutive Violations 205

Generating Events for X of Y Violations 206

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

ix

Contents

Generating Events for Averaged Violations 207

Specifying Reaction Events 208

Configuration Examples for Implementing IP Service Level Agreements 209

Configuring IP Service Level Agreements: Example 209

Configuring IP SLA Reactions and Threshold Monitoring: Example 210

CHAPTER 4

Implementing Logging Services 213

Prerequisites for Implementing Logging Services 213

Information About Implementing Logging Services 214

System Logging Process 214

Format of System Logging Messages 214

Duplicate Message Suppression 215

Message Suppression 215

Logging History and Syslog Comparison 216

Syslog Message Destinations 216

Guidelines for Sending Syslog Messages to Destinations Other Than the Console 217

Logging for the Current Terminal Session 217

Syslog Messages Sent to Syslog Servers 217

UNIX System Logging Facilities 217

Hostname Prefix Logging 218

Syslog Source Address Logging 219

UNIX Syslog Daemon Configuration 219

Archiving Logging Messages on a Local Storage Device 219

Setting Archive Attributes 219

Archive Storage Directories 220

Severity Levels 220

Logging History Table 221

Syslog Message Severity Level Definitions 222

Syslog Severity Level Command Defaults 222

How to Implement Logging Services 223

Setting Up Destinations for System Logging Messages 223

Configuring Logging to a Remote Server 224

Configuring the Settings for the Logging History Table 226

Modifying Logging to the Console Terminal and the Logging Buffer 227

Modifying the Format of Time Stamps 228

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

x

Contents

Disabling Time Stamps 229

Suppressing Duplicate Syslog Messages 230

Disabling the Logging of Link-Status Syslog Messages 230

Displaying System Logging Messages 231

Archiving System Logging Messages to a Local Storage Device 232

Configuration Examples for Implementing Logging Services 234

Configuring Logging to the Console Terminal and the Logging Buffer: Example 234

Setting Up Destinations for Syslog Messages: Example 235

Configuring the Settings for the Logging History Table: Example 235

Modifying Time Stamps: Example 235

Configuring a Logging Archive: Example 235

Where to Go Next 236

Additional References 236

CHAPTER 5

Onboard Failure Logging 239

Prerequisites 240

Information About Implementing OBFL 240

Data Collection Types 240

Baseline Data Collection 240

Supported Cards and Platforms 240

Where to Go Next 241

Additional References 241

CHAPTER 6

Implementing Performance Management 245

Prerequisites for Implementing Performance Management 246

Information About Implementing Performance Management 246

PM Functional Overview 246

PM Statistics Server 246

PM Statistics Collector 246

PM Benefits 247

PM Statistics Collection Overview 248

PM Statistics Collection Templates 248

Guidelines for Creating PM Statistics Collection Templates 249

Guidelines for Enabling and Disabling PM Statistics Collection Templates 249

Exporting Statistics Data 250

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

xi

Contents

Binary File Format 250

Binary File ID Assignments for Entity, Subentity, and StatsCounter Names 251

Filenaming Convention Applied to Binary Files 255

PM Entity Instance Monitoring Overview 255

PM Threshold Monitoring Overview 259

Guidelines for Creating PM Threshold Monitoring Templates 259

Guidelines for Enabling and Disabling PM Threshold Monitoring Templates 272

How to Implement Performance Management 273

Configuring an External TFTP Server for PM Statistic Collections 273

Configuring Local Disk Dump for PM Statistics Collections 273

Configuring Instance Filtering by Regular-expression 274

Creating PM Statistics Collection Templates 275

Enabling and Disabling PM Statistics Collection Templates 276

Enabling PM Entity Instance Monitoring 278

Creating PM Threshold Monitoring Templates 278

Enabling and Disabling PM Threshold Monitoring Templates 279

Configuration Examples for Implementing Performance Management 281

Creating and Enabling PM Statistics Collection Templates: Example 281

Creating and Enabling PM Threshold Monitoring Templates: Example 282

Additional References 282

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

xii

Contents

Preface

The System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers preface contains these

sections:

•Changes to This Document, page xiii

•Obtaining Documentation and Submitting a Service Request, page xiii

Changes to This Document

This table lists the technical changes made to this document since it was first printed.

Table 1: Changes to this Document

Change SummaryDate

Initial release of this documentAugust 2014

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service

request, and gathering additional information, see What's New in Cisco Product Documentation.

To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the What's

New in Cisco Product Documentation RSS feed. RSS feeds are a free service.

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

xiii

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

xiv

Preface

Obtaining Documentation and Submitting a Service Request

CHAPTER 1

Implementing and Monitoring Alarms and Alarm

Log Correlation

This module describes the concepts and tasks related to configuring alarm log correlation and monitoring

alarm logs and correlated event records. Alarm log correlation extends system logging to include the ability

to group and filter messages generated by various applications and system servers and to isolate root messages

on the router.

This module describes the new and revised tasks you need to perform to implement logging correlation and

monitor alarms on your network.

For more information about system logging on Cisco IOS XR Software and complete descriptions of the

alarm management and logging correlation commands listed in this module, see the Related Documents,

on page 39 section of this module.

Note

Feature History for Implementing and Monitoring Alarms and Alarm Log Correlation

ModificationRelease

The feature was introduced.Release 5.0.0

•Prerequisites for Implementing and Monitoring Alarms and Alarm Log Correlation, page 2

•Information About Implementing Alarms and Alarm Log Correlation, page 2

•How to Implement and Monitor Alarm Management and Logging Correlation, page 10

•Configuration Examples for Alarm Management and Logging Correlation, page 36

•Additional References, page 39

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

1

Prerequisites for Implementing and Monitoring Alarms and

Alarm Log Correlation

You must be in a user group associated with a task group that includes the proper task IDs. The command

reference guides include the task IDs required for each command. If you suspect user group assignment is

preventing you from using a command, contact your AAA administrator for assistance.

Information About Implementing Alarms and Alarm Log

Correlation

Alarm Logging and Debugging Event Management System

Cisco IOS XR Software Alarm Logging and Debugging Event Management System (ALDEMS) is used to

monitor and store alarm messages that are forwarded by system servers and applications. In addition, ALDEMS

correlates alarm messages forwarded due to a single root cause.

ALDEMS enlarges on the basic logging and monitoring functionality of Cisco IOS XR Software, providing

the level of alarm and event management necessary for a highly distributed system with potentially hundreds

of modular service cards (MSCs) and thousands of interfaces.

Cisco IOS XR Software achieves this necessary level of alarm and event management by distributing logging

applications across the nodes on the system.

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

2

Implementing and Monitoring Alarms and Alarm Log Correlation

Prerequisites for Implementing and Monitoring Alarms and Alarm Log Correlation

Figure 1: ALDEMS Component Communications, on page 3 illustrates the relationship between the

components that constitute ALDEMS.

Figure 1: ALDEMS Component Communications

Correlator

The correlator receives messages from system logging (syslog) helper processes that are distributed across

the nodes on the router and forwards syslog messages to the syslog process. If a logging correlation rule is

configured, the correlator captures messages searching for a match with any message specified in the rule. If

the correlator finds a match, it starts a timer that corresponds to the timeout interval specified in the rule. The

correlator continues searching for a match to messages in the rule until the timer expires. If the root case

message was received, then a correlation occurs; otherwise, all captured messages are forwarded to the syslog.

When a correlation occurs, the correlated messages are stored in the logging correlation buffer. The correlator

tags each set of correlated messages with a correlation ID.

For more information about logging correlation, see the Logging Correlation, on page 4 section.Note

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

3

Implementing and Monitoring Alarms and Alarm Log Correlation

Alarm Logging and Debugging Event Management System

System Logging Process

By default, routers are configured to send system logging messages to a system logging (syslog) process.

Syslog messages are gathered by syslog helper processes that are distributed across the nodes on the system.

The system logging process controls the distribution of logging messages to the various destinations, such as

the system logging buffer, the console, terminal lines, or a syslog server, depending on the network device

configuration.

Alarm Logger

The alarm logger is the final destination for system logging messages forwarded on the router. The alarm

logger stores alarm messages in the logging events buffer. The logging events buffer is circular; that is, when

full, it overwrites the oldest messages in the buffer.

Alarms are prioritized in the logging events buffer. When it is necessary to overwrite an alarm record, the

logging events buffer overwrites messages in the following order: nonbistate alarms first, then bistate

alarms in the CLEAR state, and, finally, bistate alarms in the SET state. For more information about bistate

alarms, see the Bistate Alarms, on page 6 section.

Note

When the table becomes full of messages caused by bistate alarms in the SET state, the earliest bistate message

(based on the message time stamp, not arrival time) is reclaimed before others. The buffer size for the logging

events buffer and the logging correlation buffer, thus, should be adjusted so that memory consumption is

within your requirements.

A table-full alarm is generated each time the logging events buffer wraps around. A threshold crossing

notification is generated each time the logging events buffer reaches the capacity threshold.

Messages stored in the logging events buffer can be queried by clients to locate records matching specific

criteria. The alarm logging mechanism assigns a sequential, unique ID to each alarm message.

Logging Correlation

Logging correlation can be used to isolate the most significant root messages for events affecting system

performance. For example, the original message describing a card online insertion and removal (OIR) of a

modular services card (MSC) can be isolated so that only the root-cause message is displayed and all subsequent

messages related to the same event are correlated. When correlation rules are configured, a common root

event that is generating secondary (non-root-cause) messages can be isolated and sent to the syslog, while

secondary messages are suppressed. An operator can retrieve all correlated messages from the logging correlator

buffer to view correlation events that have occurred.

Correlation Rules

Correlation rules can be configured to isolate root messages that may generate system alarms. Correlation

rules prevent unnecessary stress on ALDEMS caused by the accumulation of unnecessary messages. Each

correlation rule hinges on a message identification, consisting of a message category, message group name,

and message code. The correlator process scans messages for occurrences of the message.

If the correlator receives a root message, the correlator stores it in the logging correlator buffer and forwards

it to the syslog process on the RP. From there, the syslog process forwards the root message to the alarm

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

4

Implementing and Monitoring Alarms and Alarm Log Correlation

Logging Correlation

logger in which it is stored in the logging events buffer. From the syslog process, the root message may also

be forwarded to destinations such as the console, remote terminals, remote servers, the fault management

system, and the Simple Network Management Protocol (SNMP) agent, depending on the network device

configuration. Subsequent messages meeting the same criteria (including another occurrence of the root

message) are stored in the logging correlation buffer and are forwarded to the syslog process on the router.

If a message matches multiple correlation rules, all matching rules apply and the message becomes a part of

all matching correlation queues in the logging correlator buffer.

The following message fields are used to define a message in a logging correlation rule:

•Message category

•Message group

•Message code

Wildcards can be used for any of the message fields to cover wider set of messages. Configure the appropriate

set of messages in a logging correlation rule configuration to achieve correlation with a narrow or wide scope

(depending on your objective).

Types of Correlation

There are two types of correlation that are configured in rules to isolate root-cause messages:

Nonstateful Correlation—This correlation is fixed after it has occurred, and non-root-cause alarms that are

suppressed are never forwarded to the syslog process. All non-root-cause alarms remain buffered in correlation

buffers.

Stateful Correlation—This correlation can change after it has occurred, if the bistate root-cause alarm clears.

When the alarm clears, all the correlated non-root-cause alarms are sent to syslog and are removed from the

correlation buffer. Stateful correlations are useful to detect non-root-cause conditions that continue to exist

even if the suspected root cause no longer exists.

Application of Rules and Rule Sets

If a correlation rule is applied to the entire router, then correlation takes place only for those messages that

match the configured cause values for the rule, regardless of the context or location setting of that message.

If a correlation rule is applied to a specific set of contexts or locations, then correlation takes place only for

those messages that match the configured cause values for the rule and that match at least one of those contexts

or locations.

In the case of a rule-set application, the behavior is the same; however, the apply configuration takes place

for all rules that are part of the given rule set.

The show logging correlator rule command is used to display apply settings for a given rule, including

those settings that have been configured with the logging correlator apply ruleset command.

Root Message and Correlated Messages

When a correlation rule is configured and applied, the correlator starts searching for a message match as

specified in the rule. After a match is found, the correlator starts a timer corresponding to the timeout interval

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

5

Implementing and Monitoring Alarms and Alarm Log Correlation

Application of Rules and Rule Sets

that is also specified in the rule. A message search for a match continues until the timer expires. Correlation

occurs after the root-cause message is received.

The first message (with category, group, and code triplet) configured in a correlation rule defines the root-cause

message. A root-cause message is always forwarded to the syslog process. See the Correlation Rules, on page

4section to learn how the root-cause message is forwarded and stored.

Alarm Severity Level and Filtering

Filter settings can be used to display information based on severity level. The alarm filter display indicates

the severity level settings used to report alarms, the number of records, and the current and maximum log

size.

Alarms can be filtered according to the severity level shown in this table.

Table 2: Alarm Severity Levels for Event Logging

System ConditionSeverity Level

Emergencies0

Alerts1

Critical2

Errors3

Warnings4

Notifications5

Informational6

Bistate Alarms

Bistate alarms are generated by state changes associated with system hardware, such as a change of interface

state from active to inactive, the online insertion and removal (OIR) of a modular service card (MSC), or a

change in component temperature. Bistate alarm events are reported to the logging events buffer by default;

informational and debug messages are not.

Cisco IOS XR Software software provides the ability to reset and clear alarms. Clients interested in monitoring

alarms in the system can register with the alarm logging mechanism to receive asynchronous notifications

when a monitored alarm changes state.

Bistate alarm notifications provide the following information:

•The origination ID, which uniquely identifies the resource that causes an alarm to be raised or cleared.

This resource may be an interface, a line card, or an application-specific integrated circuit (ASIC). The

origination ID is a unique combination of the location, job ID, message group, and message context.

System Monitoring Configuration Guide for Cisco NCS 6000 Series Routers, Release 5.2.x

6

Implementing and Monitoring Alarms and Alarm Log Correlation

Alarm Severity Level and Filtering

Page is loading ...

Cisco IOS XR Software Release 5.2 Configuration Guide

Cisco IOS XR Software Release 5.2 Configuration Guide

Related papers

Other documents