McAfee M4050 - Network Security Platform Troubleshooting Manual

McAfee®

Network Protection

Industry-leading network security solutions

Troubleshooting Guide

McAfee® Network Security Platform

version 6.0

Revision 6.0

COPYRIGHT

any language in any form or by any means without the written permission of McAfee, Inc., or its suppliers or affiliate companies.

TRADEMARKS

ACTIVE FIREWALL, ACTIVE SECURITY, ACTIVESECURITY (AND IN KATAKANA), ACTIVESHIELD, CLEAN-UP, DESIGN (STYLIZED E), DESIGN (STYLIZED N),

ENTERCEPT, EPOLICY ORCHESTRATOR, FIRST AID, FOUNDSTONE, GROUPSHIELD, GROUPSHIELD (AND IN KATAKANA), INTRUSHIELD, INTRUSION PREVENTION

THROUGH INNOVATION, McAfee, McAfee (AND IN KATAKANA), McAfee AND DESIGN, McAfee.COM, McAfee VIRUSSCAN, NET TOOLS, NET TOOLS (AND IN KATAKANA),

NETSCAN, NETSHIELD, NUTS & BOLTS, OIL CHANGE, PRIMESUPPORT, SPAMKILLER, THREATSCAN, TOTAL VIRUS DEFENSE, VIREX, VIRUS FORUM, VIRUSCAN,

VIRUSSCAN, VIRUSSCAN (AND IN KATAKANA), WEBSCAN, WEBSHIELD, WEBSHIELD (AND IN KATAKANA) are registered trademarks or trademarks of McAfee, Inc. and/or

its affiliates in the US and/or other countries. The color red in connection with security is distinctive of McAfee brand products. All other registered and unregistered trademarks

herein are the sole property of their respective owners.

LICENSE AND PATENT INFORMATION

License Agreement

NOTICE TO ALL USERS: CAREFULLY READ THE APPROPRIATE LEGAL AGREEMENT CORRESPONDING TO THE LICENSE YOU PURCHASED, WHICH SETS FORTH

THE GENERAL TERMS AND CONDITIONS FOR THE USE OF THE LICENSED SOFTWARE. IF YOU DO NOT KNOW WHICH TYPE OF LICENSE YOU HAVE ACQUIRED,

PLEASE CONSULT THE SALES AND OTHER RELATED LICENSE GRANT OR PURCHASE ORDER DOCUMENTS THAT ACCOMPANIES YOUR SOFTWARE PACKAGING

OR THAT YOU HAVE RECEIVED SEPARATELY AS PART OF THE PURCHASE (AS A BOOKLET, A FILE ON THE PRODUCT CD, OR A FILE AVAILABLE ON THE WEB SITE

FROM WHICH YOU DOWNLOADED THE SOFTWARE PACKAGE). IF YOU DO NOT AGREE TO ALL OF THE TERMS SET FORTH IN THE AGREEMENT, DO NOT INSTALL

THE SOFTWARE. IF APPLICABLE, YOU MAY RETURN THE PRODUCT TO McAfee OR THE PLACE OF PURCHASE FOR A FULL REFUND.

License Attributions

This product includes or may include:

* Software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/). * Cryptographic software written by Eric A. Young and software written by

Tim J. Hudson. * Some software programs that are licensed (or sublicensed) to the user under the GNU General Public License (GPL) or other similar Free Software licenses

which, among other rights, permit the user to copy, modify and redistribute certain programs, or portions thereof, and have access to the source code. The GPL requires that for

any software covered under the GPL, which is distributed to someone in an executable binary format, that the source code also be made available to those users. For any such

software covered under the GPL, the source code is made available on this CD. If any Free Software licenses require that McAfee provide rights to use, copy or modify a software

program that are broader than the rights granted in this agreement, then such rights shall take precedence over the rights and restrictions herein. * Software originally written by

Douglas W. Sauder. * Software developed by the Apache Software Foundation (http://www.apache.org/). A copy of the license agreement for this software can be found at

California, Berkeley and its contributors. * Software developed by Ralf S. Engelschall <rse@engelschall.com> for use in the mod_ssl project (http:// www.modssl.org/). * Software

2002. See http://www.boost.org/libs/bind/bind.html

Cleary (shammah@voyager.net

contributed to Berkeley by Chris Torek.

Issued APRIL 2011 / Troubleshooting Guide

700-2380-00/ 6.0 - English

iii

Contents

Preface ........................................................................................................... v

Introducing McAfee Network Security Platform............................................................................. v

About this Guide............................................................................................................................ v

Audience ....................................................................................................................................... v

Conventions used in this book ......................................................................................................vi

Related Documentation................................................................................................................vii

Contacting Technical Support.....................................................................................................viii

Information requested for Troubleshooting ......................................................................... viii

Chapter 1 Before You Install........................................................................ 1

Pre-installation recommendations.................................................................................................1

Planning for installation..........................................................................................................1

Functional requirements.........................................................................................................2

Using anti-virus software with the Manager ...........................................................................4

User interface responsiveness...............................................................................................5

Chapter 2 Hardening the Manager Server for Windows 2003 .................. 6

Introduction....................................................................................................................................6

Install a desktop firewall................................................................................................................6

Harden the MySQL installation......................................................................................................6

Remove test database...........................................................................................................7

Remove local anonymous users............................................................................................7

Remove remote anonymous users........................................................................................7

Secure MySQL remote access ..............................................................................................8

Rolling back your changes.....................................................................................................9

Remove debug shell at port 9001 ..........................................................................................9

Other best practices for securing Manager...................................................................................9

Chapter 3 Hardening the Manager Server for Windows 2008 ................ 10

Pre-installation.............................................................................................................................10

Installation...................................................................................................................................10

Post Installation...........................................................................................................................10

Disabling non-required Services..........................................................................................11

Setting System Policies........................................................................................................11

Setting User Policies............................................................................................................11

Setting a Desktop Firewall ...................................................................................................11

Configuring Audit Events......................................................................................................12

Chapter 4 Troubleshooting Network Security Platform.......................... 14

Facilitating troubleshooting..........................................................................................................14

Starting your troubleshooting ......................................................................................................15

Difficulties connecting Sensor and Manager...............................................................................15

Network connectivity............................................................................................................15

Inconsistency in Sensor and Manager configuration ...........................................................15

Software or signature set incompatibility..............................................................................15

Firewall between the devices...............................................................................................16

Management port configuration ...........................................................................................16

Connectivity issues between the Sensor and other network devices .........................................17

Duplex mismatches..............................................................................................................17

Valid auto-negotiation and speed configurations.................................................................17

Explanation of CatOS show port Command Counters.........................................................20

Auto-negotiation...................................................................................................................21

iv

Checking Sensor health..............................................................................................................22

Pinging a Sensor..................................................................................................................22

Ensuring that the Sensor is receiving traffic................................................................................22

Checking Sensor failover status..................................................................................................23

Cabling failover through a network device...........................................................................23

Checking whether a signature or software update was successful............................................. 24

Checking status of a download or upload ...................................................................................24

Conditions requiring a Sensor reboot..........................................................................................24

Rebooting a Sensor via the Manager...................................................................................25

Rebooting a Sensor using the reboot command..................................................................25

Sensor doesn’t boot ....................................................................................................................25

Debugging critical Sensor issues................................................................................................ 25

Loss of connectivity between the Sensor and Manager.............................................................. 29

How Sensor handles new alerts during connectivity loss ....................................................30

Manager connectivity to the database.........................................................................................30

Manager database is full......................................................................................................31

Error on accessing the Configuration page................................................................................. 31

Sensor response if its throughput is exceeded ...........................................................................31

MySQL issues.............................................................................................................................32

How Sensors handle various types of traffic............................................................................... 32

Jumbo Ethernet frames........................................................................................................32

ISL frames............................................................................................................................32

Sensor failover issues................................................................................................................. 33

External fail-open kit issues in connecting to the monitoring port ...............................................33

XC cable connection issues for M8000 Sensors......................................................................... 33

Chapter 5 Determining False Positives .................................................... 34

Reducing false positives..............................................................................................................34

Tune your policies....................................................................................................................... 34

About false positives and “noise”.........................................................................................35

Determining a false positive versus noise............................................................................36

Chapter 6 System Fault Messages............................................................ 38

Critical faults................................................................................................................................38

Error faults...................................................................................................................................55

Warning faults ............................................................................................................................. 61

Informational faults...................................................................................................................... 65

Other faults..................................................................................................................................76

Chapter 7 Error Messages.......................................................................... 77

Error messages for RADIUS servers ..........................................................................................77

Error messages for LDAP server ................................................................................................78

Chapter 8 Using the InfoCollector tool..................................................... 79

Introduction..................................................................................................................................79

Running the InfoCollector............................................................................................................80

Using InfoCollector......................................................................................................................80

Chapter 9 Automatically restarting a failed Manager with Manager

Watchdog..................................................................................................... 81

Introduction..................................................................................................................................81

How the Manager Watchdog Works............................................................................................ 81

Installing Manager Watchdog......................................................................................................82

Starting Manager Watchdog........................................................................................................82

Using Manager Watchdog with Manager in an MDR configuration ............................................82

Tracking Manager Watchdog activities .......................................................................................82

Chapter 10 Utilizing the McAfee Knowledge Base .................................. 84

Index............................................................................................................. 86

v

Preface

This preface provides a brief introduction to the product, discusses the information in this

document, and explains how this document is organized. It also provides information such

as, the supporting documents for this guide and how to contact McAfee Technical Support.

Introducing McAfee Network Security Platform

McAfee

®

Network Security Platform [formerly McAfee

®

IntruShield

®

] delivers the most

comprehensive, accurate, and scalable Network Access Control (NAC), network Intrusion

Prevention System (IPS) and Network Threat Behavior Analysis (NTBA) for mission-critical

enterprise, carrier and service provider networks, while providing unmatched protection

against spyware; known, zero-day, and encrypted attacks.

McAfee

®

Network Threat Behavior Analysis Appliance provides the capability of monitoring

network traffic by analyzing NetFlow information flowing through the network in real time,

thus complementing the NAC and IPS capabilities in a scenario in which McAfee Network

Security Sensor, NAC Sensor, and NTBA Appliance are installed and managed through a

single Manager.

About this Guide

This guide provides the basic troubleshooting techniques for Network Security Platform.

You get information on the key issues to be taken care of in the McAfee

®

Network Security

Manager [formerly McAfee

®

IntruShield

®

Security Manager] and McAfee

®

Network Security

Sensor [formerly McAfee

®

IntruShield

®

Sensor] software in a step-by- step manner; right

from installing Network Security Platform to troubleshooting the system.

This guide provides detailed sections on the following topics:

 Pre-installation recommendations

 Hardening McAfee Network Security Manager (Manager) Server

 Troubleshooting techniques

 How to use the InfoCollector tool and Manager Watchdog

Audience

This guide is intended for use by network technicians responsible for maintaining the

Network Security Platform and analyzing and disseminating the resulting data. It is

assumed that you are familiar with IPS-related tasks, the relationship between tasks, and

the commands necessary to perform particular tasks.

McAfee® Network Security Platform 6.0

Preface

vi

Conventions used in this book

This document uses the following typographical conventions:

Convention Example

Terms that identify fields, buttons,

tabs, options, selections, and

commands on the User Interface

(UI) are shown in

Arial Narrow bold

font.

The

Service field on the Properties tab specifies the

name of the requested service.

Menu or action group selections

are indicated using a right angle

bracket.

Select My Company > Admin Domain > Summary.

Procedures are presented as a

series of numbered steps.

1. On the Configuration tab, click Backup.

Names of keys on the keyboard

are denoted using UPPER CASE.

Press ENTER.

Text such as syntax, key words,

and values that you must type

exactly are denoted using

Courier New font.

Type: setup and then press ENTER.

Variable information that you must

type based on your specific

situation or environment is shown

in italics.

Type: Sensor-IP-address and then press

ENTER.

Parameters that you must supply

are shown enclosed in angle

brackets.

set Sensor ip <A.B.C.D>

Information that you must read

before beginning a procedure or

that alerts you to negative

consequences of certain actions,

such as loss of data is denoted

using this notation.

Caution:

Information that you must read to

prevent injury, accidents from

contact with electricity, or other

serious consequences is denoted

using this notation.

Warning:

Notes that provide related, but

non-critical, information are

denoted using this notation.

Note:

McAfee® Network Security Platform 6.0

Preface

vii

Related Documentation

The following documents and on-line help are companions to this guide. Refer to Quick Tour

for more information on these guides.

 Quick Tour

 Installation Guide

 Upgrade Guide

 Getting Started Guide

 IPS Deployment Guide

 Manager Configuration Basics Guide

 I-1200 Sensor Product Guide

 I-1400 Sensor Product Guide

 I-2700 Sensor Product Guide

 I-3000 Sensor Product Guide

 I-4000 Sensor Product Guide

 I-4010 Sensor Product Guide

 M-1250/M-1450 Sensor Product Guide

 M-1250/M-1450 Quick Start Guide

 M-2750 Sensor Product Guide

 M-2750 Quick Start Guide

 M-3050/M-4050 Sensor Product Guide

 M-3050/M-4050 Quick Start Guide

 M-6050 Sensor Product Guide

 M-6050 Quick Start Guide

 M-8000 Sensor Product Guide

 M-8000 Quick Start Guide

 Gigabit Optical Fail-Open Bypass Kit Guide

 Gigabit Copper Fail-Open Bypass Kit Guide

 10 Gigabit Fail-Open Bypass Kit Guide

 M-8000/M-6050/M-4050/M-3050 Slide Rail Assembly Procedure

 M-2750 Slide Rail Assembly Procedure

 M-series DC Power Supply Installation Procedure

 Administrative Domain Configuration Guide

 Manager Server Configuration Guide

 CLI Guide

 Device Configuration Guide

 IPS Configuration Guide

 NAC Configuration Guide

 Integration Guide

 System Status Monitoring Guide

 Reports Guide

 Custom Attack Definitions Guide

 Central Manager Administrator's Guide

 Best Practices Guide

 Special Topics Guide—In-line Sensor Deployment

McAfee® Network Security Platform 6.0

Preface

viii

 Special Topics Guide—Sensor High Availability

 Special Topics Guide—Virtualization

 Special Topics Guide—Denial-of-Service

 NTBA Appliance Administrator's Guide

 NTBA Monitoring Guide

 NTBA Appliance T-200 Quick Start Guide

 NTBA Appliance T-500 Quick Start Guide

Contacting Technical Support

If you have any questions, contact McAfee for assistance:

Online

Contact McAfee Technical Support http://mysupport.mcafee.com.

Registered customers can obtain up-to-date documentation, technical bulletins, and quick

tips on McAfee's 24x7 comprehensive KnowledgeBase. In addition, customers can also

resolve technical issues with the online case submit, software downloads, and signature

updates.

Phone

Technical Support is available 7:00 A.M. to 5:00 P.M. PST Monday-Friday. Extended 24x7

Technical Support is available for customers with Gold or Platinum service contracts.

Global phone contact numbers can be found at McAfee Contact Information

http://www.mcafee.com/us/about/contact/index.html page.

Note: McAfee requir

es that you provide your GRANT ID and the serial number of

your system when opening a ticket with Technical Support. You will be provided with

a user name and password for the online case submission.

Information requested for Troubleshooting

McAfee wants to provide you with the best possible support. When you contact Technical

Support, we will request a variety of information to use to troubleshoot your deployment.

This section describes the information we ask that you have available for troubleshooting.

General information

 your GRANT ID. This was provided to you when you purchased the product.

 the version number of the Manager software you are using

 the version number of the McAfee Network Security Sensor (Sensor) software you are

using

 Is this a new or existing issue?

 any physical changes made to the environment recently

McAfee® Network Security Platform 6.0

Preface

ix

 Did you make any changes in your environment/setup/configuration that may have

introduced the issue?

Manager-specific information

We may ask you to use our troubleshooting tool, which is called InfoCollector. This tool will

collect all Manager-related log files (For example, ems.log, emsout, output.bin, config

back, and the Sensor trace file, if you have uploaded it to the Manager) and return them to

us for analysis

As of this writing, the tool is available at the following link:

http://serviceweb/McAfee/backline/escalations/MER_TOOL/IPSInfoCollector.zip

Sensor issues

 the Sensor deployment configuration

 information on the GBICs you are using with Sensor GE ports; this information is

extremely helpful for troubleshooting link issues

 the volume of traffic through the Sensor

 in some cases, a network diagram (particularly for troubleshooting asymmetric traffic

issues)

 a Sensor trace file, which you can create using the process described in Providing a

Sensor diagnostics trace.

 Sensor operating mode (i.e., In-line, SPAN or TAP). This information can be obtained

from:

Sensor_Name > Interface > View Details

 peer device port settings (For example, for Cisco switches/routers, you would provide

the output of the show port [mod[/port] command.

 Management port configuration (obtained by issuing a show mgmtport command)

Signature set issues

 the signature set and software versions you are running

 the frequency at which you see the false positive

 whether the alert condition is reproducible

 policy configuration

 alert evidence reports

 traffic volume, if possible

 traffic type

 what software and systems are on the affected systems

 your network topology

1

C HAPTER 1

Before You Install

This chapter lists pre-installation recommendations.

Pre-installation recommendations

These McAfee

®

Network Security Platform [formerly McAfee

®

IntruShield

®

] pre-installation

recommendations are a compilation of the information gathered from individual interviews

with some of the most seasoned McAfee Network Security Platform System Engineers at

McAfee.

Planning for installation

Before installation, ensure that you complete the following tasks:

 The server, on which McAfee

®

Network Security Manager software will be installed,

should be configured and ready to be placed online.

 You must have administrator privileges for McAfee Network Security Manager

(Manager) server.

 This server should be dedicated, hardened for security, and placed on its own subnet.

This server should not be used for programs like instant messaging or other non-

secure Internet functions.

 Make sure your hardware requirements meet the requirements. See Server

requirements.

 Ensure the proper static IP address has been assigned to the Manager server. For the

Manager server, McAfee strongly recommends assigning a static IP against using

DHCP for IP assignment.

 If applicable, configure name resolution for the Manager.

 Ensure that all parties have agreed to the solution design, including the location and

mode of all McAfee

®

Network Security Sensor, the use of sub-interfaces or interface

groups, and if and how the Manager will be connected to the production network.

 Get the required license file and grant number.

 Accumulate the required number of wires and (supported) GBICs, SFPs, or XFPs.

Ensure these are approved hardware from McAfee or a supported vendor. Ensure

that the required number of Network Security Platform dongles, which ship with the

McAfee Network Security Sensors (Sensors), are available.

 Crossover cables will be required for 10/100 or 10/100/1000 monitoring ports if they

are directly connected to a firewall, router, or end node. Otherwise, standard patch

cables are required for the Fast Ethernet ports.

 If applicable, identify the ports to be mirrored, and someone who has the knowledge

and rights to mirror them.

 Allocate the proper static IP addresses for the Sensor. For the Sensors, you cannot

assign IPs using DHCP.

McAfee® Network Security Platform 6.0

Before You Install

2

 Identify hosts that may cause false positives, for example, HTTP cache servers, DNS

servers, mail relays, SNMP managers, and vulnerability scanners.

Functional requirements

Following are the functional requirements to be taken care of:

 Install Wireshark (formerly known as Ethereal http://www.wireshark.com

http://www.wireshark.org) on the client PCs. Etherea

l is a network protocol analyzer

for Unix and Windows servers, used to analyze the packet logs created by Sensors.

 Ensure the correct version of JRE is installed on the client system, as described in the

Release Notes. This can save a lot of time during deployment.

 Determine a way in which the Manager maintains the correct time. To keep time from

drifting, for example, point the Manager server to an NTP timeserver. (If the time is

changed on the Manager server, the Manager will lose connectivity with all Sensors

and the McAfee

®

Network Security Update Server because SSL is time sensitive.)

 If Manager Disaster Recovery (MDR) is configured, ensure that the time difference

between the Primary and Secondary Managers is less than 60 seconds. (If the spread

between the two exceeds more than two minutes, communication with the Sensors

will be lost.)

 If you are upgrading from a previous version, we recommend that you follow the

instructions in the respective version’s release notes or, if applicable, the Upgrade

Guide

.

Install a desktop firewall

McAfee strongly recommends that you configure a packet-filtering firewall to block

connections to ports 8551, 3306, 8007, 8009, and 8552 of your Manager server. The

firewall can either be a host-based or a network-based.

Set your firewall to deny connections to these ports if the connections are not initiated by

the localhost. The only connections that should be allowed are those from the Manager

server itself; that is, the localhost.

For example, if another machine attempts to connect to port 8551, 8552, 3306, 8007 and

8009 the firewall should automatically block any packets sent. If you need assistance in

blocking these, contact Technical Support.

If a firewall will reside between the Sensor, Manager, or administrative client, which

includes a personal firewall on the Manager, the following ports must be opened:

Port # Protocol Description Direction of communication

4167 (high ports)

(source port on the Manager)

and

8500

(destination port on the

Sensor)

UDP

Default SNMPv3

(command

channel)

Manager-->Sensor

McAfee® Network Security Platform 6.0

Before You Install

3

Port # Protocol Description Direction of communication

8501 TCP Proprietary

(install port)

Sensor-->Manager

8502 TCP Proprietary

(alert

channel/control

channel)

Sensor-->Manager

8503 TCP Proprietary

(packet log

channel)

Sensor-->Manager

8504 TCP Proprietary

(file transfer

channel)

Sensor-->Manager

8555 TCP SSL/TCP/IP

(Threat Analyzer)

client-->Manager

443 TCP HTTPS client-->Manager

80 TCP Web-based user

interface

client-->Manager

(Webstart/JNLP, Console

Applets)

22 TCP SSH Remote console access

Note: If you choose to use non-default ports for the Install port, Alert port, and Log

port, ensure that those ports are also open on the firewall.

 Note that 3306/TCP is used internally by the Manager to connect to the MySQL

database.

 If you have Email Notification or SNMP Forwarding configured on the Manager, and

there is firewall residing between the Manager and your SMTP or SNMP server,

ensure the following ports are available as well.

Additional communication ports

Port # Protocol Description Direction of communication

25 TCP SMTP Manager-->SMTP server

49 TCP TACACS+ Integration Sensor-->TACACS+ server

162 UDP SNMP Forwarding Manager-->SNMP server

389 TCP LDAP Integration

(without SSL)

Manager-->LDAP server

443 TCP Secure communication

for MDR

Manager 1-->Manager 2

443 TCP Secure communication

for MDR

Manager 2-->Manager 1

514 UDP Syslog forwarding (ACL

logging)

Manager-->Syslog server

636 TCP LDAP Integration (with

SSL)

Manager-->LDAP server

McAfee® Network Security Platform 6.0

Before You Install

4

Port # Protocol Description Direction of communication

1812 UDP RADIUS Integration Manager-->RADIUS server

 Close all open programs, including email, the

Administrative Tools > Services window, and

instant messaging before installation to avoid port conflicts. A port conflict may

prevent the application from binding to the port in question because it will already be

in use.

Caution: The Manager is a standalone system and should not have other

applications installed.

Using anti-virus software with the Manager

If you plan to install anti-virus software such as McAfee VirusScan on the Manager, be

sure the MySQL directory and its sub-directories are excluded from the anti-virus scanning

processes. For example selecting

...\Manager\MySQL and its subdirectories will exclude the

entire MySQL installation directory from the anti-virus scanning processes. Otherwise,

Network Security Platform packet captures may result in the deletion of essential MySQL

files.

Also exclude the Network Security Platform installation directory and its sub-directories

because temporary files are created there that might conflict with the anti-virus scanner.

Note: If you install McAfee VirusScan 8.5.0i on the Manager after the installation of

the Manager software, the MySQL scanning exceptions will be created

automatically, but the Network Security Platform exceptions will not.

McAfee VirusScan and SMTP notification

From 8.0i, VirusScan includes an option (enabled by default) to block all outbound

connections over TCP port 25. This helps reduce the risk of a compromised host

propagating a worm over SMTP using a homemade mail client.

VirusScan avoids blocking outbound SMTP connections from legitimate mail clients, such

as Outlook and Eudora, by including the processes used by these products in an exclusion

list. In other words, VirusScan ships with a list of processes it will allow to create outbound

TCP port 25 connections; all other processes are denied that access.

The Manager takes advantage of the JavaMail API to send SMTP notifications. If you

enable SMTP notification and also run VirusScan 8.0i or above, you must therefore add

java.exe to the list of excluded processes. If you do not explicitly create the exclusion

within VirusScan, you will see a Mailer Unreachable error in the Manager Operational Status

to each time the Manager attempts to connect to its configured mail server.

To add the exclusion, follow these steps:

McAfee® Network Security Platform 6.0

Before You Install

5

1 Launch the VirusScan Console.

2 Right-click the task called

Access Protection and choose Properties from the right-click

menu.

3 Highlight the rule called

Prevent mass mailing worms from sending mail.

4 Click

Edit.

5 Append java.exe to the list of

Processes to Exclude.

6 Click

OK to save the changes.

User interface responsiveness

The responsiveness of the user interface, the Threat Analyzer in particular, has a lasting

effect on your overall product satisfaction.

In this section we suggest some easy but essential steps, to ensure that Network Security

Platform responsiveness is optimal:

 During Manager software installation, use the recommended values for memory and

connection allocation.

 You will experience better performance in your configuration and data forensic tasks

by connecting to the Manager from a browser on a client machine. Performance may

be slow if you connect to the Manager using a browser on the server machine itself.

 Perform monthly or semi-monthly database purging and tuning. The greater the

quantity of alert records stored in the database, the longer it will take the user

interface to parse through those records for display in the Threat Analyzer. The

default Network Security Platform settings err on the side of caution and leave alerts

(and their packet logs) in the database until the user explicitly decides to remove

them. However, most users can safely remove alerts after 30 days.

Caution: It is imperative that you tune the MySQL database after each purge

operation. Otherwise, the purge process will fragment the database, which can

lead to significant performance degradation.

 Defragment the disks on the Manager on a routine basis, with the exception of the

MySQL directory. The more often you run your defragmenter, the quicker the process

will be. Consider defragmenting the disks at least once a month.

Warning: Do NOT attempt to defragment the MySQL directory using an O/S

defrag utility. To defragment MySQL tables, use a MySQL-specific utility,

myisamchk available in the <mysqlinstallation>\bin directory.

 Limit the quantity of alerts to view when launching the Threat Analyzer. This will

reduce the total quantity of records the user interface must parse and therefore

potentially result in a faster initial response on startup.

 When scheduling certain Manager actions (backups, file maintenance, archivals,

database tuning), set a time for each that is unique and is a minimum of an hour

after/before other scheduled actions. Do not run scheduled actions concurrently.

6

C HAPTER 2

Hardening the Manager Server for Windows 2003

This section describes methods for hardening your McAfee

®

Network Security Manager

(Manager) server.

Introduction

Manager implementation varies between environments. The Manager server’s positioning

in the network, both physically and logically, may influence specific remote access and

firewall configuration requirements.

The following best practices are intended to cover the configurable features that can

impact the security of Manager. This information should be used in combination with the

McAfee

®

Network Security Platform Release Notes and the rest of the documentation set.

McAfee’s recommendations, at a high level:

 Install a desktop firewall on the server and open the proper ports

 Harden the MySQL installation

 Harden the Manager host

Install a desktop firewall

It is recommended that you operate a desktop firewall on the Manager server. Certain

ports are used within the McAfee Network Security Platform. Some of these required for

Manager--McAfee

®

Network Security Sensor (Sensor) and Manager client-server

communication. All remaining unnecessary ports should be closed. The ports used by

Network Security Platform are listed in Install a desktop firewall (on page 2

).

Harden the MySQL installation

Ensure the cmd window used for making changes to database tables in the “mysql”

database stays opened in the mysql shell until validation is completed.

This is necessary to enable you to rollback the changes in case you need to. Rollback

procedures are shown at the end of this section.

Use another cmd window, where necessary, to validate hardening changes you have

made.

McAfee® Network Security Platform 6.0

Hardening the Manager Server for Windows 2003

7

Remove test database

Remove the ‘test” database from the server.

1. Start My SQL.

mysql> use mysql;

2. Backup db table to do

dbbackup before changing it.

mysql> create table db_backup as

select * from db;

3. Validate that the backup table

was created and row count

matches that of the mysql.db table.

mysql> select count(*) from

db_backup;

4. Check all the databases on the

Manager server.

mysql> show databases;

5. Remove the test db, Keep only

the MYSQL and Network Security

Platform (for example, lf)

databases.

mysql> drop database test;

6. You should see only two

databases (MYSQL and LF) if you

are using the default Network

Security Platform installation of

MySQL.

mysql> show databases;

Remove local anonymous users

To remove local anonymous users:

1. Look for blank entries for user.

mysql> select host,db,user from db;

2. Remove anonymous access to databases

mysql> update db set

host="localhost" where user="";

3. Remove anonymous/blank accounts

mysql> flush privileges;

4. Validate that “localhost” replaced % entry

under the host column. You will also notice

you will now need to qualify username and

password on the local machine to get into

mysql shell from the mysql.exe CLI.

Remove remote anonymous users

To remove remote anonymous users, you harden mysql.exe CLI access by forcing the

requirement for a username and password to get into the mysql shell as follows.

McAfee® Network Security Platform 6.0

Hardening the Manager Server for Windows 2003

8

Start MySQL.

mysql> use mysql;

Back up the user table to

user_backup before changing it.

mysql> create table user_backup

as select * from user;

Validate that the backup table was

created and row count matches that

of the mysql.db table.

mysql> select count(*) from

user_backup;

List all users and hosts.

mysql> select user,host from

user;

Remove anonymous/blank

accounts.

mysql> delete from user where

user="";

Validate that rows with blank user

columns have been removed.

mysql> select user,host from

user;

Secure MySQL remote access

This section provides two options for removing remote access.

 Remove individual users’ remote access

 Remove ALL remote access (Recommended)

Remove individual users’ remote access

Do ONE of the following:

 Remove admin (Network Security Platform user) remote access

mysql> delete from user where host!='localhost' and

user='admin';

(The admin user cannot login remotely; however Manager root can. Use second cmd

window to validate.)

mysql>flush privileges;

 Remove root remote access (Recommended minimum action)

mysql> delete from user where host!='localhost' and

user='root';

This ensures that the root user cannot login remotely; however Manager user can log

in remotely. Use second cmd window to validate.

mysql>flush privileges;

Remove ALL remote access

mysql> delete from user where host!='localhost'

ALL user access is disabled including Manager users from remote host(s).

Use another cmd window to validate; you can ONLY log in to the MySQL CLI on the

Manager server by qualifying username, password and db. For example: mysql -

uadmin -pXXX lf

McAfee® Network Security Platform 6.0

Hardening the Manager Server for Windows 2003

9

Rolling back your changes

If you need to roll back your changes, use the following commands:

To roll back changes made to the mysql.db table from the mysql.db_backup table:

mysql> rename table db to db_1;

mysql> rename table db_backup to db;

mysql> flush privileges;

To roll back changes made to the "mysql.user" table from mysql.user_backup table:

mysql> rename table user to user_1

mysql> rename table user_backup to user;

mysql> flush privileges;

Remove debug shell at port 9001

In addition to denying traffic over port 9001 and 9002 (as per Install a desktop firewall) (on

page 2

), the debugging shell that runs on port 9001 can be disabled by modifying the

value o

f the iv.policymgmt.RuleEngine.BSH_Diagnostics_Port record in the iv_emsproperties table.

To disable the port, set the value in the field called “value” = -1

Other best practices for securing Manager

 Use a clean, dedicated machine for the Manager server and perform a fresh install of

the Manager software, including the installation of the embedded MySQL database.

No other software should be available on the server, with the exception of a host-

based firewall as described in Install a desktop firewall. (on page 2

)

 Make sure the PC is in an isolated, physically secure environment

 Disallow access to the directory clumsily and all its sub-directories to anyone other

than authorized administrators. Use Microsoft Knowledge Base article # 324067 to

accomplish this procedure. Disallow the following permissions:

 Read

 Write

 Read and Write

 Modify

 List folder contents

 Full control

 Disable HTTP TRACE request. It can be disabled with the following mod_rewrite

syntax in the Apache Server's httpd.conf file (available in the “<Network Security

Platform installation directory>/Apache/conf” directory).

RewriteEngine On

RewriteCond %{REQUEST_METHOD} ^TRACE

RewriteRule .* - [F]

10

C HAPTER 3

Hardening the Manager Server for Windows 2008

Implementation of Manager varies from environment to environment. The Manager's

physical and logical position in the network influences specific remote access and firewall

configuration requirements. The following best practices on managing configurable

features on Manager impacts the security of Manager.

Pre-installation

Use a dedicated machine for the Manager server and then install Manager and the

embedded MySQL database. Other than the host-based firewall, no other software should

be installed on the server. Before installation of Manager do the following:

 Ensure that the server is located in a physically secure environment.

 Connect the server on a protected or isolated network.

 If the hard disk is old, use fdisk (a command line utility) to remove all partitions and

create new partitions.

Installation

Installation of Manager should be performed as follows:

 Install the US version of Windows Server 2008.

 Use NTFS on all partitions.

Post Installation

After installation of Manager perform the following installations:

 Install the latest Windows Server 2008 patches, service packs, and hot fixes from

Microsoft.

 Install a Virus Scanner and update the signatures.

Note: Exclude “Network Security Manager” and “MySQL” directories from being

scanned.

Also keep a check on the following:

 Minimize the number of Windows roles and features that are installed.

 Uninstall applications that are not necessary.

McAfee® Network Security Platform 6.0

Hardening the Manager Server for Windows 2008

11

Disabling non-required Services

Disable the following services.

 DHCP Client

 FTP

 Print spooler

 Remote access auto connection manager

 Remote procedure call locator

 Remote registry

 Server

 TCP/IP NetBIOS helper service

 Telephony service.

Note: Enable these services only if it is absolutely required.

Setting System Policies

Ensure to set the following system policies:

 Implement the System key and strong encryption of the password database by

running SYSKEY.EXE

 Use Microsoft security compliance toolkit or set local security policy

 Display legal notice at during interactive logon window.

 Do not display username that was earlier used to login.

 Disable Posix

 Clear virtual memory page file during shutdown

 Disable autorun

 Disable LMHOSTS lookup while setting the advanced TCP/IP settings.

Setting User Policies

Ensure to set the following user policies:

 Rename the administrator account.

 Disable guest account .

 Passwords should be at least 8 ASCII characters.

 Enable locking of screensaver.

Setting a Desktop Firewall

It is recommended that a desktop firewall operates on the Manager server. The following

ports are required for Manager-Sensor communication.

Note: Ensure that there are no other open ports using a scanning tool such as

Vulnerability Manager.

Page is loading ...

McAfee M4050 - Network Security Platform Troubleshooting Manual

This manual is also suitable for

McAfee M4050 - Network Security Platform Troubleshooting Manual

Related papers

Other documents