Pelco VideoXpert System User manual

Type
User manual
C5673M-C | 1/17
SYSTEM DESIGN GUIDE
VideoXpert
System Design Guide
Contents
Standard System Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Media Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Recorders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
VideoXpert Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
VSM/NSM5200 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Digital Sentry (DSSRV2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Ops Center Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
Enhanced Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Additional System Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
VideoXpert Accessory Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
NTP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
DHCP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Independent Backup Storage/Export Archive Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Load Balancer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
Load Balancer Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
System Design and Typical Deployments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Active-Active failover (Single CMG vs Multi-CMG Environments) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Clustering Core Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Configurations on Clustered Cores and Time Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
VideoXpert Storage Failover and Redundant Recording . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Aggregating Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Settings for Aggregated Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
LDAP Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Extended Workstation Topology and Enhanced Decoders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Advanced Workstation Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Network Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Traffic and System Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Secondary Stream Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Optera Camera Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Client-side Display Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Stream Delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Ops Center Stream Adjustments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
MJPEG Video . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Media Gateway Communication Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Configuring Secondary Streams to Support VideoXpert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Network Port Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Exports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Media Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
VSM / NSM5200 Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
Enhanced Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Recording Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Additional Storage Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Camera Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Assigning Cameras to Recorders in VideoXpert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
Using VideoXpert Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Using VSM and NSM5200 Models as VideoXpert Recorders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Using Digital Sentry Servers as VideoXpert Recorders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Advanced VideoXpert Storage Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Checking VideoXpert Storage Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Storage Assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Assigning an NTP Address to VideoXpert Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Using External NAS Storage (Archive Volume Group) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Configuring an External Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Database Backups and Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Restoring the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Rebuilding the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
VideoXpert Storage Failover Recording . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Configuring Failover Recording . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Discovering, Adding, and Commissioning Devices in VideoXpert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Adding Pelco Cameras and Devices or Third-Party Servers Running VideoXpert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Adding Third-Party Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Ensuring Driver Support Using VX Storage Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
VideoXpert 3rd Party Edge Device Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Licensing the System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Licensing Aggregated Video Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
SNMP Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Enabling SNMP Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Recommended Monitoring Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Software/Services to Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Hardware Conditions to Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
Migrating from Endura to VideoXpert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Migrating the Endura Database to VideoXpert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Migrating from Digital Sentry to VideoXpert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Integrations and Plug-ins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39
Mapping Plug-in . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40
Event Viewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41
Event Types and Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Camera Configuration Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
Online/Offline Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
Recorder Camera Assignment Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Recorder Diagnostics Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
User Actions Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
Event History Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
VX Cluster Configuration Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Analytic and Other Video-Related Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
Logs and Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
Appendix: Network Operation Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
PIM Dense Mode for Multicast Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
PIM Dense Mode Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
Considerations When Using PIM-DIM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47
PIM Sparse Mode Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Considerations when using PIM-SIM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Topology Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Layer 2 Network Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Layer 3 Network Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Static Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Unicast Routing Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Multicast Routing Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50
5
Standard System Components
Your system must possess at least one Core, one Media Gateway, and one storage server. In a basic system, a single server can host
both the Core and Media Gateway applications, or you can host both applications on separate servers.
Figure 1: Basic System Topology
Servers
VideoXpert requires both Core and Media Gateway servers. While you can leverage separate Core and Media Gateway servers for
systems of sufficient scale, most systems can easily support servers running both the Core and Media Gateway services. A single CMG
server provides the complete range of VideoXpert functionality that you would expect for systems with fewer than 2000 cameras and
100 simultaneous users.
However, for environments that are especially large, require exceptional redundancy, or incorporate a high number of low bandwidth
and aggregated users, you may want to install individual Core and Media Gateway servers.
Core
The Core maintains the system database, manages user permissions, and hosts the Admin Portal, from which you can configure and
manage your VideoXpert system. The Admin Portal also provides basic access to video streaming resources, ensuring that users with
appropriate permissions can view video from the system even without access to a thick client.
You can add Core servers to your VideoXpert system modularly, to both expand the capabilities of your VideoXpert system and to build
in redundancies that maximize system uptime.
LOCAL
OPERATORS
CORE/
MEDIA GATEWAY
3RD PARTY
SYSTEM
CUSTOMER FACILITIES
OPS CENTER CLIENT
LOCAL
ADMINISTRATORS
ADMIN PORTAL
STORAGE
eConnect
ULTIMATE SYSTEM
6
Media Gateway
The Media Gateway routes video traffic to users. When a user requests a video stream, the Core fields the request and determines
whether the user possesses the appropriate permissions to view the video stream. If the user can view the stream, the Media Gateway
connects the user with the appropriate device — camera, encoder, or recorder as necessary. The Media Gateway also transcodes
MJPEG streams for low bandwidth connections (roughly, but not limited to, 20 at a time).
Like Core servers, Media Gateways can be added to VideoXpert modularly, to both expand the system’s capabilities or build redundan-
cies within the system to maximize video availability.
Recorders
VideoXpert supports multiple recording platforms. In general, traffic and video delivery operates much the same using any of the avail-
able recording platforms. If building a new system, you would typically use VideoXpert Storage and VSM recorders; if migrating from an
existing platform, you would typically use NSM5200 and Digital Sentry servers as recorders.
VideoXpert Storage
VideoXpert Storage is the native recording platform for VideoXpert, supporting all recorder and camera assignment functionality within
VideoXpert. It supports 3rd-party cameras natively, eliminating the need for DSSRV2 storage or UDI5000-CAM units with NSM5200 and
VSM recorders, and has a higher maximum capacity (up to 96 TB) than all other recording platforms for VideoXpert.
VSM/NSM5200
VSM and NSM5200 recorders support 3rd party cameras within VideoXpert using a UDI5000-CAM, and provide basic assignment
controls from within VideoXpert, but require Pelco Utilities to perform some advanced configuration. Both can act as VideoXpert
recorders while continuing to support an Endura environment, providing an easy migration path and a smooth transition from Endura into
VideoXpert.
In addition to the native solution, VideoXpert supports VSM, NSM5200, and Digital Sentry servers as recorders. NSM5200 and Digital
Sentry servers provide migration paths to VideoXpert; both can operate as recorders within VideoXpert while continuing to support
within their respective Endura and Digitial Sentry environments.
However, these recording platforms may require additional setup outside of VideoXpert. For VSM and NSM5200 models, you may have
to perform some configuration using Pelco Utilities.
Digital Sentry (DSSRV2)
DS recorders provide a migration path from DS to VideoXpert; when using a DS server as a VideoXpert recorder, the DS server will
continue to run all DS operations and is still available to clients using DS ControlPoint. However, when using DS servers as VideoXpert
recorders, users must perform all camera assignment and recorder configuration through the DS platform.
Ops Center Client
The Ops Center client is the Windows-based client application, providing an optimal environment from which users can watch live and
recorded video. With the support of Enhanced Decoders, Ops Centers can support up to six monitors, each capable of displaying up to
16 video streams simultaneously. Through the Ops Center, users can configure and recall complete workspaces, enabling operators to
quickly log in and get to work.
The Ops Center supports modular plug-ins, exposing additional functionality for operators as required by your surveillance operations.
Many plug-ins are configured directly through the Ops Center to the 3rd-party server feeding information into VideoXpert.
7
Enhanced Decoder
The Ops Center supports up to six monitors: one connected directly to the workstation itself, and the other five driven independently by
Enhanced Decoders. The Enhanced Decoders enable each monitor connected through a workstation to display up to 16 high definition
video streams or independent plug-ins while maintaining a seamless user experience. Enhanced Decoder-driven monitors operate just
like native monitors; users can move windows across monitors seamlessly. But, when the user requests a video stream or plug-in, the
Enhanced Decoder communicates directly with VideoXpert servers to get and decode video. This enables operators to maximize the
display-capabilities of the Ops Center without complicating the user experience.
Additional System Components
Depending on the shape and scale of your system, you may need one or more of the following components.
VideoXpert Accessory Server
The VideoXpert Accessory Server provides NTP, simple DHCP, and Load Balancing services for small to moderately-sized systems. The
load balancer functionality within the accessory server is designed for dual CMG environments, providing failover monitoring services
and acting as the dual-CMG cluster’s arbiter. For larger environments, you will want to consider an independent load balancer with a
higher capacity.
NTP Server
VideoXpert requires NTP synchronization. If components within the system fall out of time synchronization, the system may experience
recording gaps or inaccurate video retrieval. It is highly recommended that you have a dedicated NTP server for VideoXpert, like the Vide-
oXpert Accessory Server.
DHCP Server
You should use a DHCP server to assign and manage addresses for the devices within your VideoXpert network. The VideoXpert acces-
sory server acts as a simple DHCP server. For more advanced installations with multiple ranges and a need to edit address reservations,
use a separate, dedicated DHCP server.
Independent Backup Storage/Export Archive Storage
For any system with more than one Core, you must backup to a network location. It is recommended that you perform backups from a
server independent of other VideoXpert hardware, preferably containing its own RAID array, ensuring that your backups are safe from
catastrophic failures.
Presently, VideoXpert has been tested using Pelco’s backupInteractive utility and Mongo Management Services (MMS).
You can also store exports in a network location off the Core. Storing exports off the Core saves Core bandwidth in the export process
(both in creating and downloading), and ensures greater availability to exported video, especially in clustered environments. When
storing exports on the Cores in a clustered environment, exports are not shared among Cores, nor are they a part of the backup process;
each export is only stored on the Core server on which it was created. If the server storing an export fails, users lose access to the
exported video; if the server fails and you have to restore from a backup, you will lose your exports on that Core server. Storing exports
in a separate location ensures availability independent of any individual Core server, and allows you to easily backup video exports at
whatever interval is most convenient for you.
8
Load Balancer
In most cases, you should use the VideoXpert Accessory Server as your load balancer.
However, for systems with more than 3 CMGs or multiple independent Core and Media Gateway servers, you may need an independent
load balancer. The load balancer provides the virtual IP address used to target clustered servers. For clustered systems, the load balancer
may provide additional services assisting in device discovery. VideoXpert has been tested with the VideoXpert Accessory Server and
Brocade ADX load balancers.
NOTE: When using a Brocade ADX load balancer, your VideoXpert environment must be configured for Multicast.
Load Balancer Requirements
If selecting a load balancer other than the Brocade ADX Series or the VideoXpert Accessory Server, your load balancer should meet the
following requirements.
High-Level Requirements
HTTP
HTTPS
websocket support
RTSP support; alternatively, support for raw TCP/IP connections
Support for multiple sets of backend servers; necessary if using separate Core and Media Gateway servers
Support for application-level HTTP healthchecks
Support for application-level RTSP or TCP/IP health-checks
High Availability Requirements
Mut sbe able to configure load balancer appliances such that if an appliance becomes unavailable (e.g. power loss), the
other appliance(s) can assume functionality of the unavailable appliance.
Recommended Features
Allow administrators to gracefully disable servers during upgrades
SSL offloading
Support for UDP (for auto-discovery spoofing)
Source IP stickiness (for auto-discovery spoofing)
9
System Design and Typical Deployments
The total system capacity for VideoXpert is governed by Core and Media Gateway servers. Most customers and deployments should
consider CMG servers (servers running both Core and Media Gateway services). In standard CMG deployments, you can add Media Gate-
ways to the system when there are significant remote access requirements (transcoding/transcasting), or there are significant aggrega-
tion requirements.
If your system grows to support additional users, cameras, sites, or you just want to provide redundancy within your VideoXpert system,
you can separate your Core and Media Gateway servers, and increase system capacity by clustering servers. For Ultimate distributions
of VideoXpert, you can also aggregate other VideoXpert systems.
You should consider separate Core and Media Gateway deployments when:
There are an inordinately high number of simultaneous users.
The system must scale to an extremely high number of cameras and users.
You have extremely high expectations for availability and redundancy.
Table A: Basic Cluster Topology (left) vs Advanced Cluster Topology (right)
LOCAL
OPERATORS
CORE/
MEDIA GATEWAY
3RD PARTY
SYSTEM
CUSTOMER FACILITIES
OPS CENTER CLIENT
(Hosting Arbiter)
LOCAL
ADMINISTRATORS
ADMIN PORTAL
VSM5200
eConnect
ULTIMATE SYST EM
Windows NLB
Service
LOCA L
OPERATORS
CORE
MEDIA
GATEWAY
LOAD
BALANCER
3RD PARTY
SYSTEM
CUSTOMER FACILITIES
OPS CENTER CLIENT
LOCA L
ADMINISTRATORS
ADMIN PORTAL
Storage
eConnect
ULTIMATE SYSTEM
LOCA L
OPERATORS
REMOTE
OBSERVER
CUSTOMER FACILITIES
REMOTE CLIENT
ADMIN PORTAL
ENTERPRISE
10
The table below lists typical deployment scenarios, with the maximum number of cameras and concurrent system users for each deploy-
ment; these numbers represent the limits at which the system becomes unusable (high latency in control requests). You should build
your system with leaving at least 10% additional capacity (in terms of cameras, users, or, preferably, both), to ensure that the system is
both responsive and has additional capacity to take on new users or cameras.
NOTE: The table below shows absolute maximums for VideoXpert deployment scenarios. Your experience may differ based on your
network configuration, network equipment, average video bitrates, and other criteria. When planning a VideoXpert deployment, contact
Pelco to ensure that you get a system with the capacity to support your environment and needs.
Active-Active failover (Single CMG vs Multi-CMG Environments)
While a single CMG can host nearly 2500 cameras and 100 concurrent users, the system is not at all fault tolerant; anything that could
bring down the server will interrupt access to VideoXpert. It is recommended that even in ssmaller installations, you build a system with
at least two CMG servers.
When using the VideoXpert Accessory Server with a dual CMG server, you are engaging both an active-active failover system, and
increasing the system capacity. The Accessory Server acts as a load balancer between the two CMGs, keeping both servers active; in
the event that either server fails, the other will shoulder the load.
Deployment Cameras Users Availability
Additional
Requirements
Single CMG 2500 100 Not fault tolerant N/A
Dual CMG 2500 100 Active-Active single
failover
VideoXpert Accessory
Server
Triple CMG 10000 500 High Availability VideoXpert Accessory
Server
Single Core/ Gateway 3000 200 Not Fault Tolerant N/A
Multi-Core / Gateway >10000 >500 High Availability Independent load
balancer (Brocade
ADX); Pelco Services
11
Clustering Core Servers
In your VideoXpert environment, Core (or CMG) servers host the database. Clustering your Core or CMG servers provides redundancy and
enables VideoXpert to scale.
In a clustered environment, each Core or CMG server in the cluster hosts a complete copy of the VideoXpert database, and each server
is assigned to a replica set – a group of servers organized to maximize the availability of VideoXpert. The number of replica sets in your
cluster is equal to the number of servers divided by three. Individual servers or a dual-CMG environments represent a single replica set.
NOTE: VideoXpert itself performs much of the cluster configuration automatically during the setup process. This automatic configuration
is based on a single database replica set. If configuring a system containing multiple replica sets or more than three servers, contact
Pelco Professional Services.
Within each replica set, one server acts as the primary and the other servers operate as secondaries. The primary processes all write
operations and pushes data to the secondaries. Secondary servers replicate the primary server’s database asynchronously and can, if
necessary, process “read” operations. (By default, VideoXpert is configured to read from the primary server when available.)
If the primary server fails, the secondary servers elect a new primary; this election requires the remaining two servers in the replica set
to be available and able to communicate with each other. If the primary server and a secondary server within the replica set fall offline,
or the primary CMG in a dual-CMG cluster falls offline, the replica set becomes read only until at least one of the remaining servers is
restored. In a read-only state, users can still call up video, but would be unable to apply bookmarks, export investigations, apply tags,
and other similar operations within the system.
To prevent the cluster from falling into a read-only state, you can configure a single server as an arbiter. The arbiter does not host the
database, but rather casts the deciding vote in elections determining which secondary server is promoted if a primary server fails. Any
non-Core server in the VideoXpert environment, including VideoXpert Accessory Servers, can act as the arbiter. Typically, you should
employ an arbiter to ensure that your cluster contains an odd number of servers.
VideoXpert automatically assigns three “config” servers, no matter the size of your installation. For VideoXpert installations with fewer
than three servers, one of the servers will host multiple “config” servers. Config servers store the metadata information for the data-
base, which takes up very little disk space. VideoXpert assigns config servers automatically when you configure your Core or a cluster
of Cores.
Configurations on Clustered Cores and Time Considerations
You cannot configure an Accessory Server to manage your cluster until you configure your Core or CMG servers to use the same virtual
IP address.
When attempting to cluster CMG servers using an Accessory Server, you must at least perform basic configuration for one CMG servers,
providing the IP address of the other CMG servers in the cluster and a virtual IP address. When you save the configuration on the initial
server, including the virtual IP address, the additional CMG servers in the (not yet complete) cluster will copy configuration information
and the system database from the initial server so long as the system time for all the servers in the cluster are within 30 seconds
of each other. If the difference in the system time between servers is greater than 30 seconds, cluster setup will fail.
Primary CMG
Secondary CMG
Accessory Server/Arbiter
Heartbeat
Database Replication
12
VideoXpert Storage Failover and Redundant Recording
Recorders are not part of the standard Core/CMG cluster. However, typical deployments should arrange for some measure of redundancy
to ensure that required streams and video never go unrecorded. VideoXpert Storage supports two methods for redundancy:
Redundant recording: assigning the same camera to multiple recorders
Failover monitoring: assigning a recorder to act as a hot standby for one or more recorders.
Redundant recording is a quick way to ensure video uptime and redundancy, but requires an equal amount of storage across each
recorder set to record a stream.
Designating a VideoXpert Storage recorder allows you to assign the recorder to monitor and act as a hot standby for one or more
recorders; if any of the recorders the failover unit is assigned to monitor fails, the failover unit will take over and continue recording the
camera groups and schedules previously belonging to the failed recorder.You should not over-assign failover capacity. If the failover unit
is already engaged and another recorder fails, those cameras will go unrecorded until you can correct the issue. Failover recording also
doesn’t ensure access to historical data from failed recorders; it only ensures uninterrupted recording. If you want to maintain uninter-
rupted access to recorded video, even when a recorder fails, then you should record cameras redundantly across multiple recorders.
A designated failover recorder operates one-to-one with the recorders it monitors; should a unit fail, it will take over for the cameras
and schedules belonging to the failed unit; it cannot absorb additional capacity from any other failures within the group. So, while a
single, designated failover recorder can monitor up to 8 active recorders, you should implement a failover strategy that ensures full time
recording and video availability in your environment in the event of possible network, power, or hardware failures.
NOTE: During a failover, you may experience a recording gap of up to 30 seconds.
Aggregating Systems
VideoXpert Ultimate Core systems include an aggregation server, through which you can provide centralized access to a series of Vide-
oXpert member systems. Through the VideoXpert Ultimate server acting as the aggregation host, you can access and control settings
and video for distributed VideoXpert systems.
When adding a member to the aggregation server, you will select your connection speed to the aggregation server. Your connection
speed determines both the performance of video within the VideoXpert Ultimate environment hosting the aggregation server and the
number of video streams you can reasonably expect to get simultaneously from the aggregated site.
Settings for Aggregated Systems
At present, you cannot change settings for aggregated systems from the VideoXpert instance hosting the aggregation server. You must
change settings for member systems from the member itself.
The aggregation server does not inherit permissions, roles, or users from aggregated members. If aggregating a VideoXpert Enterprise
environment containing roles with restricted permissions, you must re-create these roles and permissions with resource restrictions with
the Ultimate system acting as the aggregation server.
13
LDAP Authentication
You can configure VideoXpert to validate user credentials from an LDAP server; VideoXpert supports LDAP for Microsoft Active Directory.
While the system can validate credentials over LDAP, you must create corresponding users within VideoXpert to validate credentials
against. You must also assign roles and permissions through VideoXpert; there are no analogs for VideoXpert permissions in Active
Directory or LDAP.
NOTE:
After changing a password in an environment using LDAP authentication, a user’s old password may still work for a short period of
time (less than one hour). This is a Windows NTLM authentication behavior, designed to allow accounts logged in to multiple
computers or network services to access the network while the password change propagates.
LDAP Authentication does not affect restricted system users. When LDAP is enabled, you can still change passwords for admin,
aggregator, and rule_engine_user accounts through the Admin Portal; these accounts are authenticated by VideoXpert no matter
how you authenticate other users.
While VideoXpert may support other LDAP authentication schemes, this guide focuses on -- and the product has been tested against --
two common methods for LDAP authentication: simple (single bind) and two-stage bind with a service account:
Simple (single-bind) authentication, which associates a user name and password with a distinguished name (DN) to validate user
credentials.
Two-stage bind authentication: which uses a service DN and a service DN password to establish the initial bind with the LDAP
server. Upon a successful initial bind, a search is performed using the base DN, the first key from the Search Attributes box, and
the user name. If the user’s information is found, the second bind is performed using the DN to authenticate user credentials.
14
Extended Workstation Topology and Enhanced Decoders
The Ops Center supports up to six monitors: one connected directly to the workstation itself, and the other five driven independently by
Enhanced Decoders. The Enhanced Decoders enable each monitor connected through a workstation to display up to 16 high definition
video streams or independent plug-ins while maintaining a seamless user experience.
Enhanced Decoder-driven monitors operate just like native monitors; users can move windows across monitors seamlessly. But, when
the user requests a video stream or plug-in, the Enhanced Decoder communicates directly with VideoXpert servers to get and decode
video. This enables operators to maximize the display-capabilities of the Ops Center without complicating the user experience.
Because the decoders do not run the Ops Center application themselves, you may experience better performance in quantity and respon-
siveness of HD streams on decoder-driven monitors over directly-connected monitors.
NOTE: Enhanced Decoders work best with Windows 8 or later. When using Enhanced Decoders with Windows 7, you should install the
VF Mirage driver for best performance.
Ops Center
Monitor 5
Monitor 4
Monitor 3Monitor 2
Monitor 1
Monitor 6
Legend
CAT5e
HDMI
DVI-D
Enhanced
Decoder
Enhanced
Decoder
Enhanced
Decoder
Enhanced
Decoder
Enhanced
Decoder
15
Advanced Workstation Controls
In addition to the standard keyboard and mouse, Ops Center workstations support an Enhanced Keyboard, and Enhanced Mouse, and the
KBD5000.
The Enhanced Keyboard has 10 keys to which users can map tab and workspace shortcuts, enabling users to quickly recall their
most-used layouts and cameras.
The KBD5000 and Enhanced Mouse are mapped to the full range of Ops Center controls. A user can perform any and all actions within
the system using either item.
KBD5000 ENHANCED KEYBOARD SPACEMOUSE
16
Network Traffic Flow
Command and control traffic - user actions within the system, occur over HTTP or HTTPS depending on your system. You can configure
workstations to operate over HTTP or HTTPS, and you can select the port for communications with VideoXpert Servers.
Video is delivered to clients either via RTSP or RTP, depending on the Media Gateway Communication method for which your system is
configured. When configured for unicast delivery to clients, the Media Gateway re-streams video to the client. When configured for
multicast delivery to clients, video streams directly from the camera or non-DS recording device from which the stream is requested.
Note that when using Digital Sentry systems as storage for VideoXpert, you must set the Media Gateway Communication Method
to unicast.
Figure 1: Traffic in a VideoXpert System
MEDIA GATEWAY
OPS CENTER CLIENT
ADMIN PORTAL
VSM/NSM5200
LOAD BALANCER/NLB
CORE
HTTP/HTTPS
RTSP
RTP
NSM
Pelco API
VX Storage
17
Traffic and System Limitations
VideoXpert systems were tested to determine how many users and cameras a system supports before performance degrades signifi-
cantly. Testing determined degraded performance by command latency (opening cameras, executing PTZ commands, etc) of 6 second or
greater, or failed commands. The systems tested represented environments with strong network connections using VSM models for
storage.
For purposes of performance tests, “users” are simultaneous operators performing continuous, expected duties, including streaming
video, receiving events, controlling (PTZ) cameras, and exporting video.
Figure 2: VideoXpert Performance
NOTE: Actual system performance is dependent on the speed and quality of network connections, network switch and router choices,
and any non-VideoXpert traffic impacting the network.
Secondary Stream Settings
While the Ops Center employs step down behaviors when under heavy load, optimal camera configurations will ensure you always view
the highest possible quality video and prevent the system from entering i-Frame only or disconnection step-down scenarios.
Set Secondary Streams to 640 x 352 at 5 images per second; these settings ensure secondary stream performance in a 4x4 layout.
Optera Camera Settings
While the Ops Center employs step down behaviors when under heavy load, optimal camera configurations will ensure you always view
the highest possible quality video and prevent the system from entering i-Frame only or disconnection step-down scenarios.
For Optera Cameras, set the I-Frame interval to 6 and the bit rate to 10000 kbps. Optera cameras consume more system resources than
other cameras. You should not expect to view more than 2 Optera cameras per decoder.
18
Client-side Display Limitations
As an Ops Center workstation uses an increasing amount of memory, users may experience “jittery” mouse controls on decoder-driven
monitors. This problem is most likely to occur if users are running applications in tandem with the Ops Center client, or is viewing a large
number of Optera or HD streams on locally-connected monitors.
For Ops Center workstations using Enhanced Decoders, install the Mirage driver for best performance. The driver is available from the
Monitor Configuration screen, which you can access during initial setup or by logging in as the workstation’s local administrator.
Stream Delivery
Stream delivery, largely the responsibility of the Media Gateway, is dependent on the communication method you select, your connec-
tion to the system, and the connection of aggregated devices to the system. In general, you can receive a primary stream, a secondary
stream, a primary stream in i-Frame only mode, or an MJPEG stream.
Because the system delivers secondary streams as a step-down adjustment, you’ll want to configure secondary streams accordingly,
with reduced resolution and framerate by comparison with the primary.
Ops Center Stream Adjustments
By default, the Ops Center displays streams based on the size of the cells in a tab. Layouts at 2x3 or fewer cells display full resolution
primary streams; layouts with more than 6 cells deliver secondary streams.
When CPU load on the decoding device (the Workstation or Enhanced Decoder) displaying streams crosses the 75% threshold, the Ops
Center will begin requesting lower-quality streams in an attempt to reduce CPU load below 50% utilization. Stream quality is reduced
from primary to secondary, and secondary to i-frame only mode until the CPU load drops below 50%. If the reduction in stream quality
does not reduce the CPU load below the 50% threshold, the Ops Center will begin disconnecting streams citing limited CPU resources.
Stream Quality Ops Center or Admin Portal Factors
Primary Ops Center Bandwidth >100 Mbps; layout <2x3
Secondary Ops Center Layout >2x3; quality step-down when the
Ops Center is under heavy load
i-Frame Only Ops Center Recording; quality step down (from
secondary stream) when Ops Center
under heavy load
MJPEG Ops Center/Admin Portal Admin Portal: Live View Page
Ops Center: Client bandwidth <100
Mbps; Aggregated System bandwidth
<100 Mbps; Aggregated Cameras
19
MJPEG Video
The system streams MJPEG video for users with connections under 100 Mbps, or cameras belonging to aggregated sites with connection
speeds less than 100 Mbps. MJPEG Video includes the following limitations:
PTZ controls for MJPEG streams are limited to click-to-center controls.
MJPEG streams may not synchronize perfectly when in sync playback mode; this is due to the nature of the MJPEG pull mechanism.
You can still use the sync playback controls to manipulate video, but synchronization may be noticeably affected.
When the Media Gateway CPU utilization reaches its threshold, it will reject requests to transcode new MJPEG streams.
NOTE: The Media Gateway transcodes MJPEG video, and returns the full I-Frame rate requested by the camera. Cameras configured to
deliver more than 2 I-frames per second will stress the performance of the Media Gateway and consume more bandwidth. It is recom-
mended that cameras expected to return MJPEG video are configured to deliver no more than 2 I-frames per second to prevent cameras
from taxing the Media Gateway.
Media Gateway Communication Method
You can set the communication method, unicast or multicast, from the Video Source to the Media Gateway, and from the Media Gateway
to the client. The media gateway is capable of transcasting multicast from the source to unicast for the client. The transcasting feature
will be updated in a future release to support unicast from the source to multicast for clients.
In general, when requesting video, the Media Gateway forwards multicast requests and proxies unicast requests.
Configuring Secondary Streams to Support VideoXpert
Because the system delivers secondary streams as a step-down adjustment to prevent Ops Center decoders from becoming overloaded,
you’ll want to configure secondary streams accordingly, with reduced resolution and framerate by comparison with the primary.
Secondary streams for cameras should be configured at 640 x 352 (or the corresponding 4:3 equivalent, depending on available aspect
ratios) at 5 frames per second, or lower.
20
Network Port Reference
The tables below correspond to VideoXpert software components; in some cases, the same server can host multiple components. The
ports listed must be open to ensure VideoXpert functions properly.
Core
TCP/UDP Port Service
TCP 80 HTTP, used for camera configuration as necessary
TCP 443 HTTPS
UDP 1900 SSDP discovery target on 239.255.255.250
TCP 5701 Hazelcast communications (default)
UDP 10000 Outbound SSDP traffic
TCP 27017 The MongoS service listens on port 27017 and is responsible for routing read/write requests for
the database.
TCP 31001 The MongoD service implements the cluster’s primary/secondary hierarchy and is responsible
for reading/writing data to/from the disk
TCP 31002-31004 The MongoDBConfig1, 2, and 3 servers store a small amount of metadata about shards for your
VideoXpert cluster and correspond to TCP ports 31002, 31003, and 31004 respectively. No
matter how many servers in your VideoXpert environment, there will always be three config
servers (or services, even if hosted on the same physical server).
N/A 224.2.2.4 The server joins this multicast group for Hazelcast multicast discovery.
UDP 54327 Hazelcast multicast discovery
  • Page 1 1
  • Page 2 2
  • Page 3 3
  • Page 4 4
  • Page 5 5
  • Page 6 6
  • Page 7 7
  • Page 8 8
  • Page 9 9
  • Page 10 10
  • Page 11 11
  • Page 12 12
  • Page 13 13
  • Page 14 14
  • Page 15 15
  • Page 16 16
  • Page 17 17
  • Page 18 18
  • Page 19 19
  • Page 20 20
  • Page 21 21
  • Page 22 22
  • Page 23 23
  • Page 24 24
  • Page 25 25
  • Page 26 26
  • Page 27 27
  • Page 28 28
  • Page 29 29
  • Page 30 30
  • Page 31 31
  • Page 32 32
  • Page 33 33
  • Page 34 34
  • Page 35 35
  • Page 36 36
  • Page 37 37
  • Page 38 38
  • Page 39 39
  • Page 40 40
  • Page 41 41
  • Page 42 42
  • Page 43 43
  • Page 44 44
  • Page 45 45
  • Page 46 46
  • Page 47 47
  • Page 48 48
  • Page 49 49
  • Page 50 50
  • Page 51 51
  • Page 52 52
  • Page 53 53

Pelco VideoXpert System User manual

Type
User manual

Ask a question and I''ll find the answer in the document

Finding information in a document is now easier with AI