FSM CALA Users Guide
User Manual:
Open the PDF directly: View PDF 
.
Page Count: 308
| Download | |
| Open PDF In Browser | View PDF |
FSM CALA User’s Guide FileNet System Monitor 4.0.0 FileNet Corporation Table of Contents 1. Copyright Notice ..................................................................................................................... 1 Trademarks........................................................................................................................... 1 Notice ................................................................................................................................... 1 2. Notices ..................................................................................................................................... 3 3. About this document .............................................................................................................. 4 Who Should Read This Guide .............................................................................................. 4 List of documents ................................................................................................................. 4 General information .............................................................................................................. 4 Where you find this guide ............................................................................................ 4 Typeface Conventions .................................................................................................. 4 Contacting FileNet Support.......................................................................................... 5 4. FSM CALA Overview............................................................................................................... 6 FSM CALA (CALA)............................................................................................................... 6 CALA Binaries.............................................................................................................. 6 CALA GUI .................................................................................................................... 6 5. Installation ............................................................................................................................... 7 General Installation Information............................................................................................ 7 Non-Tivoli CALA Installer...................................................................................................... 7 General description...................................................................................................... 7 Installing a product on the local machine..................................................................... 9 Remote installation .................................................................................................... 10 Further installation options......................................................................................... 13 Relationship between installer GUI install_cala.sh .................................................... 14 6. Component Architecture ...................................................................................................... 16 CALA System platforms ..................................................................................................... 16 Supported JAVA JRE or JDK versions (CALAGUI and CALA V2S Editor prerequisite) ..... 16 Implementation on Microsoft Windows based systems ...................................................... 16 CALA installation as Windows Service ...................................................................... 16 CALA de-installation on Windows systems................................................................ 16 Configuration file logctlsrv.conf for a Windows service installation..................................... 16 Client / Server Architecture................................................................................................. 17 Implemented components .................................................................................................. 18 Read-only component (Reader)................................................................................. 18 Filter component (Filter)............................................................................................. 19 Event generating components ................................................................................... 19 Processing server msgclsfsrv (Message Classification Server) ................................ 19 Sub components Rules Engine and Message Mapping (msgclsfsrv sub components) 20 Sub component Completer (msgclsfsrv sub component) .......................................... 20 Sub component Remapper (msgclsfsrv sub component) .......................................... 20 Emitter components ................................................................................................... 20 T/EC transmit component .......................................................................................... 21 Application proxy for DMZ.......................................................................................... 21 Control component logctlsrv ...................................................................................... 21 CLI logctlcmd ............................................................................................................. 22 Supported logctlcmd commands ............................................................................... 22 Generating test events ............................................................................................... 23 Possible component architecture (predecessors / successors).......................................... 23 ii FSM CALA User’s Guide Communication between CALA components ..................................................................... 24 Default tcp ports used by CALA components..................................................................... 25 Event caching ..................................................................................................................... 25 7. Configuration file logctlsrv.conf ...................................................................................... 27 Global configuration instructions applicable to all components .......................................... 27 Configuration instruction serverlist............................................................................. 27 run instruction ............................................................................................................ 28 target Instruction ........................................................................................................ 29 port instruction ........................................................................................................... 29 Port list functionality ................................................................................................... 30 conf instruction........................................................................................................... 30 ip instruction............................................................................................................... 31 Serverlist functionality ................................................................................................ 31 Broadcast functionality............................................................................................... 31 8. Configuration GUI ................................................................................................................. 33 Using the CALA Configuration GUI .................................................................................... 33 Starting the GUI ......................................................................................................... 33 Setting up a new configuration (Create ...) ................................................................ 34 Opening an existing configuration (Open ...).............................................................. 35 Saving a created or changed configuration (Save, Save as) ..................................... 35 Exporting parts of the configuration for use with the CALA configurator ................... 36 Differences between CALAGUI configurations and CALA Configurator .................... 38 Altering the global configuration settings ................................................................... 38 Configuration instructions logctlsrv_port and logctlcmd_port............................ 39 Configuration instruction cala_srv_port (Windows systems only) ..................... 40 The configuration instructions logctlsrv_adapters and logctlcmd_adapters...... 40 Maintenance instruction .................................................................................... 41 More global settings................................................................................................... 42 Configuration check ................................................................................................... 42 component configuration............................................................................................ 43 9. Component-specific configuration...................................................................................... 45 Common settings................................................................................................................ 45 Display version information ........................................................................................ 48 ascfileread .......................................................................................................................... 50 ascfileread specific parameters and their setting in the configuration file .................. 50 ascfileread command line parameters ....................................................................... 54 ntevtlogread ........................................................................................................................ 55 ntevtlogread specific parameters and their setting in the configuration file................ 55 ntevtlogread command line parameters..................................................................... 58 tecfmtfilt .............................................................................................................................. 59 tecfmtfilt specific parameters and their setting in the configuration file ...................... 59 tecfmtfilt command line parameters ........................................................................... 61 v2fmtfilt ............................................................................................................................... 62 v2fmtfilt specific parameters and their setting in the configuration file ....................... 62 v2fmtfilt command line parameters ............................................................................ 63 calamon .............................................................................................................................. 65 calamon specific parameters and their setting in the configuration file...................... 68 calamon command line parameters........................................................................... 69 Structure of FIRs created by calamon ....................................................................... 69 snmpread............................................................................................................................ 71 snmpread specific parameters and their setting in the configuration file ................... 71 iii FSM CALA User’s Guide Snmpread command line parameters........................................................................ 73 Snmpread generated Events ..................................................................................... 73 mssqlread and oracleread .................................................................................................. 75 mssqlread/oracleread specific parameters and their setting in the configuration file. 75 mssqlread and oracleread command line parameters ............................................... 81 jdbcread.............................................................................................................................. 82 jdbcread specific parameters and their setting in the configuration file ..................... 82 msgclsfsrv........................................................................................................................... 84 Definition MessageMap File....................................................................................... 84 Definition RulesMap File ............................................................................................ 84 The basic msgclsfsrv window .................................................................................... 85 The Message Map Types window .............................................................................. 85 Definition of MessageMap Classification Type (MCT) ............................................... 86 MCT parameters and their setting in the configuration file................................ 86 MCT configuration parameters ................................................................. 87 The Message Map definition window......................................................................... 91 Default Mapping ................................................................................................ 92 Deleting slots..................................................................................................... 92 Special slots for duplicate detection .................................................................. 92 Another example for a complete message map definition................................. 93 Operations on FIR fields per Message Maps .................................................... 95 The Rules definition window ...................................................................................... 95 Definition of Rules Map Type (RMT) .......................................................................... 96 RMT parameters and their setting in the configuration file................................ 96 RMT configuration line parameters........................................................... 97 The Rules maps window.......................................................................................... 100 Rules Map Parameters and their setting in the configuration file .................... 100 Definition Base Event............................................................................................... 101 Reserved fieldnames and their meaning ................................................................. 101 Condition values ...................................................................................................... 103 Rules Map Example................................................................................................. 103 Completer definition window .................................................................................... 106 Completer Parameters and their setting in the configuration file..................... 106 Remapper definition window.................................................................................... 108 Remapper parameters and their setting in the configuration file..................... 108 Auxkeys definition window ....................................................................................... 110 Auxkeys parameters and their setting in the configuration file ........................ 110 the msgclsfsrv flowlimiter ......................................................................................... 111 msgclsfsrv command line parameters ..................................................................... 115 calaproxy .......................................................................................................................... 116 calaproxy specific parameters and their setting in the configuration file .................. 116 calaproxy command line parameters ....................................................................... 117 tecfmtemit ......................................................................................................................... 118 tecfmtemit specific parameters and their setting in the configuration file................. 118 tecfmtemit command line parameters...................................................................... 119 tecifcsrv ............................................................................................................................ 120 tecifcsrv specific parameters and their setting in the configuration file .................... 120 tecifcsrv command line parameters ......................................................................... 121 cmdemit ............................................................................................................................ 123 cmdemit specific parameters and their setting in the configuration file.................... 123 cmdemit command line parameters......................................................................... 124 cmdemit input events ............................................................................................... 124 iv FSM CALA User’s Guide smtpemit ........................................................................................................................... 125 smtpemit specific parameters and their setting in the configuration file................... 125 smtpemit command line parameters........................................................................ 126 smtpemit input events .............................................................................................. 126 snmpemit .......................................................................................................................... 126 snmpemit specific parameters and their setting in the configuration file.................. 127 snmpemit command line parameters....................................................................... 127 snmpemit input events ............................................................................................. 128 SNMPv1 .......................................................................................................... 128 SNMPv2c ........................................................................................................ 129 SNMPv3 .......................................................................................................... 129 Hint for SNMPv2c and SNMPv3 users ............................................................ 129 mysqlemit ......................................................................................................................... 131 mysqlemit specific parameters and their setting in the configuration file ................. 131 mysqlemit command line parameters ...................................................................... 134 jdbcemit ............................................................................................................................ 135 jdbcemit specific parameters and their setting in the configuration file .................... 135 reportemit ......................................................................................................................... 137 reportemit specific parameters and their setting in the configuration file ................. 137 reportemit command line parameters ...................................................................... 141 javasrv .............................................................................................................................. 143 javasrv specific parameters and their setting in the configuration file ...................... 143 javasrv command line parameters ........................................................................... 143 javasrv/pchread........................................................................................................ 144 remote component............................................................................................................ 145 remote component specific parameters and their setting in the configuration file ... 145 10. Security .............................................................................................................................. 147 Encrypted Communication ............................................................................................... 147 The one-time-pad encryption algorithm ................................................................... 147 Configuring encryption............................................................................................. 148 The crypttool..................................................................................................................... 150 Supervision of connections............................................................................................... 151 Encryption error events ............................................................................................ 152 Connection accepted event...................................................................................... 152 Accept timeout events.............................................................................................. 152 Connection lost events............................................................................................. 152 CALA communication over firewalls ................................................................................. 152 CALA communication over DMZ ...................................................................................... 153 Revert connections: Servers connecting to clients ........................................................... 155 clients waiting for servers to connect ....................................................................... 155 servers connecting to clients.................................................................................... 156 A sample client/server configuration using demand clients ..................................... 157 A. The v2 format ...................................................................................................................... 159 Storage form ..................................................................................................................... 159 Identifiers .......................................................................................................................... 159 General design of the v2 format ....................................................................................... 159 Comments................................................................................................................ 159 Header ..................................................................................................................... 160 Global Variables....................................................................................................... 160 Automatically assigned variables ............................................................................. 160 Variables to set timestamp....................................................................................... 161 v FSM CALA User’s Guide Classes and sub-expressions........................................................................................... 161 Classes .................................................................................................................... 161 Sub-expressions ...................................................................................................... 162 Expressions ...................................................................................................................... 162 Matching types......................................................................................................... 163 Character Match (individual characters).......................................................... 163 Character Match (individual characters by ASCII code).................................. 163 Multi match (multiple match) ........................................................................... 163 Constant string match ..................................................................................... 165 Subexpression match...................................................................................... 165 Mandatory, optional and repetitive expressions................................................................ 165 Mandatory expression.............................................................................................. 166 Optional expression ................................................................................................. 166 Optional repetitive expression.................................................................................. 166 Group binding ................................................................................................................... 167 Example of format file sna.v2s ......................................................................................... 168 B. The command table file format.......................................................................................... 170 C. msgclsfsrv Text Formatting............................................................................................... 172 Some examples how text formatting works ...................................................................... 173 D. Pchread XML Configuration .............................................................................................. 174 Properties ......................................................................................................................... 174 Request for historic data .......................................................................................... 175 Clusters, Hosts and Applications ...................................................................................... 175 Events............................................................................................................................... 176 Conditions ................................................................................................................ 176 Actions ..................................................................................................................... 177 E. CALA created events.......................................................................................................... 179 CALA Testevent ................................................................................................................ 179 Connection Accepted Event ............................................................................................. 179 Connection Lost Event ..................................................................................................... 180 Accept Timeout Event....................................................................................................... 180 Encryption Error Event ..................................................................................................... 181 Heartbeat Event................................................................................................................ 181 Status Events (Startup/Shutdown) ................................................................................... 182 F. Additional tools ................................................................................................................... 183 install_cala.sh ................................................................................................................... 183 General description.................................................................................................. 183 Parameters............................................................................................................... 183 Installation process .................................................................................................. 185 cala_untar.sh .................................................................................................................... 185 General description.................................................................................................. 185 Parameters............................................................................................................... 185 Examples ................................................................................................................. 186 brdcsttool .......................................................................................................................... 186 General description.................................................................................................. 186 Parameters............................................................................................................... 186 Examples ................................................................................................................. 187 testv2sfile and testfmtfile .................................................................................................. 187 General description.................................................................................................. 187 Parameters............................................................................................................... 187 Examples ................................................................................................................. 188 vi FSM CALA User’s Guide sendfir............................................................................................................................... 188 General description.................................................................................................. 188 Parameters............................................................................................................... 189 Examples ................................................................................................................. 189 d_v2fmtfilt and d_tecfmtfilt................................................................................................ 189 G. CALA Configurator............................................................................................................. 191 CALA Configurator Basics................................................................................................ 191 Supported components............................................................................................ 191 Standard architectures............................................................................................. 191 Restrictions .............................................................................................................. 191 General............................................................................................................ 191 ascfileread/ntevtlogread and tecfmtfilt/v2fmtfilt................................................ 192 calamon........................................................................................................... 192 TEC interface .................................................................................................. 192 Templates................................................................................................................. 193 Creating your own templates........................................................................... 193 Directory structure in the export directory of CALAGUI ........................................... 193 Directory structure on each Tivoli server ................................................................. 194 Synchronizing the Configurator repository............................................................... 194 Step 1: Synchronizing CALAGUI and TMR server.......................................... 194 Step 2: Synchronizing TMR server and Gateways.......................................... 195 Where to put files referenced from within a configuration ........................................ 195 Directory structure on client ..................................................................................... 195 Starting the Configurator.......................................................................................... 196 Input files (.cala files) ............................................................................................... 196 General parameters ........................................................................................ 196.cala .................................................................................................. 196 ascfileread .............................................................................................. 197 ntevtlogread ............................................................................................ 197 snmpread................................................................................................ 197 mssqlread, oracleread ............................................................................ 197 reportemit (datatype specific definitions) ................................................ 198 msgclsfsrv............................................................................................... 198 cmdemit, mysqlemit, reportemit, smtpemit, snmpemit, tecfmtemit, remote_component ........................................................................ 199 aux_*.cala........................................................................................................ 199 completer_*.cala.............................................................................................. 200 remapper_*.cala .............................................................................................. 200 calamon_*.cala................................................................................................ 201 javasrv_ .cala .......................................................................... 201 report_*.cala.................................................................................................... 201 tec_*.cala......................................................................................................... 201 remote_*.cala .................................................................................................. 202 Referenced files ....................................................................................................... 202 Naming convention.......................................................................................... 202 Standard location ............................................................................................ 203 Details............................................................................................................................... 203 Detailed description of configuration........................................................................ 203 ascfileread ....................................................................................................... 203 ntevtlogread..................................................................................................... 203 tecfmtfilt / v2fmtfilt............................................................................................ 204 calamon........................................................................................................... 204 vii FSM CALA User’s Guide javasrv ............................................................................................................. 205 snmpread ........................................................................................................ 205 mssqlread / oracleread.................................................................................... 206 db_log_types ................................................................................................... 206 msgclsfsrv ....................................................................................................... 207 MCT................................................................................................................. 207 MessageMap entry.......................................................................................... 208 RMT................................................................................................................. 208 RulesMap entry ............................................................................................... 209 Auxkey entry.................................................................................................... 209 Completer entry............................................................................................... 210 Remapper entry .............................................................................................. 210 tecfmtemit........................................................................................................ 210 tecifsrv............................................................................................................. 210 reportemit ........................................................................................................ 211 remote components......................................................................................... 211 Example for .cala files, templates and the resulting configuration ........................... 211 fndw4log.cala - Definition for secondary datatype fndw4log ........................ 212 remapper_fnislog.cala - Definition for remapper ........................................ 212 tec_panagon.cala - Definition for tecifcsrv.................................................... 212 remote_panagon.cala - Definition for remote component ............................. 212 Template.......................................................................................................... 212 Resulting configuration file .............................................................................. 213 Configured components .................................................................................. 214 Configuration details of msgclsfsrv ............................................................... 214 H. A complete logctlsrv.conf .................................................................................................. 215 I. Detailed description of the status report ........................................................................... 218 configuration status .......................................................................................................... 218 environment ...................................................................................................................... 218 log control server queues ................................................................................................. 220 component status general properties ............................................................................... 220 target status...................................................................................................................... 222 client status....................................................................................................................... 223 ascfileread and ntevtlogread............................................................................................. 224 tecfmtfilt and v2fmtfilt........................................................................................................ 224 oracleread and mssqlread ................................................................................................ 225 mysqlemit ......................................................................................................................... 225 reportemit ......................................................................................................................... 226 How to detect configuration errors using the status output............................................... 226 J. Supported character sets................................................................................................... 227 List of supported character sets ....................................................................................... 227 K. Licenses .............................................................................................................................. 229 Overview........................................................................................................................... 229 The Apache Software License.......................................................................................... 231 The PHP License.............................................................................................................. 232 MySQL Commercial License ............................................................................................ 234 Non-Profits, Academic Institutions, and Private Individuals ..................................... 234 Recommendations ................................................................................................... 235 FOSS Exception ...................................................................................................... 235 Older Versions ......................................................................................................... 235 When in Doubt ......................................................................................................... 235 viii FSM CALA User’s Guide Cygwin API Licensing Terms ............................................................................................ 236 Mozilla Public License 1.1 (MPL 1.1) ............................................................................... 237 The Artistic License .......................................................................................................... 245 Sun Microsystems and Java Licenses.............................................................................. 248 JavaTM 2, Standard Edition (J2SETM) Specification (Specification)........................... 248 Sun Microsystems, Inc. Binary Code License Agreement ....................................... 250 BORLAND JBUILDER PROFESSIONAL VERSION 5..................................................... 253 SAX LICENSE .................................................................................................................. 264 Copyright Status ...................................................................................................... 264 No Warranty ............................................................................................................. 264 Copyright Disclaimers .............................................................................................. 264 W3C SOFTWARE NOTICE AND LICENSE ..................................................................... 265 The GNU Public License .................................................................................................. 266 The GNU Lesser General Public License......................................................................... 272 The MIT License............................................................................................................... 281 RSA Security Releases RSA Encryption Algorithm into Public Domain........................... 282 Net-SNMP License ........................................................................................................... 283 OpenSSL License............................................................................................................. 287 CookSwing License .......................................................................................................... 290 The java tar public domain license ................................................................................... 291 The MX4J License............................................................................................................ 292 ix FSM CALA User’s Guide List of Tables 5-1. Relationship between GUI parameters and command line options of install_cala.sh.......... 14 List of Figures 7-1. Format of the serverlist instruction....................................................................................... 27 7-2. Format of run instruction ...................................................................................................... 28 7-3. Format of targets instruction ................................................................................................ 29 7-4. Format of port instruction ..................................................................................................... 29 7-5. Format of conf instruction .................................................................................................... 30 7-6. Format of ip instruction ........................................................................................................ 31 8-1. Format of logctlsrv_port and logctlcmd_port instruction ...................................................... 40 8-2. Format of cala_srv_port instruction ..................................................................................... 40 8-3. Format of logctlsrv_adapters and logctlcmd_adapters instructions ..................................... 41 8-4. Format of maintenance instruction....................................................................................... 41 9-1. The port list configuration entry has the following format:.................................................... 46 9-2. Format of argument line containing port assignment........................................................... 46 9-3. Format of run statement....................................................................................................... 46 9-4. Format of run statement containing debug arguments ........................................................ 47 9-5. Displaying version information of snmpread ......................................................................... 49 9-6. An example configuration line for ascfileread....................................................................... 50 9-7. Format of pathlist instruction................................................................................................ 51 9-8. Format of ptrnlist instruction................................................................................................. 52 9-9. Format of assoc instruction.................................................................................................. 53 9-10. Format of evtlog instruction................................................................................................ 56 9-11. Format of spacereplacement instruction............................................................................ 56 9-12. Format of skip_old instruction ............................................................................................ 56 9-13. Format of prefilt_in and prefile_out instructions ................................................................. 57 9-14. Format of assoc instruction................................................................................................ 58 9-15. Format of formatlist instruction........................................................................................... 60 9-16. Format of formatlist instruction........................................................................................... 63 9-17. Format of cmdtab instruction ............................................................................................. 68 9-18. Format of type instruction ................................................................................................. 72 9-19. Format of class instruction ................................................................................................. 72 9-20. Format of prefilt_in and prefilt_out instructions .................................................................. 72 9-21. Some example configuration lines for mssqlread (identical for oracleread).................... 75 9-22. Format of db_log_types instruction.................................................................................... 76 9-23. Format of dbuser instruction .............................................................................................. 76 9-24. Format of database instruction .......................................................................................... 77 9-25. Format of table instruction ................................................................................................. 77 9-26. SQL statement used to find events .................................................................................... 77 9-27. Format of db_entry_id instruction ...................................................................................... 77 9-28. Format of map instruction .................................................................................................. 78 9-29. Format of copy_unmapped instrucion................................................................................ 78 9-30. Format of defaultclass instruction ...................................................................................... 79 9-31. Format of classmap instruction .......................................................................................... 79 9-32. Format of type instruction .................................................................................................. 79 9-33. Format of prefilt_in and prefilt_out instructions .................................................................. 80 9-34. Format of timestamp instruction......................................................................................... 80 9-35. Format of types instruction................................................................................................ 87 x FSM CALA User’s Guide 9-36. Format of mct configuration line......................................................................................... 87 9-37. Format of mct type instruction............................................................................................ 88 9-38. Format of mct handledby instruction ................................................................................. 88 9-39. Format of mct msgmaps instruction................................................................................... 88 9-40. Format of mct eventframe instruction................................................................................. 89 9-41. Format of mct dupekey instruction ..................................................................................... 89 9-42. Format of message map definition..................................................................................... 91 9-43. Excerpt from the related message map file ....................................................................... 93 9-44. Format of rmt rules instruction ........................................................................................... 97 9-45. Format of rmt configuration line ......................................................................................... 97 9-46. Format of rmt for instruction............................................................................................... 97 9-47. Format of rmt type instruction ............................................................................................ 98 9-48. Format of rmt rulesmap instruction .................................................................................... 98 9-49. Format of rules map definition ......................................................................................... 100 9-50. Format of completers instruction...................................................................................... 106 9-51. Format of completers configuration line ........................................................................... 107 9-52. Format of remappers instruction ...................................................................................... 108 9-53. Format of remapper configuration line ............................................................................. 109 9-54. Format of auxkeys instruction .......................................................................................... 111 9-55. Format of auxkeys configuration line ............................................................................... 111 9-56. Format of flowlimiter instruction ....................................................................................... 112 9-57. Format of eventquota instruction.................................................................................... 113 9-58. Format of eventperiod instruction.................................................................................. 113 9-59. Format of unblock instruction.......................................................................................... 114 9-60. Format of blockedevent and unblockedevent instructions .......................................... 114 9-61. Event definition................................................................................................................. 114 9-62. Format of logfile instruction.......................................................................................... 115 9-63. Format of databaseinstruction ............................................................... 131 9-64. Format of the dbuser instruction ...................................................................................... 131 9-65. Format of tableconf instruction......................................................................................... 132 9-66. Format of db_status instruction...................................................................................... 132 9-67. Format:............................................................................................................................. 133 9-68. Format of dest_file instruction.......................................................................................... 138 9-69. Format of report_slots instruction .................................................................................... 139 9-70. Format of critical_slot instruction ..................................................................................... 139 9-71. Format of critical_slots instruction.................................................................................... 140 9-72. Format of report_file instruction ....................................................................................... 140 9-73. Format:............................................................................................................................. 143 9-74. Format of javasrv commandline..................................................................................... 143 10-1. The one-time-pad communication schema ...................................................................... 147 10-2. The crypttool usage screen.............................................................................................. 150 10-3. CALA sending events over a firewall................................................................................ 152 10-4. CALA sending over a DMZ .............................................................................................. 153 10-5. Format of demand_targets instruction ............................................................................. 155 10-6. Format of demand_clients instruction .............................................................................. 156 A-1. Format of v2s spec expression.......................................................................................... 160 A-2. Format of v2s global bind expression ................................................................................ 160 A-3. Format of v2s class expression ......................................................................................... 161 A-4. Format of v2s subexpression definition ............................................................................. 162 A-5. Format of v2s subexpression call ...................................................................................... 162 A-6. Format of v2s constant string match ................................................................................. 165 A-7. Format of v2s constant string match with alternatives....................................................... 165 xi FSM CALA User’s Guide A-8. Format of v2s subexpression match.................................................................................. 165 D-1. the xml file header ............................................................................................................. 174 D-2. the xml file footer ............................................................................................................... 174 D-3. Format of event path ......................................................................................................... 176 F-1. Usage of brdcsttool............................................................................................................ 186 F-2. Usage of testv2sfile and testfmtfile .................................................................................... 187 F-3. Usage of sendif.................................................................................................................. 189 List of Examples 5-1. Example for using Connected drive/dir on MS Windows ..................................................... 11 5-2. Example for using Connected drive/dir on Unix ................................................................... 12 7-1. Example using serverlist ...................................................................................................... 27 7-2. Example using the run statement ........................................................................................ 28 7-3. Example targets usage ........................................................................................................ 29 7-4. Example targets usage with outgoing port........................................................................... 29 7-5. Example port usage ............................................................................................................. 30 7-6. Example conf usage ............................................................................................................ 30 7-7. Example ip usage ................................................................................................................ 31 7-8. Example ip instruction using the serverlist functionality....................................................... 31 8-1. Global settings in the logctlsrv.conf file.......................................................................... 39 8-2. Example for logctlsrv_port and logctlcmd_port usage ......................................................... 40 8-3. Example for cala_srv_port usage ........................................................................................ 40 8-4. Example for logctlsrv_adapters and logctlcmd_adapters usage.......................................... 41 8-5. Example for logctlsrv_adapters and logctlcmd_adapters usage.......................................... 41 8-6. Example for additional settings in logctlsrv.conf ............................................................ 42 9-1. Example configuration of logical server name ..................................................................... 46 9-2. Example portlist ................................................................................................................... 46 9-3. Example run statement containing port assignment ............................................................ 46 9-4. Example run statement ........................................................................................................ 47 9-5. Example of run statement containing debug arguments (not from the window above): ...... 47 9-6. Example pathlist instruction ................................................................................................. 51 9-7. Example ptrnlist instruction .................................................................................................. 52 9-8. Example assoc instruction ................................................................................................... 53 9-9. Files that would match with above pathlist, pattrnlist and assoc configuration .................... 53 9-10. An example configuration line for ntevtlogread............................................................... 55 9-11. Example evtlog instruction ................................................................................................. 56 9-12. Example spacereplacement instruction ............................................................................. 56 9-13. Example skip_old instruction ............................................................................................. 56 9-14. Example prefilt_in and prefilt_out instructions ................................................................... 57 9-15. Example for a pre-filter file to match all events from the SNMP and Print source:............. 57 9-16. Example assoc instruction ................................................................................................. 58 9-17. Another example for assoc instruction using different primary types................................. 58 9-18. An example configuration line for tecfmtfilt .................................................................. 59 9-19. Example formatlist instruction ............................................................................................ 60 9-20. An example configuration line for v2fmtfilt .................................................................... 63 9-21. Example formatlist instruction ............................................................................................ 63 9-22. An example message template.......................................................................................... 67 9-23. An example configuration line for calamon ........................................................................ 68 9-24. Example cmdtab instruction............................................................................................... 68 9-25. An example configuration line for snmpread ...................................................................... 71 xii FSM CALA User’s Guide 9-26. Example type instruction................................................................................................... 72 9-27. Example class instruction .................................................................................................. 72 9-28. Example prefilt_in and prefilt_out instructions ................................................................... 73 9-29. Example db_log_types instrcution ..................................................................................... 76 9-30. Example dbuser instruction................................................................................................ 76 9-31. Example database instruction............................................................................................ 77 9-32. Example table instruction ................................................................................................... 77 9-33. Example db_entry_id instruction........................................................................................ 78 9-34. Example map instruction.................................................................................................... 78 9-35. Example copy_unmapped instrucion ................................................................................. 78 9-36. Example defaultclass instruction........................................................................................ 79 9-37. Example classmap instrucion ............................................................................................ 79 9-38. An example mapfile mapping the class of the created FIR to APP_Logon if the map field’s value is LOGON:.................................................................................................................... 79 9-39. Example type instruction.................................................................................................... 80 9-40. Example prefilt_in and prefilt_out instructions ................................................................... 80 9-41. Examples for the timestamp instruction ............................................................................. 81 9-42. An example database string for jdbcread ......................................................................... 82 9-43. An example jdbread configuration using a mysql connector for connection to the local database fndsdb ................................................................................................................ 82 9-44. These are the configuration lines created for the message map classification type: ......... 86 9-45. Example types instruction ................................................................................................. 87 9-46. Example mct configurationline ........................................................................................... 87 9-47. Example mct type instruction ............................................................................................. 88 9-48. Example mct handledby instruction ................................................................................... 88 9-49. Example mct msgmaps instruction .................................................................................... 88 9-50. Example mct eventframe instruction .................................................................................. 89 9-51. Example dupekey instruction ............................................................................................. 89 9-52. Another example of a complete MCT definition ................................................................. 90 9-53. Example message map definition ...................................................................................... 91 9-54. An example mapfile for the above map definition .............................................................. 92 9-55. Another example for a complete message map definition ................................................. 93 9-56. An example msgclsfsrv configuration with a rules map type definition .............................. 96 9-57. Example Format of rmt rules instruction ............................................................................ 97 9-58. Example rmt configuration line........................................................................................... 97 9-59. Example rmt for instruction ................................................................................................ 98 9-60. Example of rmt type instruction.......................................................................................... 98 9-61. Example rmt rulesmap instruction ..................................................................................... 98 9-62. Example of rmt corrkey instruction..................................................................................... 99 9-63. Another example of a complete RMT definition ................................................................. 99 9-64. Example rules map definition........................................................................................... 100 9-65. Example of rules engine usage with base events ............................................................ 101 9-66. Some example conditions ................................................................................................ 103 9-67. Example rules map definition........................................................................................... 104 9-68. Example rules map file..................................................................................................... 104 9-69. An example msgclsfsrv configuration using a completer ................................................. 106 9-70. Example completers instruction ....................................................................................... 107 9-71. Example completers configuration line ............................................................................ 107 9-72. Another completer example ............................................................................................. 107 9-73. An example msgclsfsrv configuration using a remapper.................................................. 108 9-74. Example remappers instruction ....................................................................................... 108 9-75. Example remapper configuration line .............................................................................. 109 xiii FSM CALA User’s Guide 9-76. Another example for a remapper configuration line ......................................................... 109 9-77. An example message map using auxkeys ....................................................................... 110 9-78. An example msgclsfsrv configuration using auxkeys....................................................... 110 9-79. Example auxkeys instruction............................................................................................ 111 9-80. Examples auxkeys configuration line ............................................................................... 111 9-81. An example msgclsfsrv configuration using auxkeys....................................................... 111 9-82. Example flowlimiter instruction......................................................................................... 112 9-83. Format of the flowlimiter configuration line....................................................................... 112 9-84. An example flowlimiter configuration line ......................................................................... 112 9-85. Examples eventquota configuration line......................................................................... 113 9-86. Examples eventperiod configuration line....................................................................... 113 9-87. Examples unblock configuration line .............................................................................. 114 9-88. Examples blockedevent and unblockedevent configurations ...................................... 114 9-89. Examples logfile configuration line .............................................................................. 115 9-90. An example configuration line for calaproxy..................................................................... 116 9-91. An example configuration line for tecfmtemit ................................................................ 118 9-92. An example configuration line for tecifcsrc .................................................................. 120 9-93. An example configuration line for cmdemit ...................................................................... 123 9-94. An example configration line for smtpemit ...................................................................... 125 9-95. An example configuration line for snmpemit .................................................................... 127 9-96. Setting sysUpTime to current time ................................................................................... 129 9-97. Setting the snmpTrapOID variable ................................................................................... 130 9-98. An example mysqlemit configuration line........................................................................ 131 9-99. Exampledatabaseinstruction................................................................. 131 9-100. Example the dbuser instruction...................................................................................... 131 9-101. Example tableconfinstruction ......................................................................................... 132 9-102. Example db_status instruction ..................................................................................... 132 9-103. Example: ........................................................................................................................ 134 9-104. An example database string for jdbcread ..................................................................... 135 9-105. An example jdbread configuration using a mysql connector for connection to the local database fndsdb .............................................................................................................. 135 9-106. An example configuration line for reportemit .............................................................. 137 9-107. Example dest_file instruction ......................................................................................... 138 9-108. Example dest_file instruction using % expressions......................................................... 138 9-109. Example report_slots instruction.................................................................................... 139 9-110. Example critical_slot instruction..................................................................................... 139 9-111. Example critical_slots instruction ................................................................................... 140 9-112. Example report_file instruction ...................................................................................... 140 9-113. A sample report template:.............................................................................................. 141 9-114. An example result for the above template...................................................................... 141 9-115. An example configuration line for javasrv .................................................................... 143 9-116. Example: ........................................................................................................................ 143 9-117. An example configuration line for a remote component ................................................. 145 10-1. Example ip chains rules ................................................................................................... 154 10-2. Example demand_targets instruction ............................................................................. 156 10-3. Example demand_clients instruction ............................................................................... 157 10-4. A client configuration using demand targets .................................................................... 157 10-5. A server configuration: using demand clients .................................................................. 157 A-1. Example of comments in v2s............................................................................................. 159 A-2. Example of a v2s spec expression .................................................................................... 160 A-3. Example of a v2s global bind expression .......................................................................... 160 A-4. Example of a v2s class expression.................................................................................... 161 xiv FSM CALA User’s Guide A-5. Example of a v2s subexpression definition........................................................................ 162 A-6. Example of a v2s subexpression call................................................................................. 162 A-7. Example v2s character match: matching the letter A ......................................................... 163 A-8. Example v2s: matching a sequence of alphanumeric chars ............................................. 163 A-9. Example for a mandatory v2 expression group ................................................................. 166 A-10. Example for an optional v2 expression group.................................................................. 166 A-11. Example for an optional-repetitive v2 expression group .................................................. 166 A-12. Example for a v2s group bind expression........................................................................ 167 A-13. An example v2s format file .............................................................................................. 168 C-1. Using the first 5 characters of the user field to process a Message Map definition........... 172 C-2. An example message map for the above definition........................................................... 172 D-1. An example properties configuration................................................................................. 175 D-2. An example event tag ........................................................................................................ 176 D-3. Example of a valid event rule definition ............................................................................. 178 E-1. Creating a test event for ascfileread .................................................................................. 179 F-1. Example: calling testfmtfile ................................................................................................ 188 F-2. An example configuration line for d_v2fmtfilt ..................................................................... 190 H-1. a complete logctlsrv.conf ............................................................................................. 215 I-1. An example status output.................................................................................................... 218 I-2. An example status output of environment settings ............................................................. 219 I-3. An example status output of queue entries......................................................................... 220 I-4. An example process status output ...................................................................................... 222 I-5. An example target status output ......................................................................................... 223 I-6. An example encryptionlevel output ..................................................................................... 223 I-7. An example stream status output........................................................................................ 224 I-8. An example formatfile status output.................................................................................... 225 I-9. An example database connection status output (reader).................................................... 225 I-10. An example database connection status output (emitter) ................................................. 226 I-11. An example reportemit status output ................................................................................ 226 xv FSM CALA User’s Guide List of Screenshots Installer login screen ..................................................................................................................... 7 The installers’ main window .......................................................................................................... 7 Selecting the product in the installer main window........................................................................ 8 The installers main window for installation of calamoma .............................................................. 8 The installation confirmation dialog............................................................................................... 9 The file transfer progress window ............................................................................................... 10 The installation result window ..................................................................................................... 10 The installer remote settings panel ............................................................................................. 11 Choosing the file transfer method ............................................................................................... 12 Choosing the remote execution methd........................................................................................ 12 Setting MS Windows ftp server to UNIX mode............................................................................ 13 CALA installer further options ..................................................................................................... 13 CALA installer informational messagebox for completing installation ......................................... 14 The CALAGUI Configuration menu .............................................................................................. ?? The CALAGUI new configuration dialog...................................................................................... 34 The default HP-UX client configuration. ...................................................................................... 34 The open configuration dialog ..................................................................................................... 35 The save configuration file dialog................................................................................................ 36 The options dialog when saving a configuration ......................................................................... 36 The .cala files export dialog ........................................................................................................ 36 The Check secondary data types dialog ..................................................................................... 37 The Global configuration parameters dialog ............................................................................... 38 Theresult window of Check loaded configuration........................................................................ 42 The components context menu ................................................................................................... 43 The Settings of connection to Source dialog............................................................................... 43 The Settings of connection to Target dialog ................................................................................ 43 A sample settings window for a CALA component...................................................................... 45 The Settings of ascfileread dialog ............................................................................................... 50 The Settings of ntevtlogread dialog............................................................................................. 55 The Settings of tecfmtfilt dialog ................................................................................................... 59 The Settings of v2fmtfilt dialog .................................................................................................... 62 The Settings of calamon dialog................................................................................................... 65 The Settings of snmpread dialog ................................................................................................ 71 The Settings of mssqlread dialog................................................................................................ 75 The Settings of msgclsfsrv dialog ............................................................................................... 85 The MessageMap Types dialog .................................................................................................. 85 The message map definition dialog ............................................................................................ 91 The rules definition dialog ........................................................................................................... 96 The rules maps dialog............................................................................................................... 100 The completer definition dialog ................................................................................................. 106 The remapper dialog ................................................................................................................. 108 The auxkeys definition dialog .................................................................................................... 110 The tecfmtemit settings dialog .................................................................................................. 118 The tecifcsrv settings dialog...................................................................................................... 120 The cmdemit settings dialog ..................................................................................................... 123 The settings of smtpemitdialog ................................................................................................. 125 The settings of snmpemit dialog ............................................................................................... 126 The settings of reportemit dialog............................................................................................... 137 The settings of remote_component dialog ................................................................................ 145 xvi FSM CALA User’s Guide Chapter 1. Copyright Notice FileNet System Monitor (June, 2007) Copyright © 2000-2007 by CENIT AG Systemhaus, Germany, including this documentation and all software. All rights reserved. May only be used pursuant to a CENIT AG Systemhaus Software License Agreement. No part of this publication maybe reproduced, transmitted, transcribed, stored in a retrieval system, or translated into any computer language, in any form or by any means, electronic, mechanical, magnetic, optical, chemical, manual, or otherwise, without prior written permission of CENIT AG Systemhaus. CENIT AG Systemhaus grants you limited permission to make hardcopy or other reproductions of any machine-readable documentation for your own use, provided that each such reproduction shall carry the CENIT AG Systemhaus copyright notice. No other rights under copyright are granted without prior written permission of CENIT AG Systemhaus. The document is not intended for production and is furnished as is without warranty of any kind. All warranties on this document are hereby disclaimed including the warranties of merchantability and fitness for a particular purpose. Note to U.S. Government Users Documentation related to restricted rights Use, duplication or disclosure is subject to restrictions set forth in GSA Trademarks The following product names are trademarks of Tivoli Systems or IBM Corporation: AIX, IBM, OS/2, RS/6000, Tivoli Management Environment, TME 10, Tivoli, Tivoli Enterprise Console (T/EC). Microsoft, Windows, Windows NT, Windows 95 and the Windows logo are trademarks or registered trademarks of Microsoft Corporation. UNIX is a registered trademark in the United States and other countries licensed exclusively through X/Open Company Limited. Hewlett Packard, HP, and HP-UX are trademarks or registered trademarks of Hewlett Packard Corporation. Other company, product, and service names mentioned in this document may be trademarks or servicemarks of others. Notice References in this publication to Tivoli Systems or IBM products, programs, or services do not imply that they will be available in all countries in which Tivoli Systems or IBM operates. Any reference to these products, programs, or services is not intended to imply that only Tivoli Systems or IBM products, programs, or services can be used. Subject to Tivoli Systems or IBM’s valid intellectual property or other legally protectable right, any functionally equivalent product, program, or service can be used instead of the referenced product, program, or service. The evaluation and verification of operation in conjunction with other products, except those expressly designated by Tivoli Systems or IBM, are the responsibility of the user. 1 Chapter 1. Copyright Notice CENIT AG Systemhaus may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to the CENIT AG Systemhaus, Product Marketing Tivoli Plus Modules, Industriestr. 52-54, 70565 Stuttgart, Germany 2 Chapter 1. Copyright Notice Chapter 2. Notices This document contains information proprietary to FileNet Corporation (FileNet). Due to continuing product development, product specifications and capabilities are subject to change without notice. You may not disclose or use any proprietary information or reproduce or transmit any part of this document in any form or by any means, electronic or mechanical, for any purpose, without written permission from FileNet. FileNet has made every effort to keep the information in this document current and accurate as of the date of publication or revision. However, FileNet does not guarantee or imply that this document is error free or accurate with regard to any particular specification. In no event will FileNet be liable for direct, indirect, special incidental, or consequential damages resulting from any defect in the documentation, even if advised of the possibility of such damages. No FileNet agent, dealer, or employee is authorized to make any modification, extension, or addition to the above statements. FileNet may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Furnishing this document does not provide any license to these patents, trademarks, copyrights, or other intellectual property. Please take a few moments to read the End User License Agreement on the FileNet System Monitor 4.0.0 documentation CD. By installing the FileNet System Monitor 4.0.0 software, the customer agrees to be bound by the terms of this agreement. FileNet System Monitor, copyright-protected by CENIT AG Systemhaus, is licensed and rebranded by FileNet Corporation. FileNet, ValueNet, Visual WorkFlo, and OSAR are registered trademarks of FileNet Corporation. Document Warehouse and UserNet are trademarks of FileNet Corporation. All other product and brand names are trademarks or registered trademarks of their respective companies. See the Centera License Agreement for copyright information pertaining to EMC Centera. Copyright © 1984, 2007 FileNet Corporation. All rights reserved. FileNet Corporation 3565 Harbor Boulevard Costa Mesa, California 92626 USA 800.FILENET (345.3638) Outside the U.S., call: 1.714.327.3400 www.filenet.com (http://www.filenet.com) 3 Chapter 2. Notices Chapter 3. About this document Who Should Read This Guide The target audience for this guide are system administrators who use the FSM CALA. Users of the guide should have some knowledge about Unix and/or Windows operating system and the FSM CALA. List of documents FileNet System Monitor CALA Guide Datatypes that can be processed by the FSM CALA FileNet System Monitor Monitoring Guide Description of all monitors contained in FileNet System Monitor FileNet System Monitor Task Guide Description of all tasks contained in FileNet System Monitor FileNet System Monitor Users Guide Installation guide FileNet System Monitor Release Notes Description of changes and bugfixes General information Where you find this guide You can find this documentation on the FSM installation CDROM in the following folder: UNIX:/INSTALL/docs Windows: :\INSTALL\docs Typeface Conventions The guide uses several typeface conventions for special terms and actions. These conventions have the following meaning: code Keywords and code examples occur like this varname Variable names occur like this filename File names occur like this constant Constants and names of tasks, monitors etc. appear like this 4 Chapter 3. About this document command Command names appear like this parameter Parameters and options for commands apperar like this userinput Values that th user must provide appear like this Computer output Output from programs appears like this guilabel Names of windows, dialogs, and other controls appear like this Programlistings appear like this: 001 002 ... 003 # a program listing echo "This is an example program listing (shell script) with nothing bu . t an extremly long echo command" exit 0 Note: The character . at the end of a line in a computer output or program listing shows, that the line has been wrapped and is continued in the next line. Contacting FileNet Support We are very interested in hearing from you about your experience with the product. We welcome your suggestions for improvements. If you encounter difficulties with the FSM please contact the FileNet support (http://www.filenet.com). 5 Chapter 3. About this document Chapter 4. FSM CALA Overview FSM CALA (CALA) CALA consists of the following components • CALA binaries The programs which implement the different CALA functions • CALA GUI The graphical User Interface to design, test and create CALA environments CALA Binaries • Receive (read) events from different event sources (active and passive event management, Logfile, Event log, Syslog, SNMP, active Monitoring) • Process events (filtering, manipulation, correlation, formatting) • Sends events to different destinations (T/EC, SNMP manager, SMPT email, report files, execution of commands) CALA GUI • Used to design a layout of CALA components (CALA architecture) • Design testing 6 Chapter 4. FSM CALA Overview Chapter 5. Installation General Installation Information This chapter describes the CALA installation process. Non-Tivoli CALA Installer This version of CALA supports the graphical installation of CALA on all supported platforms. The CALA Installer can be invoked by executing the script setup.sh on the FSM CDROM. On Windows platforms you can use the Batch program setup.bat to start CALA Installer. • For Windows JRE 1.4 is installed on the CD • Non-Windows users need JRE or JDK 1.4 to start the CALA installation GUI. • You need not set the environment variable JDK if the java binary can be found in your PATH. General description setup.sh or setup.bat starts the CALA installation GUI. CALA Installer is a Java GUI interface for the installation script install_cala.sh (see Annex for further information about install_cala.sh). If the CALA installer is started from the WebConsole in a full FSM environment, you must login to the cala_rex server before the installer window is shown. Installer login screen This is the installer main window: 7 Chapter 5. Installation The installers’ main window The CALA installation GUI is used to install CALA itself and its associated tools CALAGUI, V2SEdit and CalaMoMa. The product is selected from the product listbox. Depending on product selection, some of the GUI components are disabled. Selecting the product in the installer main window 8 Chapter 5. Installation The installers main window for installation of calamoma The configuration selection area and the CALA cache dir entry field are only enabled if cala is selected, they are not needed for the installation of any CALA tool. The JDK path is only editabled for the java tools calamoma, v2sedit and calagui. The source directory is shown for informational purpose only and is not editable in any case. Installing a product on the local machine To install a product on the local machine, the following steps have to be done: • select a product • customize target directory and possibly JDK path • CALA only: you may choose a default configuration from the configurations listbox and set its parameters in the settings dialog which appears after pressing the Set configuration variables button. You can copy and adjusrt an existing configuration by pressing the Copy configuration from ... button. • press the Install button This will show the installation confirmation dialog window. 9 Chapter 5. Installation The installation confirmation dialog If the installation has been confirmed, the installation files are transferred from the source (cd or ftp server) to the target directory, The file transfer progress window and the installation process is started. The installation progress is displayed in the results window. The button of this window is enabled only after the installation process has terminated (with or without success). The installation result window 10 Chapter 5. Installation Remote installation The CALA installer is enabled to install products on a remote machine if the following preconditions are fulfilled: • the target directory on the remote machine can be accessed via a local mounted drive or directory, ftp or cala_rex . • the target machine allows rlogin, telnet or cala_rex to start the installation process. If the target directory is accessible, but no remote login is allowed (neither rlogin nor telnet nor cala_rex ), at least the installation files can be copied to it (see description of the Copy files only checkbox below). If the CALA installer is started directly from CD, the File transfer method box contains the option Connected drive/dir only. The Remote execution method box contains the option Rlogin only. If the CALA installer is started from the WebConsole in a full FSM environment, the File transfer method box contains the options cala rex and Connected drive/dir. The Remote execution method box contains the option cala rex only. The configuration can be changed to allow more protocols if required. To install on a remote machine select the remote machine radio button. This enables the comboboxes for Filetransfer and Remote execution as shown in the screenshot below The installer remote settings panel The Install method area contains two lines: the first line configures the file transfer to the target host and the second configures the remote execution. If the Copy files only checkbox is selected, the installation files are only copied to the target directory without starting the installation process. A setup script or batchfile is created which can be run manually to complete the installation. A dialog box shows how the installation can be completed. The file transfer can be done either by ftp, via a locally mounted drive (Windows system) or directory (nfs on unix) or via cala_rex. Select the transfer type from the listbox. If ftp is selected, a user and password for accessing the remote host must be given. When choosing Connected drive/dir , the mount point (or drive) on the installing host and the drives or directories original name on the target host are needed. These additional parameters can be entered after pressing the ... button to the right of the Filetransfer combobox. When choosing cala rex , no further parameters are needed. 11 Chapter 5. Installation CALA should be installed on drive C: on a remote host. This drive is connected to Z: on the installing machine, so Z: is the mount point and C: is the original drive. Example 5-1. Example for using Connected drive/dir on MS Windows CALA should be installed in directory /opt/FileNet/SysMon/cala on a remote host. The directory /opt/FileNet/SysMon is connected to /mnt on the installing machine, so /mnt is the mount point and /opt/FileNet/SysMon is the original directory. Example 5-2. Example for using Connected drive/dir on Unix Choosing the file transfer method The second line sets the remote execution protocol telnet, rlogin ot cala_rex. If Rlogin is selected, a password may not be needed the appropiate field can therefor be left empty. If telnet is selected, a user and password for accessing the remote host must be given. These additional parameters can be entered after pressing the ... button to the right of the Remote execution combobox., When choosing cala rex , no further parameters are needed. Choosing the remote execution methd To start the installation, perform the following actions: • select a product • select the remote machine button 12 Chapter 5. Installation • enter the ftp user and password or the mount point and original directory if required • enter the telnet or rlogin user and password if required • customize target directory and possibly JDK path • CALA only: you may choose a default configuration from the configurations listbox and set its parameters in the settings dialog which appears after pressing the Set configuration variables button. • press the Install button The following installation process is the same as for local installation, see above for details. Information for users of the Microsoft Windows ftp server and telnet servers: When the Microsoft Windows ftp server (part of the internet information server IIS) is used, it must be ensured, that the directory listing style is set to Unix for all directories accessed by the CALA tools. Setting MS Windows ftp server to UNIX mode The Windows telnet server must be configured not to use NTLM authentification. The NTLM authentification parameter must be set to 0. (Use the tlntadmn tool to configure the telnet server.) Further installation options The options located in the Install options panel control system-specific settings such as autostart. CALA installer further options Keep monitor settings If this checkbox is selected, the current monitoring configuration on the client will not be replaced by original monitor settings contained in the selected configuration archive. Reconfigure only If this checkbox is selected, the binaries will not be transferred during the installation process. The configuration file will be recreated according to the settings in the Set configuration variables dialog. 13 Chapter 5. Installation Create environment file Tells the installation script to create the environment file set_cenit_env.sh in the directory /etc/cenit (on UNIX) or /etc/cenit (on Windows) or in a subdirectory of this directory. Uninstall Check this option to remove the product from the selected client. Autostart • Select After installation and at boot time to create the links that are required to start CALA at boot time (UNIX) or to register the CALA service for automatic startup (Windows). CALA will be started after successful installation as well. • Select After installation to start CALA only after installation. • Select None if CALA should not be started automatically at all. CALA installer informational messagebox for completing installation The Unpack binaries checkbox is available for CALA only and is selected by default. If it is unchecked, only the CALA configuration, but not the binaries are copied or updated at the target host. This feature is useful when using the CALA Configurator and should only be used by experts. Select the Verbose mode box to get further information from the installation process. This may be helpful if the installation fails. Relationship between installer GUI install_cala.sh The following table shows the relation between the parameters and the command line options of install_cala.sh. parameter corresponds to Product -product source directory -sourcedir Chapter 5. Installation Default value (if any) subdirectory Images on the same level as current directory; if ../Images does not exist, current directory 14 parameter corresponds to Default value (if any) Target directory -targetdir current directory CALA cache dir: -cachedir current directory JDK path -jdkdir Directory where the java binary used to start the CALA installation GUI is located Unpack binaries -untar Uninstall product -remove (not specified) Keep monitor settings -keepmonitors (not specified) Table 5-1. Relationship between GUI parameters and command line options of install_cala.sh For details see description of install_cala.sh in Annex 9: Additional tools. 15 Chapter 5. Installation Chapter 6. Component Architecture CALA is realized as a multi component Client/Server architecture, which enables customers to realize any kind of centralized and distributed Logfile and monitoring architecture. Almost all components are available on a comprehensive list of platforms (see restrictions on below site). CALA System platforms For detailed information about supported server and client platforms check the latest release notes. Supported JAVA JRE or JDK versions (CALAGUI and CALA V2S Editor prerequisite) For detailed information about required Java JRE or JDK versions for JAVA tools check latest release notes. Implementation on Microsoft Windows based systems Implementation of CALA on the Windows system platform has been implemented as an Windows Service. CALA installation as Windows Service Installation of the Windows Service is performed using the program cala_srv.exe. Installation with start mode manual : cala_srv.exe -install Installation with start mode automatic : cala_srv.exe -auto CALA de-installation on Windows systems To remove the CALA Windows service start cala_srv.exe -remove from the command line. Configuration file logctlsrv.conf for a Windows service installation If CALA is installed as an Windows service, configuration file logctlsrv.conf must either be placed in directory/folder %SystemRoot%\system32\config or in a directory/folder of your choice, which is mapped to the environment variable CALA_DIR of the Windows system environment. The CALA Windows service reads environment variables CALA_DIR and CALA_CACHE_DIR out of the registry (registry key 16 Chapter 6. Component Architecture HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\cala_srv) if they are not mapped in the environment. If all registry keys are set properly (see chapter Tivoli integration, Post distribution: CALA installation for details) there is no need to reboot the Windows system. Note: The CALA processes can also be started as a normal program instead of an Windows service. In this case the CLI-program logctlcmd (refer to description) should be used should be used for starting and stopping of the CALA components. Note: On Windows, CALA needs the file PSAPI.DLL to be available on the system. The file is automatically installed with CALAin the CALA installation directory and must not be removed. Client / Server Architecture To satisfy various requirements (source-independent duplicate recognition, performance, configuration during operation), a specific client/server architecture was developed for the CALA system which distinguishes between the following functions: • Reading of event sources • Event filtering • Classification and duplicate recognition • Transmitting data (e.g. sending events to the T/EC) All component of this architecture can be implemented on various systems, or just on a single system (computer). This diagram illustrates a possible architecture of the CALA system: 17 Chapter 6. Component Architecture This open architecture enables CALA to be implemented at almost any desired level of complexity or heterogeneity. In addition, the CALA firewall component calaproxy can be interposed between any FIR-based component. Implemented components Read-only component (Reader) Readers can be used to read event sources. Event sources can be regular files (Logfiles) or pipes, but also Windows Event Logs. The following readers are components of the CALA module: Component name Component type Description ascfileread ASCII File reader This component reads files and pipes available in ASCII format. ntevtlogread Windows Event Log reader This component reads out the Windows Event Log. mssqlread MS SQL database reader This component reads logfiles written into a ms sql database. oracleread Oracle database reader This component reads logfiles written into a oracle database. jdbcread JDBC database reader This component reads logfiles written into a database accessible via JDBC. 18 Chapter 6. Component Architecture For all readers there is a special CLI parameter (-E ), which should be used whenever the intention is solely to process events from the sources being read and which have been written to (generated) since the adapter was started. Note: As an option, other readers can be implemented on a customer-specific basis. Filter component (Filter) Filters are used to disassemble data streams, which were read out of event sources by readers. In all cases, readers are only able to transmit to filters because only they have the ability to disassemble unstructured data streams into events (FIRs) based on format definitions (format files). In technical terms, filters are arranged between the reader and the processing process or the emitter process. Component name Component type Description tecfmtfilt T/EC Format filter Interprets and classifies input from components ascfileread and ntevtlogread based on Tivoli .fmt files. This protects existing customer investment in format files. v2fmtfilt Complex filter This component filters and interprets input from reader ascfileread based on the CALA-format description. Event generating components Component name Component type Description snmpread SNMP trap receiver Receives SNMP traps and forwards them as CALA events (FIRs) to the specified targets. calamon Monitoring engine Executes monitoring scripts/programs and generates events (FIRs) depending on return codes or output. Processing server msgclsfsrv (Message Classification Server) Correlation system msgclsfsrv is the brain behind CALA and is used for duplicate detection, event handling and computing (basic forms of computing). Event handling includes the functions of event suppression as well as escalation (change in severity). Component name Component type Description Chapter 6. Component Architecture 19 Component name Component type Description msgclsfsrv Classification server This component is the brain behind CALA. It contains the functions of classification, duplicate detection, event suppression and escalation. Sub components Rules Engine and Message Mapping (msgclsfsrv sub components) The rules engine and the massage mapping component are subcomponents of the message classification server and are used to process correlating events and timer on events as well as any manipulation on events. For detailed information about the Rules Engine and the Message Mapping Component refer to related sub chapters within this chapter. Sub component Completer (msgclsfsrv sub component) The sub component Completer in process msgclsfsrv is used for downstream (i.e. after processing by the central processing server) bulk setting or deleting of slots (Tivoli T/EC slots) as a function of other existing or unmapped slots, or to fade out slots. Language constructs such as if!, unless! or for! are implemented for processing purposes based on the existence (if) or absence (unless) of slots. Application area: Mapping of unmapped default slots, e.g. severity. Fading out of mapped slots, e.g. those required for internal processing. Sub component Remapper (msgclsfsrv sub component) Sub component Remapper in process msgclsfsrv is capable of re-mapping slot contents, or of defining new slots. This enables the system to rename event class names in a customer-specific manner. Example: Changing LogfileBase to _LogfileBase. Comment: the event class _LogfileBase must be defined in a BAROC file. Emitter components Emitters (Senders) are used for sending or subsequently processing events, e.g. after a report has been raised. The following emitters/senders are available: Component name Component type Description Chapter 6. Component Architecture 20 Component name Component type Description tecfmtemit T/EC-EventPreparation This component prepares the logs to be processed and sends them to the T/EC transmit component (see below). cmdemit Task Engine This component is capable of executing any commands. Parameters are read out of Message Map files. reportemit Reporting Emitter This component is used for all the events. Events may be reported in T/EC Event dump format or in a own format specified in a template file. snmpemit SNMP Trap Emitter This components throws SNMP events from received CALA FIRs. smtpemit SMTP Emitter This component sends emails to report the received CALA FIRs. jdbcemit Database Emitter This component writes events to a database accessible via JDBC. T/EC transmit component Component name Component type Description tecifcsrv T/EC Interface Server This component sends prepared events to the T/EC. There are 3 variants of this component: • Secure Version: oserv communication (ManagedNode) • Unsecure Version: TCP/IP communication (EIF) • Endpoint Version: Tivoli TMA communication Application proxy for DMZ Component name Component type Description calaproxy Application proxy This component is used as an application proxy in Demilitarized Zones (DMZ). This sends received FIRs to a downstream component on a computer on the far side of a firewall. Control component logctlsrv Component name Component type Description Chapter 6. Component Architecture 21 Component name Component type Description logctlsrv Control server This component is used to control and configure all other CALA components. Logctlsrv is controlled using the CLI (command line interface) logctlcmd. Logctlcmd reads configuration information from the file logctlsrv.conf and starts the process logctlsrv, which then takes control of the configuration management of all other CALA component. CLI logctlcmd The Command Line Interface logctlcmd is used on all platforms supported by external control of the CALA component. Note: Starting and stopping the CALA component on Windows systems can be implemented by the Windows Service Manager (refer to CALA installation as an Windows service) as well. Supported logctlcmd commands startup Starting the CALA component. A CALA Windows installation is started by the command net start cala_srv if CALA is installed as a service. shutdown Stopping the CALA component. A CALA Windows installation is stopped by the command net stop cala_srv if CALA is installed as a service. restart Restarts the CALA component status Status query of the CALA component. In addition to the output of all status information for the installed CALA component, output includes information about the Tivoli environment employed as well as important CALA environment variables. reconfigure Reconfiguration of the CALA component during runtime. Changes to the configuration (configuration file logctlsrv.conf) are taken into account at this time. maintenance_on Activation of the Maintenance Level (processing is delayed) maintenance_off Deactivation of the Maintenance Level (processing is restarted) 22 Chapter 6. Component Architecture test This command is employed in order to generate a CALA test event from a component. This test event can be used to test communication between the component (local on a computer or on a remote computer). Note: The CALA programs need a shared library which has to be available to them. See the following table for the name of the library and the environment variable to be set to it s path. operating system filename of shared library environment variable MS Windows libcala.dll PATH AIX libcala.so LIBPATH Solaris libcala.so LD_LIBRARY_PATH Linux libcala.so LD_LIBRARY_PATH HP-UX libcala.sl SHLIB_PATH Generating test events Apart from the emitters (senders), all components are able to generate test events. This means that communication can be tested between the individual components. Test events can be generated using the CLI call: logctlcmd test Possible component architecture (predecessors / successors) The following table illustrates the possible component architecture with predecessors (previous stage) and successor (subsequent stage). Componentname predecessor successor ascfileread - tecfmtfilt, v2fmtfilt ntevtlogread - tecfmtfilt v2fmtfilt tecfmtfilt ascfileread, ntevtlogread any FIR processing componenta v2fmtfilt ascfileread, ntevtlogread any FIR processing componenta snmpread - any FIR processing componenta calamon - any FIR processing componenta mssqlread - any FIR processing componenta oracleread - any FIR processing componenta jdbcread - any FIR processing componenta Chapter 6. Component Architecture 23 Componentname predecessor successor javasrv / pchread - any FIR processing componenta msgclsfsrv any FIR generating componentb any FIR processing componenta cmdemit any FIR generating componentb - reportemit any FIR generating componenta - snmpemit any FIR generating componenta - smtpemit any FIR generating componenta - jdbcemit any FIR generating componenta - tecfmtemit any FIR generating componenta tecifcsrv (end, sec, uns) calaproxy any FIR generating componenta any FIR processing componenta remote component any FIR generating componenta any FIR processing componenta, tecifcsrv (end, sec, uns) tecifcsrv (end, sec, uns) tecfmtfilt - Notes: a. FIR processing components are: • msgclsfsrv • cmdemit • reportemit • tecfmtmit • calaproxy • snmpemit • smtpemit • jdbcemit • remote component b. FIR generating components are: • tecfmtfilt • v2fmtfilt • calamon • snmpread • msslread, oracleread, jdbcread • javasrv / pchread • msgclsfsrv • calaproxy • remote component Communication between CALA components Communication between individual CALA components is based on TCP/IP communication with variable package size. The data records read in (Logs, Windows Event Log, Syslog, etc.) are transferred by the filter processes to Filter Input Records (FIR) which form the basis for communication between all other CALA components. When this standardized data object (FIR) is implemented for CALA component communication, CALA components are able to link up in almost any conceivable order. 24 Chapter 6. Component Architecture The data (FIRs) can be transmitted through ports configured in any desired manner, and every component can also receive or transmit data via any desired number of ports. Default tcp ports used by CALA components The following table shows the default tcp ports used by CALA components. Chapter 10 "Configuration file logctlsrv.conf" describes how the port settings can ne changed. component name default port logctlsrv 23861 logctlcmd 23860 ascfileread 23831 ntevtlogread 23832 calamon 23833 snmpread 23834 oracleread 23835 mssqlread 23836 jdbcread 23837 tecfmtfilt 23838 V2fmtfilt 23839 msgclsfsrv 23840 calaproxy 23841 tecfmtemit 23842 cmdemit 23843 reportemit 23844 snmpemit 23845 smtpemit 23846 tecifcsrv 23847 jdbcemit 23848 Event caching If a component looses contact with a downstream component during the transmission of events, these events are then stored in a cache file. In this case, the client process tries to reconnect to the server every 5 seconds. 25 Chapter 6. Component Architecture As soon as a new connection can be established, the cached event can be transmitted. Once the transmission confirmation has been received, the cache entries are deleted. Cache files are stored in the directory/folder defined by the environment variable CALA_CACHE_DIR. If CALA_CACHE_DIR has not been set, environment variables TEMP and TMP (with Windows also the SystemRoot) are evaluated. If none of these variables has been set, the cache file is stored in the current directory/folder. Note: The cache files are named . . .cache, so they may not be displayed by a normal ls call. 26 Chapter 6. Component Architecture Chapter 7. Configuration file logctlsrv.conf The entire configuration of CALA is performed by the file called logctlsrv.conf which must be present on every system equipped with any CALA component. logctlsrv.conf contains all communication and start information required for the CALA configuration component implemented on the computer to operate locally, as well as other component-dependent options. Important: This chapter is only intended to explain the format. Configuration file logctlsrv.conf is generated using the graphic configuration program CALAGUI which is part of the CALA Plus Module. CALAGUI only supports configuration files for later changes (re-opening a configuration) which were generated from itself. It may not be possible for the CALAGUI to further process manually changed logctlsrv.conf files (i.e. to read them back in). Format of configuration instructions All logctlsrv.conf file entries follow the same format = {, } Comment: these instructions are constructed hierarchically and reflect the tree structure. Global configuration instructions applicable to all components The following chapter contains the global (once per configuration) and supra-component parameters/instructions. These instructions are now described in abridged form, each one being explained with an example. Configuration instruction serverlist The instruction is the only instruction which is a mandatory requirement for the configuration file. serverlist describes the list of components which have to be started on the corresponding system by the control command logctlcmd. serverlist= Figure 7-1. Format of the serverlist instruction If a component has to send data (FIRs) for a downstream component to another computer, the name of the component on the downstream computer has to be part of the list. 27 Chapter 7. Configuration file logctlsrv.conf 001 ... serverlist=tecifcsrv,tecfmtemit,msgclsfsrv,tecfmtfilt,v2fmtfilt,ascfiler . ead Example 7-1. Example using serverlist Note: Comment: The sequence reflects the starting sequence for components. Every component featured in the serverlist instructions list requires a separate configuration entry which controls the parameters and communication routes governing this component and downstream components. The general structure is as follows:
=! {,! } run instruction The run instruction describes the commandline call of the relevant component with parameters run! Figure 7-2. Format of run instruction The parameter -P is a mandatory requirement for each component, it defines the communication port for the component. All CALA components also support parameter -d which, in the event of error diagnosis being required, describes the diagnosis file for all outputs. Parameter -d must directly be followed by the name of the logfile (no blanks are allowed) with or without path indication. Caution Diagnosis files from various components can reach a size of several hundred Megabytes within just a few minutes. 001 run!tecfmtemit P 51967 dtecfmtemit.diag Example 7-2. Example using the run statement The above example defines port 51967 as the communication port for the component tecfmtemit. Diagnosis information is directed to filename tecfmtemit.diag. In case of several 28 Chapter 7. Configuration file logctlsrv.conf ports being used, these port numbers must be indicated in the form of a list, separated by a semicolon. For complete description of all parameters refer to the appendix. target Instruction The target instruction describes the list of components which follow this component and therefore receive data (FIRs) from the current component. targets! {, } Figure 7-3. Format of targets instruction 001 targets!tecifcsrv Example 7-3. Example targets usage This example defines the CALA component tecifcsrv as a target component for output data (FIRs) By default the outgoing port (the local port which is used to establish the connection with a server) is choosen by the system and can therefore by any available port. For security purpose when communication via a firewall, it may be useful to define the outgoing port, which can be done by adding : to the target component. 001 targets!tecfmtemit:5656,calaproxy:5655 Example 7-4. Example targets usage with outgoing port port instruction The port instruction is needed for inter-process communication. This port must match the port number of the run instruction. In the case of several ports being used, these port numbers must be indicated in the form of a list, separated by a semicolon. 29 Chapter 7. Configuration file logctlsrv.conf port! {; } Figure 7-4. Format of port instruction port number: tcp communication port, on which the process listens for incoming data. 001 port!51967 Example 7-5. Example port usage Note: The server components can be configured for a range of ports. Port list functionality On both client and server side, a list of ports (separated by semicolons) can be indicated. Components can use these for communication purposes (on the output end) and can be addressed via them in return (input end). conf instruction The conf instruction is used for configuration management (logctlcmd and logctlsrv). All sub instructions are defined by using the conf instruction are assigned to the relevant process, then made available to the remaining CALA components (configuration management). conf! {, } Figure 7-5. Format of conf instruction 001 conf!run;port;targets Example 7-6. Example conf usage 30 Chapter 7. Configuration file logctlsrv.conf The aforementioned conf instruction defines sub instructions run, port and targets for these components. The configuration management function supplies/updates all CALA components with this information. Note: run and port do not have to be specified explicitly. ip instruction The ip instruction is needed whenever the component being described is installed on a different computer (multi-stage CALA concept). ip! Figure 7-6. Format of ip instruction Note: If the ip instruction is set, no run and no conf instruction can exist for these components. 001 msgclsfsrv=ip!foo.bar.com Example 7-7. Example ip usage In this case, the server foo.bar.com is used as a target system. Data (FIRs) for msgclsfsrv is sent to this host for further processing. Serverlist functionality Instead of one IP address or hostname, a list of targets hosts can be specified. The single hostnames/IP addresses are separated with semicolons (;). When connecting to the server, a client will try the hosts in the given order and connect to the first host accepting its request. In other words, if the first server fails to respond, the next server on the list is used. 001 msgclsfsrv=ip!foo.bar.com;thud.grunt.com;194.39.165.97 Example 7-8. Example ip instruction using the serverlist functionality 31 Chapter 7. Configuration file logctlsrv.conf Broadcast functionality Client components can use the BROADCAST wildcard instead of an IP address/hostname to make use of the implemented broadcast functionality. After a start or after a breakdown in communication with a server, the component searches for a new server via the defined port (port list) in the local network. Entry BROADCAST must be used within the server list of the remote component, "enable broadcasting" must be ticked at the receiving component. For broadcasting on a specific subnet, BROADCAST: can be used. 32 Chapter 7. Configuration file logctlsrv.conf Chapter 8. Configuration GUI There is a graphical user interface for configuration which creates the CALA-configuration file as well as a Tivoli ACP Profile for distribution purposes. Using the CALA Configuration GUI The configuration GUI is used for creating CALA configurations. Every component-specific parameter can be set using the graphical system. The menu item Open (Configuration−→Open) is used to read an existing configurations, the menu item Create (Configuration−→Create) enables the creation of a new configuration. The configuration system is able to create pure Tivoli ACP Profiles as well as only the CALA configuration file. Starting the GUI CALAGUI can be started with the batch file calagui.bat on Windows systems or with the shell script calagui.sh on UNIX. Both scripts are located in the calagui directory. CALAGUI supports the following options: -c specifies the directory that contains the configuration files for CALAGUI. Default: conf (This value should not be changed.) -o specifies the standard output directory for configurations. It is recommended that you create a subdirectory for each configuration and its related files. Default: data -s specifies the path to additional scripts that are executed when a configuration is saved. Default: scripts -x specifies the output path for exporting .cala files. Default: export Note: The export directory is used as repository for files related to the CALA Configurator. At the moment, you must synchronize your CALAGUI installation(s) and the TMR Server manually. For details about the subdirectories created in the export directory and the synchronization process between CALAGUI, TMR Server and ManagedNodes see the annex, section Directory structure in the export directory of CALAGUI. If you want to change the default values, you must edit the corresponding start script. It is currently not possible to specify the options on the command line. 33 Chapter 8. Configuration GUI The architecture window opens whenever the configuration tool starts. All available functions (Create, Open, Save, etc.) are arranged under menu item Configuration The CALAGUI Configuration menu Setting up a new configuration (Create ...) The CALAGUI new configuration dialog To create a configuration, please select menu item Create ... in the Configuration menu. Select a configuration name of your choice, a Tivoli ProfileManager name for generating an ACP Profile, the target platform and the type of configuration. Note: The type of configuration is only used to select the default settings to be used for the architecture. These settings can then be modified later. Pressing the OK button loads the selected architecture into the architecture window, based on standard definitions. 34 Chapter 8. Configuration GUI The default HP-UX client configuration. The illustrated example contains the default client HP-UX architecture. Opening an existing configuration (Open ...) To open an existing configuration, please select menu item Open ... in the Configuration menu. Select the desired configuration by selecting the appropriate configuration file. The open configuration dialog The existing configuration is then loaded into the architecture window. Saving a created or changed configuration (Save, Save as) Saving with the menu option Save is used to save a new or existing configuration, Save as ... is used to save an existing configuration under a different name. If the configuration being saved 35 Chapter 8. Configuration GUI has not already been assigned a configuration file (using Create ... or Save as ... ), the following file browser window is opened. The save configuration file dialog Enter the desired name of the configuration file (any name can be selected) and confirm by pressing the Save button. Another menu box then appears on screen. The options dialog when saving a configuration Select the type of Save process (Only configuration file or ACP Profile) and confirm by pressing the OK button. The configuration created is then saved under the specified name.It is recommended that you create a subdirectory for each configuration and its related files. Exporting parts of the configuration for use with the CALA configurator Input files for the CALA configurator can be created by selecting the menu entry Export .cala files. A new dialog window opens, where the user can select the data types to be exported. 36 Chapter 8. Configuration GUI The .cala files export dialog Secondary data type export Each of the secondary data types of the loaded configuration can be selected for export. General data export The general data types need a unique postfix to be given. This enables for example several remote components to be configured. After selecting the data types to be exported and pressing Check and continues export , the configuration of the selected data types is checked e.g. if the name of the prefilter and message map files are conform to the naming conventions. Warnings from this check shouldn t be ignored non-conform files will be ignored by the CALA configurator. The Check secondary data types dialog 37 Chapter 8. Configuration GUI Selecting Save now from the check window creates the export files. All files are created in subdirectories of the export directory specified with the x switch in the start script of CALAGUI. A subdirectory is created for each secondary data type. All referenced files (format files, map files ) that can be found in the directory where the configuration is saved and that are conform to the naming convention are copied to the corresponding export directory. Note: If any referenced file do not exist within the directory where the configuration file is located, not all required files will be present after export. All missing files need to be copied manually afterwards. For more information about the CALA configurator and the exported file format refer to chapter CALA Configurator in the appendix. Differences between CALAGUI configurations and CALA Configurator There are some differences between configuration files that are saved from CALAGUI and configurations that are generated by CALA Configurator even if you compare an "original" configuration and a configuration that was generated from the .cala files exported from the "original": • The port numbers for the components are taken from the template file. The port for the remote component is the only one that is taken from the original configuration. • The logical names of the components as well as the binary names in the run-statements are taken from the template file. • If you want to start CALA under a specific user you must specify this user and the corresponding password as parameter for the task Generate profile for CALACFG. The user specified in the global configuration settings in CALAGUI will not be passed to the ACP record generated for CALA Configurator. • If you want to export settings for a client configuration as well as for a server configuration that contain the same secondary data types, you must include both configurations in one configuration file. This is required because the .cala files are rewritten each time you export a configuration file that contains definitions for a secondary data type that was exported before. Altering the global configuration settings The menu option Global configuration settings can be used for subsequent changes to global settings in a configuration. Note: Maintenance windows for the CALA can be also specified in this window. 38 Chapter 8. Configuration GUI The Global configuration parameters dialog Any desired number of maintenance windows can be configured. Simply ensure that settings do not overlap. Global settings are set by pressing the OK button. Global settings in the configuration file The configuration shown above will result in the following configuration lines: 001 002 003 004 logctlsrv_port=11000 logctlcmd_port=11001 cala_srv_port=10999 maintenance=Fri 2300;Sat 0300;01 2200;02 0500 Example 8-1. Global settings in the logctlsrv.conf file Configuration instructions logctlsrv_port and logctlcmd_port To change the TCP port used by the logctlsrv and logctlcmd programs, use the instructions logctlsrv_port and logctlcmd_port in the configuration file. If no port is set for one of these programs, the default port (51956 for logctlsrv and 51952 for logctlcmd) is used. The ports for log control server (logctlsrv) and log control command (logctlcmd) are taken from the GUIs entry fields Port of log control server and Port of log control command Note: If any of these ports are provided as command line argument to logctlcmd, the given port(s) is/are used instead of the port(s) from the configuration file. 39 Chapter 8. Configuration GUI Note: CALA should can only be configured to use port numbers in the range from 1025 to 65535. Do not change the ports manually to any number outside this range! 001 002 logctlsrv_port= logctlcmd_port= Figure 8-1. Format of logctlsrv_port and logctlcmd_port instruction 001 002 logctlsrv_port=11000 logctlcmd_port=11001 Example 8-2. Example for logctlsrv_port and logctlcmd_port usage Configuration instruction cala_srv_port (Windows systems only) If the Windows Service is used, the port for this service can also be set in the configuration file. If not specified cala_srv uses the default port 51951. 001 002 cala_srv_port= logctlcmd_port= Figure 8-2. Format of cala_srv_port instruction 001 cala_srv_port=10999 Example 8-3. Example for cala_srv_port usage The configuration instructions logctlsrv_adapters and logctlcmd_adapters These instruction are used to specify the network adapters used by logctlcmd and logctlsrv. By default these programs listen on the loopback device only, which means, that only local processes can connect to them. This behavior prevents the logctlsrv from being attacked by remote invaders, but is also denies requests from remote logctlcmds.To open the log control server for communication with remote processes set logctlsrv_adapters to the network adapters from which connections are allowed. 40 Chapter 8. Configuration GUI To enable a log control command to communicate to remote processes, the affected network devices have to be given in the logctlcmd_adapters instruction.This instructions can be overwritten by using the logctlcmd command line parameter 001 002 logctlsrv_adapters= {: } logctlcmd_adapters= {: } Figure 8-3. Format of logctlsrv_adapters and logctlcmd_adapters instructions 001 002 logctlsrv_adapters=10.0.114.201 logctlcmd_adapters=10.0.114.201:192.168.1.1 Example 8-4. Example for logctlsrv_adapters and logctlcmd_adapters usage Maintenance instruction The maintenance instruction defines fix maintenance windows which occur periodically. Those maintenance windows are set in the GUIs Maintenance Window settings table. Within a maintenance window, CALA does not read any events from the event sources. Reading from sources is resumed when the maintenance window is over. All events created within a maintenance window are discarded by the CALA components, even if they are read outside a maintenance window. 001 ... maintenance=[ | ] <2digit hours><2digit minutes>{ . ;[ | ] <2digit hours><2digit minutes>} Figure 8-4. Format of maintenance instruction 001 maintenance=Fri 2300;Sat 0300;01 2200;02 0500 Example 8-5. Example for logctlsrv_adapters and logctlcmd_adapters usage Fixed maintenance windows are configured in the configuration file by using the maintenance instruction. Each maintenance window consist of a pair of dates, given in the following format: [ | ] <2digit hours><2digit minutes> A daily maintenance window only contains the part for hours and minutes (no dayofweek or dayofmonth is given). Hours are given in the 24 hours format. 41 Chapter 8. Configuration GUI A weekly maintenance window also contains a three letter abbreviation of the weekday followed by a blank and the hour- and minute-string. A monthly maintenance window is configured giving the day of the month, followed by a blank and the hour- and minute-string. The example given above defines one weekly maintenance window from 23:00 on Friday to 03:00 on Saturday and one monthly maintenance windows from 22:00 on each 1 st of the month to 5:00 on each 2 nd . More global settings There are some more global settings like operating system, profile manager, configuration name, user and password, which are needed for configuration purposes. For details refer to chapter Analysis of configuration file. 001 002 003 004 005 #operating-system: nt #profile-manager: ACP for CALA #name of configuration: NT_Client #user: cala #password: 1b14460e00 Example 8-6. Example for additional settings in logctlsrv.conf Configuration check You can use the menu option Check loaded configuration to check your CALA configuration. Theresult window of Check loaded configuration 42 Chapter 8. Configuration GUI component configuration Each component has a context menu, which opens when pressing the right mouse button while the cursor is placed over the component. The components context menu If the menu item Delete component is chosen, the selected component is removed from the configuration. The menu entry Settings opens the component specific configuration window. Due to individual parameters every component has its own configuration window, see chapter Component-specific configuration for details. The components configuration window can also be opened by double clicking the component. Selecting one of the menu options Edit sources and Edit targets will open another dialog. The Settings of connection to Source dialog 43 Chapter 8. Configuration GUI The Settings of connection to Target dialog Both dialogs are very similar: • Add new Source or Add new Target will create a new component (a dialog for selecting a logical name and component type will appear) and connect it to the currently selected component. • Connect with existing Source Connect or Connect with existing Target shows a list box where the user can choose the component to connect to. • To remove a component from the source/target list, select the source/target in the left list box and choose Delete component . This will remove the selected source/target component from the list. • The changes can be applied by pressing Ok or discarded by selecting Cancel 44 Chapter 8. Configuration GUI Chapter 9. Component-specific configuration By double clicking on a component symbol, the components settings dialog window appears. This settings dialog differs between the components, but some fields are common. Common settings This is a sample settings window, the window of the component calaproxy, which is explained later. A sample settings window for a CALA component The first element of each settings dialog is the choice Component type where the type of the component is specified (calaproxy in this example). The dialogs face depends on the components type and may change if the type is changed. Logical name This entry field contains the logical name of the component, which must be unique within the configuration. 45 Chapter 9. Component-specific configuration The logical name is used to address the components configuration. The logical name will appear in the serverlist and as the identifier for the components configuration line. 001 002 ... serverlist= calaproxy calaproxy=run!calaproxy -P 11022,port!11022,targets!remote_emit, conf!p . ort;run;targets Example 9-1. Example configuration of logical server name Portlist (port instruction) The port list gives the ports, the process listens on, at least one port has to be defined for each process. The same port number must not be used for two components on the same machine. 001 port! {; } Figure 9-1. The port list configuration entry has the following format: 001 port!11022 Example 9-2. Example portlist For local processes, there is a need to given the port number on the argument line. 001 ... -P {: | - : }[- ] Figure 9-2. Format of argument line containing port assignment 001 run!calaproxy -P 11022 Example 9-3. Example run statement containing port assignment Run statement (run instruction) The entry field Run statement sets the binary of this component. The first 7 characters are fix, because they specify the components type which is calapro (which stand for calaproxy) in this case. When using the same component twice on one machine, different binary files must be used. 46 Chapter 9. Component-specific configuration 001 run! Figure 9-3. Format of run statement 001 run!calaproxy -P 11022 Example 9-4. Example run statement Debugging If the Checkbox Enable debug mode is set, the process creates a log file for debugging. If no filename is given in Debug filename , the log is written to diag_log.txt There is also a list box for setting the debug level. The debug level can be set to any number between 0 (report everything) and 9 (report only fatal failures). If no debug level is chosen, all messages are written into the debug file. 001 -d[[: :] ] Figure 9-4. Format of run statement containing debug arguments 001 run!calaproxy -P 11022 d:5:calaproxy.log Example 9-5. Example of run statement containing debug arguments (not from the window above): Supplementary parameters Any other program argument can be set here. There are some common arguments, which are described here. Some components have special arguments, please refer to the components description. parameter example description default value -M -M100 max. number of client connections to accept 100 -SF -SF stop sever if bind to socket fails disabled -NSR -NSR don’t allow socket rebind allowed -CT -CT30 set connection timeout before caching (a cache file 30 is created if no connection to a server could be created for seconds) -CM sets the maximum size of the cache file (in bytes) CM5000000 Chapter 9. Component-specific configuration 5000000 47 parameter example description default value -CD -CD1440 sets the max. age for cached events before they are discarded, 0 disabled discarding 0 -AT -AT60 timeout for receiving acknowledges from server (seconds) 120 -AS -AS2 server acknowledge sending period (seconds) 2 disabled -CLE -CLE create connection lost events -CAE -CAE create connection accepted events for all accepted disabled connection from remote clients -CAT -CAT60 Sets the connection accept timeout this is the time (in seconds) in which the client has to send a first data package after connecting. If is a positive value, a accept timeout event is created if any client connected but didn t send any data , if is negative, no event is created. -CAT0 disables this feature -30 -ZHEARTBEAT_PERIOD= Tells the component to create heartbeat events ZHEARTBEAT_PERIOD=120 periodically. (Does not work for ascfileread, snmpemit and smtpemit.) disabled -ZCREATE_STATUS_EVENTS=1 Creates CALA startup and shutdown events. disabled ZCREATE_STATUS_EVENTS=1 These Events are only send to remote servers or to T/EC. (Does not work with ascfileread.) There is a special implementation of startup/shutdown events for snmpemit and smtpemit refer to chapter Configuring status events of snmpemit and smtpemit in the appendix for details. There are some more parameters for encryption, please refer to chapter Security for further information. The following parameters are set by the CALAGUI and must not appear in the supplementary parameters text field: parameter example description default value -P -P 16001 port(s) to open for event reception several ports none can be given separated by colons, a port range can be given like this: <1st port>- . -d [: - :]filenamecreates a log file loglevel can be a number between don’t create d:3:msgclsfsrv.log 0 (log everything) and 9 (log only fatal errors), be a logfile aware, that log files may grow very fast if loglevel is set low -SB -SB enable broadcast server -AB -AB sets on which network device (give by ip address) {: } 10.0.114.201the component should listen for events, several disabled listen on all devices devices may be separated by colons the loopback device is added automatically 48 Chapter 9. Component-specific configuration Display version information All component binaries can also be called from the command line using the -v parameter, which prints a version information. The SNMP components snmpread and snmpemit also display the version of the used snmp library. [c:\opt\cenit\cala].\snmpread -v ******************************************************************* ** ** ** snmpread is part of the CENIT Advanced Logfile Adapter ** ** ** ** version: 2.01-064 - generation date: Jun 26 2003 10:57:18 ** ** ** ** (c)1999-2002 CENIT AG Systemhaus ** ** ** ******************************************************************* ** NET-SNMP version 4.2.3 ** ******************************************************************* [c:\opt\cenit\cala] Figure 9-5. Displaying version information of snmpread 49 Chapter 9. Component-specific configuration ascfileread The ASCII file reader component reads the data from one or more configured files and sends it to a filter for further processing. The ascfileread window is used for defining all files, directories, folders and the related data formats for reading out files and pipes. Wildcards (*, ?) can be used for directories/folders as well as for filenames. There are also additional wildcards to define logfiles and to specify hours, days, months and years within file names. E.g. if a logfile is to be monitored with a 2-digit month and a 4-digit year number at the end of its name (logfile_06-2000), the wildcard used for defining its name would look like this: logfile_MM-YYYY The Settings of ascfileread dialog ascfileread specific parameters and their setting in the configuration file The following configuration line was created from the window above: 001 ... ... ascfileread=run!ascfileread -P 11002,port!11002,targets!tecfmtfilt;v2fmt . filt,pathlist!1;/var/adm;2;/fnsw/local/logs/elogs,ptrnlist!1;messages;2 . ;elogYYYYMMDD,assoc!1;1;tec;solaris_syslog;2;2;v2;fn_log,conf!port;run; . 50 Chapter 9. Component-specific configuration targets;pathlist;ptrnlist;assoc ... Figure 9-6. An example configuration line for ascfileread pathlist instruction pathlist defines the list of paths (directories/folders) in which logfiles are searched. Its parameters are taken from the GUI configuration from the column Path definitions. 001 pathlist! ; {; ; } Figure 9-7. Format of pathlist instruction 001 pathlist!1;/var/adm;2;/fnsw/local/logs/elogs Example 9-6. Example pathlist instruction The above example defines two directories/folders in which the logfiles being processed may exist. A unique number must be defined for each path. This number is referenced using the assoc instruction described below. The separate configuration of paths and filename simplifies configuration if the same pattern has to be used for several paths. (E.g. if you are looking for the pattern *.log in three different paths, the pattern has to be configured only once.) Wildcards and variables requiring interpretation that can be used in pathnames. Supported wildcards: * designates any sequence of characters ? designates any character Variables requiring interpretation: HH two-digit hour display MM number of month, two-digit DD day, two-digit YY number of year, two-digit 51 Chapter 9. Component-specific configuration YYYY number of year, four-digit ptrnlist instruction (pattern list) The ptrnlist instruction defines the list of file patterns used for processing purposes. The ptrnlist parameters are taken from the GUI configuration from the column Pattern definitions. ptrnlist! ; [: ]{; ; . [: ]} 001 ... Figure 9-8. Format of ptrnlist instruction ptrnlist!1;messages:UTF-8;2;elogYYYYMMDD 001 Example 9-7. Example ptrnlist instruction Wildcards and variables requiring interpretation are used to describe filenames. Supported wildcards: * designates any sequence of characters ? designates any character Variables requiring interpretation: HH two-digit hour display MM number of month, two-digit DD day, two-digit YY number of year, two-digit YYYY number of year, four-digit For a list of supported encoding refer to Supported character sets. For every pattern entry a number is assigned which reflects the assoc instruction described below for reflecting the path/filename combination to be processed. 52 Chapter 9. Component-specific configuration • The above example defines two file patterns which are interpreted at run time. • The first pattern addresses a file named messages which is expected to be UTF-8 encoded. • Sample 2 is used to define precise daily logfiles, starting with elog. On 20.12.2000 this configuration would, for example, process filename elog20001220. The file is exptected to use the default system encoding. • sna*.err would address all filenames beginning with prefix sna and extension .err. • messages.? would identify all messages files having a one character extension. By default the ascfileread checks every 5 minutes for new matching paths and files. assoc instruction The assoc instruction associates paths from the pathlist with file patterns from the ptrnlist instruction. Each row from the Settings of ascfileread window’s table generates one assoc entry in the configuration file. 001 ... assoc! ; ; ; {; ; ; ; } Figure 9-9. Format of assoc instruction 001 assoc!1;1;tec;solaris_syslog;2;2;v2;fn_log Example 9-8. Example assoc instruction PathlistX represents a previously defined path number, ptrnlistX represents a previously defined filename number (pattern). The parameter can be selected from values and represents the relation to logfiles which can be described as .fmt, represents the logical link to complex data formats. The parameter can be selected from one of the format names defined under formatlist. (see configuration of filter components). This prompts a search for the following directory/folder combination: • /var/adm/messages (type tec/solaris_syslog) • /fnsw/local/logs/elogs/elogYYYYMMD (type v2/fn_log) Example 9-9. Files that would match with above pathlist, pattrnlist and assoc configuration Note: To read pipes on Microsoft Windows \\pipe\ must be given as path, the pipes name has to be given as filename. 53 Chapter 9. Component-specific configuration ascfileread command line parameters These parameters can be set in the field supplementary parameters parameter example description default value -E -E When opening a logfile the first time: skip all old events, send only new events disabled -e -e Skip all old events each time a logfile is opened, send disabled only new events -U -U 60 Sets the period when ascfileread looks for new files 300 in seconds. -H -H foo.bar.com Sets the name of the host, ascfileread runs on. The hostname is requested by gethostname() function if this parameter is not given. use gethostname() -B -B5 Specifies the max. number of 16K blocks to be send to the filters each second. This parameter should be used with care, because it may result in some unknown error events. no limit -O -OUTF-8 Specifies the default character set to be used for reading files if no encoding is specified. the system default 54 Chapter 9. Component-specific configuration ntevtlogread The ntevtlogread (NT Event log Reader) is used for reading the Microsoft Windows Event log. The Settings of ntevtlogread window is used to define all read functions and parameters. For every event log (system, security, application or any user defined eventlog), a dedicated secondary data type can be assigned. This makes it possible to use a separate format file for every type of eventlog. The Settings of ntevtlogread dialog ntevtlogread specific parameters and their setting in the configuration file The following configuration line was created from the window above 001 ... ... ... ... ... ntevtlogread=run!ntevtlogread -P 11003,port!11003,targets!tecfmtfilt,evt . log!1;system;2;application;3;security,spacereplacement!1;1;2;1;3;1,asso . c!1;tec;nt_system;2;tec;nt_application;3;tec;nt_security,skip_old!1;1;2 . ;1;3;0,prefilt_in!2;prefilt_nt_app.flt,prefilt_out!3;prefilt_nt_sec.flt . ,conf!port;run;targets;evtlog;spacereplacement;assoc;skip_old;prefilt_i . n;prefilt_out Example 9-10. An example configuration line for ntevtlogread evtlog instruction evtlog defines which eventlogs the reader should read. The eventlogs are given as a pair ; . 55 Chapter 9. Component-specific configuration The popup menu of the text field Eventlog name shows a selection of standard eventlog ids for Windows NT and Windows 2000 systems. Selecting a id from the popup menu pastes this id into the text field. 001 evtlog! ; {; ; } Figure 9-10. Format of evtlog instruction 001 evtlog!1;system;2;application;3;security Example 9-11. Example evtlog instruction This defines, that the Microsoft Windows system, application and security eventlogs must be read and each of them is given a numeric id (system=1, application=2, security=3). spacereplacement instruction Defines if blanks should be replaced by underscores for fields source and sid. The instruction consist of a pair ; for each logfile. If is set to 1 this means "spacereplacement on", 0 means "spacereplacement off". 001 spacereplacement! ;0|1{; ;0|1} Figure 9-11. Format of spacereplacement instruction 001 spacereplacement!1;1;2;1;3;1 Example 9-12. Example spacereplacement instruction This example switches space replacement on for the three defined event logs (system, application and security). skip_old instruction If this parameter is set for a logfile, all entries which have a timestamp before 0:00 clock of the current day, are discarded. The instruction consist of a pair ; for each logfile. If is set to 1 this means "skip old entries", 0 means "process old entries". 001 skip_old! ;[0|1]{; ;[0|1]} Figure 9-12. Format of skip_old instruction 56 Chapter 9. Component-specific configuration 001 skip_old!1;1;2;1;3;0 Example 9-13. Example skip_old instruction The example switches skip_old on for the system log (1) and the application log (2). skip_old is switched of for the security log (3). prefilt_in and prefilt_out instructions These instructions set pre-filters for each logfile. The association consist of a pair ; . Pre-filters are used to discard events before sending them to any other process. The in-filter specifies events that should not be discarded, the out-filter specifies events that should be discarded. Pre-filters are optionally. If no filter is set, all events are send to the target processes. 001 002 prefilt_in! ; {; ; } prefilt_out! ; {; ; } Figure 9-13. Format of prefilt_in and prefile_out instructions 001 002 prefilt_in!2;prefilt_nt_app.flt prefilt_out!3;prefilt_nt_sec.flt Example 9-14. Example prefilt_in and prefilt_out instructions This defines an in-filter for the application log (2) and an out-filter for the security log (3). The pre-filter files are text files, structured like this: • each line contains a list of assignments = • assignments are separated by semicolons: = ; = • several possible values for one key can be separated by a comma ( = , ) • a filter matches if any line matches • only events that match any prefilt_in and do not match any prefilt_out are sent to the filter process • if no pre-filter is set, all events are sent to the filter process Possible pre-filter keys are: eventid , eventtype and source 001 source=SNMP,Print; Example 9-15. Example for a pre-filter file to match all events from the SNMP and Print source: 57 Chapter 9. Component-specific configuration assoc instruction The assoc instruction associates an event log with a type and logical name. 001 ... assoc! ; ; {; ; ; } Figure 9-14. Format of assoc instruction 001 assoc!1;tec;nt_system;2;tec;nt_application;3;tec;nt_security Example 9-16. Example assoc instruction The parameter can be selected from values tec and v2s. tec represents the relation to logfiles which can be described as .fmt, v2s represents the logical link to complex data formats. The parameter can be selected from one of the format names defined under formatlist (see parameters of v2fmtfilt and tecfmtfilt). 001 assoc!1;tec;nt_system;2;v2;nt_application;3;tec;nt_security Example 9-17. Another example for assoc instruction using different primary types ntevtlogread command line parameters These parameters can be set in the field supplementary parameters parameter example description default value -E -E When opening an eventlogthe first time: skip all old events, send only new events disabled -e -e Skip all old events each time a eventlog is opened, send only new events disabled Sets the name of the host, ntevtlogread runs on. The hostname is requested by gethostname() function if this parameter is not given. use gethostname() Specifies the default character set to be used for reading files if no encoding is specified. UCS2-LE -H -H foo.bar.com -O -OUTF-16 58 Chapter 9. Component-specific configuration tecfmtfilt The T/EC format filter window (tecfmtfilt) is used for defining all secondary data formats, which are not multi-line (these can be described using a standard Tivoli .fmt file). The Settings of tecfmtfilt dialog Filter processing of data by component tecfmtfilt is based on Tivoli .fmt files. These are read in directly from format definitions (without prior compilation). For every format file, a logical secondary data type must be defined with the formatlist instruction. The secondary data type identifier should have any coherence with the format filename. The primary data type of the tecfmtfilt filter is always tec. tecfmtfilt specific parameters and their setting in the configuration file This is the configuration line created from the settings windows above: 59 Chapter 9. Component-specific configuration 001 ... ... ... tecfmtfilt=run!tecfmtfilt -P 11004,port!11004,targets!msgclsfsrv,formatl . ist!solaris_syslog;solaris_syslog.fmt;nt_system;nt_system.fmt;nt_applic . ation;nt_application.fmt;nt_security;nt_security.fmt,conf!port;run;targ . ets;formatlist Example 9-18. An example configuration line for tecfmtfilt formatlist instruction The formatlist instruction defines an association between secondary data types and Tivoli .fmt files which describe how to create events from the data stream. The association is taken from the GUI’s table. formatlist! ; {; ; } 001 ... Figure 9-15. Format of formatlist instruction formatlist!solaris_syslog;solaris_syslog.fmt;nt_system;nt_system.fmt;nt_ . application;nt_application.fmt;nt_security;nt_security.fmt 001 ... Example 9-19. Example formatlist instruction Any desired name can be used as a logical name (= secondary data type), e.g. aix4r1 for the format file tecad_logfile_aix4-r1.fmt The example defines the tecfmtfilt to process the four data types solaris_syslog, nt_system, nt_application and nt_security which are defined in the following format files: secondary data type format file name solaris_syslog solaris_syslog.fmt nt_system nt_system.fmt nt_application nt_application.fmt nt_security nt_security.fmt Note: Events which are assigned a classname starting with *DISCARD are discarded by this component. (This is an enhancement of the Tivoli adapter, which discards only events from the one class *DISCARD* ). Note: Since CALA 2.03 tec format files need to be saved in UTF-8 encoding if they contain Non-ASCII-127 characters. 60 Chapter 9. Component-specific configuration tecfmtfilt command line parameters These parameters can be set in the field supplementary parameters Parameter example description -Q -Q 2000000 Sets the size of the static buffer used for parsing. Increase this value if you get the message not enough quickmem in the debug file. default value 1048576 61 Chapter 9. Component-specific configuration v2fmtfilt The v2 format filter window (v2fmtfilt) is used for describing all complex data flows, which cannot be described with a conventional Tivoli .fmt file. This includes multi-line logfile formats or those formats which can only be described using complex expressions. The Settings of v2fmtfilt dialog In contrast to the standard Tivoli format descriptions, the CALA v2 format makes it possible to implement format descriptions of almost any level of complexity. v2fmtfilt specific parameters and their setting in the 62 Chapter 9. Component-specific configuration configuration file The configuration of v2fmtfilt is identical to the configuration of tecfmtfilt the difference is the format of the input files (which is Tivoli .fmt for tecfmtfilt and .v2s for v2fmtfilt). 001 ... v2fmtfilt=run!v2fmtfilt -P 11005,port!11005,targets!msgclsfsrv,formatlis . t!fn_log;fn_log.v2s,conf!port;run;targets;formatlist Example 9-20. An example configuration line for v2fmtfilt formatlist instruction The filter-specific parameter formatlist has already been described with filter tecfmtfilt .The difference with the formatlist definition for v2fmtfilt is the syntax the format files use. Format files used with the v2fmtfilt have to be in v2 format, while format files used with the tecfmtfilt have to be in Tivoli file format. 001 ... formatlist! ; {; ; } Figure 9-16. Format of formatlist instruction 001 formatlist!fn_log;fn_log.v2s Example 9-21. Example formatlist instruction The primary data type for the v2fmtfilt filter is always v2 . Note: The syntax of CALA v2 format files is explained in the appendix. Note: Events which are assigned a classname starting with *DISCARD are discarded by this component. Note: Since CALA 2.03 v2 format files need to be saved in UTF-8 encoding if they contain Non-ASCII-127 characters. v2fmtfilt command line parameters These parameter can be set in the field supplementary parameters Chapter 9. Component-specific configuration 63 Parameter example description default value -Q -Q 2000000 Sets the size of the static buffer used for parsing. Increase this value if you get the message not enough quickmem in the debug file. 1048576 -D -D disabled If this parameter is given, each event contains the following fields (which are taken from the events timestamp or from current time if no timestamp is given): • YEAR: 4 digits • MONTH:2 • DAY:2 digits digits • HOUR:2 digits • MINUTE:2 digits • SECOND:2 digits 64 Chapter 9. Component-specific configuration calamon calamon is CALA’s monitoring engine implementation. Note: There is a configuration GUI for command tables called Monitoring Manager , refer to the Monitoring Manager User’s Guide for more information of this product. The Settings of calamon window defines the name and the settings of the command table file which contains the parameters for the processes to be started. The Settings of calamon dialog There are two tables with parameters, which must all be set to a value (empty parameters are not allowed). The first table configures the scripts or programs to be started and their parameters. It contains the following columns: 65 Chapter 9. Component-specific configuration script name path and name of the script to be started command line parameters parameters which are passed to the script primary data type , secondary data type and event class type of event to be created stdout field FIR field to receive the script s output to stdout stderr field FIR field to receive the script s output to stderr return code field FIR field to receive return value of the script comment prefix prefix which marks a line of the scripts output as comment (e.g. #) comment field FIR field to receive comment lines (which are removed from stdout field) escalation field FIR field to receive escalation level (is set from escalation file) escalation file name of escalation file (see escalation table description below) the execution times specification The execution times specification is similar to the unix crontab, it uses the following columns execution months month may be given numeric (from 1 to 12) or as three letter appreviations (e.g. 11,Jan-Mar to allow execution in November, January, February and March) execution days of month e.g. 1,15 to allow scripts execution each 1st and fifteenth execution days of week the days of week are given as three letter appreviations or numeric values (0 to 7, 0 and 7 mean sunday). execution hours 24 lesson format, e.g. 23-1 to allow execution from 11 p.m. to 1 a.m. execution minutes minutes from 0 to 60 66 Chapter 9. Component-specific configuration execution seconds seconds from 0 to 60 execution period length of period in seconds or in format DD:HH:MM[:SS] (days:hours:minutes[:seconds]) message template a template for the message to be written into the message slot (may contain links to other fields) message slot slot the name of the message slot The script is run periodically within the specified execution times. The message template can contain links to other fields (e.g. the stdout and stderr fields). Links to other fields are indicated by writing the field name enclosed with < and >. 001 The disk is % full. You may run in difficulties. Example 9-22. An example message template The escalation settings are given in the lower table which contains the followings parameters: escalation file name of escalation file (the same escalation file may occur multiple times here, but should only occur in lines which are following each other) return code of script* the script s return code lower level output on stdout* lower limit of escalation level or alpha-numeric value of stdout upper level output on stdout upper limit of escalation level (used only for numeric values of lower/upper level output on stdout) lower level output on stderr* and upper level output on stderr see above, but output on stderr is used value of escalation field value the escalation field is set to 67 Chapter 9. Component-specific configuration action either DISCARD (no event is created), SEND (create an event and send it to the targets) or SENDFIRST (create an event only if the escalation value changes) Fields marked with a * may also contain wildcards with special meanings: * matches any or no output + matches any output ! matches no output When using non-numeric monitors, only the lower values are checked. The escalation files are checked top down, the first matching line is used. calamon specific parameters and their setting in the configuration file Because the configuration of calamon may be very complex, it is moved to calamon specific files. Therefore its configuration line in logctlsrv.conf is very simple: 001 ... ... calamon=run!calamon -P 11006,port!11006,targets!msgclsfsrv,cmdtab!cmdtab . .tbl,source!CALA,sub_source!cala_mon,cmdline_slot!cmdline,conf!port;run . ;targets;cmdtab Example 9-23. An example configuration line for calamon cmdtab instruction The cmdtab instruction sets the name of the calamon command table file. The filename is given in the GUI s entry field command table filename 001 cmdtab! Figure 9-17. Format of cmdtab instruction 001 cmdtab!cmdtab.tbl Example 9-24. Example cmdtab instruction Note: The format of the command table file is described in the appendix. 68 Chapter 9. Component-specific configuration source instruction The source instructions set s the value of the source slots of the created FIRs. If source is not configured, the source slot is set to FSM CALA . sub_source instruction Like the source instruction sets the value of the source slot, the sub_source instruction sets the value of the standard T/EC field sub_source . If not configured, sub_source is set to the logical name of the calamon process. cmdline instruction This instruction defines the name of the field to receive the command line (program + arguments) called for the monitor. If unset, this information is written into the slot cmdline . calamon command line parameters This parameter can be set in the field supplementary parameters parameter example -H -H foo.bar.com description default value Changes the hostname written into the events HOSTNAME field. tcp/ip hostname -T -T 120 Sets the timeout for monitor scripts/programs. 60 -O -OUTF-8 Specifies the default character set to be used for passing parameters to the monitor and to parse the input streams. the system default encoding Structure of FIRs created by calamon The following table shows the fields of a calamon created FIR: slot name value source from source instruction sub_source from sub_source instruction cmdline (configurable) command line (program + arguments) date monitors execution time hostname name of the host running calamon origin ip address of the host running calamon rc monitors return code Chapter 9. Component-specific configuration 69 slot name value stdout monitors output to stdout stderr monitors output to stderr comment monitors output to stdout lines beginning with comment prefix msg message generated from message template severity events severity according to escalation table The names of the fields rc, stdout, stderr, comment, severity and msg are defined in the command table. If stdout and stderr are written into the same slot, stderr is redirected to stdout . Events can also be suppress depending on their return code or output to stdout or stderr (see documentation of escalation table above). 70 Chapter 9. Component-specific configuration snmpread snmpread is the CALA component to receive SNMP traps and forward them as CALA events (FIRs). The Settings of snmpread dialog snmpread specific parameters and their setting in the configuration file This is the configuration line created from the settings window 001 002 snmpread=run!snmpread -P 11008,port!11008, targets!msgclsfsrv,type!tec;snmp, 71 Chapter 9. Component-specific configuration 003 class!SNMP_Event,conf!port;run;targets;type;class Example 9-25. An example configuration line for snmpread type instruction The type instruction specifies the primary and secondary data type assigned to received events. Its values are taken from the primary data type and secondary data type entry fields. (Either both or none has to be given.) 001 type! ; Figure 9-18. Format of type instruction 001 type!tec;snmp Example 9-26. Example type instruction This defines received SNMP events to be of primary type tec and of secondary type snmp. The type instruction is optional. If it is not given, the default values for primary (cala) and secondary type (snmpreader) are used. class instruction This instruction defines the class received SNMP events will get. This value is taken from the entry field Name of class . This instruction is optional, if it is not given (no value in entry field), the default class CALA_SNMP is used. 001 class! Figure 9-19. Format of class instruction 001 class!SNMP_Event Example 9-27. Example class instruction prefilt_in and prefilt_out instructions These instructions sets pre-filters for the SNMP reader. Pre-filters are used to discard events before sending them to any other process. The in-filter specifies events that should not be discarded, the out-filter specifies events that should be discarded. Pre-filters are optionally. If no filter is set, all events are send to the target processes. 72 Chapter 9. Component-specific configuration 001 002 prefilt_in! prefilt_out! Figure 9-20. Format of prefilt_in and prefilt_out instructions 001 002 prefilt_in!snmp_in.flt prefilt_out!snmp_out.flt Example 9-28. Example prefilt_in and prefilt_out instructions The pre-filter keys are the received SNMP fields (description see below). For further information about pre-filters, refer to the pre-filter section in the ntevtlogread description. Pre-filters are optional. Every event is sent to the targets if no pre-filter is set. Snmpread command line parameters These parameter can be set in the field supplementary parameters parameter example description default value -p -p 20015 Sets the tcp port on which to listen for SNMP events. (This parameter can be set with the CALA-GUI entry field Snmp port) 162 (the default SNMP port) Snmpread generated Events Events (FIRs) generated by the CALA component snmpread process contain the following fields: field name description ORIGIP IP address of trap sender (IP address or resolved hostname) SENDOID OID of sending system COMMUNITY community string TRAPTYPE trap type SPECTYPE spectype OID OID of Variable TYPE Type of Variable Possible values are: ASN_OCTET_STRING, ASN_INTEGER and ASN_BIT_STRING VALUE Value of Variable is the number of the SNMP event variable. These variables are numbered starting from 1. 73 Chapter 9. Component-specific configuration E.g. the first variable OID will be stored in the field OID1, while the TYPE of the third variable will be stored in field TYPE3. 74 Chapter 9. Component-specific configuration mssqlread and oracleread The MS SQL and ORACLE readers are designed for reading log entries from a database. The configuration of these readers is completely identical. The mssqlread is used to read events from a MS SQL database (MS SQL-Server >= 7) and is only available for the Microsoft Windows platforms. For reading ORACLE databases , the oracleread can be used. It is available for Windows 2000/2003/Xp Professional, AIX and Solaris. It supports oracle databases with version 8.0 or higher. The Settings of mssqlread dialog For each table to be read, a database log type has to be defined. When using the CALA configuration GUI, each database log type is displayed as a row in the table on the bottom of the window. There are several parameters to be set for each database log type, those marked with an asterisk (*) are mandatory. mssqlread/oracleread specific parameters and their setting in the configuration file These are the configuration lines created from the settings window: 001 ... 002 mssqlread=run!mssqlread -P 11011 -AB 127.0.0.1,port!11011,targets!msgcls . fsrv, db_log_types!log_audit_fndsdb,conf!port;run;targets;db_log_types 75 Chapter 9. Component-specific configuration 003 004 ... 005 ... 006 ... 007 ... 008 log_audit_fndsdb=table!AUDIT_LOG,database!fndsdb,db_entry_id!AL_DATETIM E;ASCE, map!AL_PROCESSID;pid;AL_STATUS;AL_STATUS;AL_WORKSTATN_ADDR;workstation; AL_EVENT_PARAM1;msg; AL_EVENT_PARAM2;PARAM2;AL_EVENT_PARAM3;PARAM3;AL_EVENT_PARAM4;PARAM4;AL _USER;USER, copy_unmapped!0,timestamp!AL_DATETIME,type!tec;FNDS_MSSQL,defaultclass! FNDS_AUDITLOG_Error, classmap!ds_class.map;AL_EVENT_ID,pollinterval!30 . . . . Figure 9-21. Some example configuration lines for mssqlread (identical for oracleread) db_log_types instruction The db_log_types instruction contains a list of all configured database log types to be monitored by this server. Each db_log_type needs to be configured in a own line. 001 db_log_types! {; } Figure 9-22. Format of db_log_types instruction 001 db_log_types!log_audit_fndsdb Example 9-29. Example db_log_types instrcution This defines the database log type labeled log_audit_fndsdb. The following instructions are set in the database log type configuration line: dbuser instruction The dbuser instruction gives a database user and password to be used when connection to the database. The password is encrypted and can only be configured using the CALA configuration GUI. 001 dbuser! [; ] Figure 9-23. Format of dbuser instruction 001 dbuser!tec;11161219140600 Example 9-30. Example dbuser instruction The dbuser parameter is optional. If no user is specified, the reader tries to connect the database without any user and password (system user authentification). 76 Chapter 9. Component-specific configuration database instruction This instruction defines to database to be used. 001 database! Figure 9-24. Format of database instruction 001 database!fndsdb Example 9-31. Example database instruction table instruction The name of the database table is given with this instruction. 001 table! Figure 9-25. Format of table instruction 001 table!AUDIT_LOG Example 9-32. Example table instruction db_entry_id instruction The db_entry_id instructions sets the table column which is used for entry identification and if the order. The values in this field must be unique for each entry and they must either be descending or ascending. The first time the SQL reader is started, it must read the whole table to find the newest entry (the one with the highest or lowest value in the id-field). To find new events, the following SQL statement is used (line 001 shows the statement used for descending, line 002 the statement used for ascending tables): 001 002 select * from table where id-field < id-of-last-entry select * from table where id-field > id-of-last-entry Figure 9-26. SQL statement used to find events 77 Chapter 9. Component-specific configuration 001 db_entry_id! ;[DESC|ASCE] Figure 9-27. Format of db_entry_id instruction 001 db_entry_id!AL_DATETIME;ASCE Example 9-33. Example db_entry_id instruction map instruction To map database field names to event slot names, the map instruction is used. It takes pairs of parameters, each consisting of the database field name and the FIR field name. 001 map! ; {; ; } Figure 9-28. Format of map instruction 001 map!AL_PROCESSID;pid;AL_STATUS;AL_STATUS;AL_WORKSTATN_ADDR;workstation Example 9-34. Example map instruction This parameter is optional. If it is not set, the database field names are used as FIR field names. copy_unmapped instruction The copy_unmapped instruction is a boolean instruction which only takes one of the values 0 or 1. It defines whether or not mapped database fields (fields, which names do not have a map entry, see above) should be copied into the resulting FIR (FIR field name = database field name) or should be discarded. 001 copy_unmapped![0|1] Figure 9-29. Format of copy_unmapped instrucion 001 copy_unmapped!0 Example 9-35. Example copy_unmapped instrucion The copy_unmapped parameter is optional, it s default value is 1. 78 Chapter 9. Component-specific configuration defaultclass instruction This instruction sets the default class to be used if no class mapping is defined or no matching class is found in the classmap file. 001 defaultclass! Figure 9-30. Format of defaultclass instruction 001 defaultclass!FNDS_AUDITLOG_Error Example 9-36. Example defaultclass instruction This instruction is optional, if it isn’t set, the default class is set to MSSQLREAD_Base (mssqlread) or ORACLE_Base (oracleread). classmap instruction The classmap instruction takes two arguments: a filename and a database field. The classmap file is an ASCII file containing one mapping instruction per line. A mapping consists of the database fields value and the class name separated by white spaces. 001 classmap! ; Figure 9-31. Format of classmap instruction 001 classmap!ds_class.map;AL_EVENT_ID Example 9-37. Example classmap instrucion 001 LOGON APP_Logon Example 9-38. An example mapfile mapping the class of the created FIR to APP_Logon if the map field’s value is LOGON: The classmap instruction is optional. The default class (given with the default class instruction) is used if no configuration is given. type instruction The type instruction specifies the primary and secondary data type assigned to created events. Its values are taken from the primary data type and secondary data type entry fields. (Either both or none has to be given.) 79 Chapter 9. Component-specific configuration 001 type! ; Figure 9-32. Format of type instruction 001 type!tec;FNDS_MSSQL Example 9-39. Example type instruction The type instruction is optional. If it is not given, the default values for primary (tec) and secondary type (mssql/oracle) are used. prefilt_in and prefilt_out instructions These instructions sets pre-filters for each database logfile. Pre-filters are used to discard events before sending them to any other process. The in-filter specifies events that should not be discarded, the out-filter specifies events that should be discarded. Pre-filters are optionally. If no filter is set, all events are send to the target processes. 001 002 prefilt_in! prefilt_out! Figure 9-33. Format of prefilt_in and prefilt_out instructions 001 002 prefilt_in!sql_in.flt prefilt_out!sql_out.flt Example 9-40. Example prefilt_in and prefilt_out instructions The pre-filter keys are the database fields. For further information about pre-filters, refer to the pre-filter section in the ntevtlogread description. Pre-filters are optional. Every event is sent to the targets if no pre-filter is set. timestamp instruction There are two possibilities to use the timestamp expression. • If the database table contains a numerical timestamp field or one from any date or time type, only the fieldname has to be given to this instruction. • If the timestamp is split into several fields or is given within any text field, a extended usage of timestamp is needed. The field names and text position of the date parts have to be given. 80 Chapter 9. Component-specific configuration 001 002 ... timestamp! timestamp! ; ; {; ;< . db-field>; } Figure 9-34. Format of timestamp instruction date-part-id can be one of $YEAR, $MONTH, $DAY, $HOUR, $MINUTE and $SECOND. The text position argument is given similar to the text position in message map description, see for details. 001 002 003 ... timestamp!date timestamp!AL_DATETIME timestamp!datestr;$YEAR;F0L3;datestr;$MONTH;F4L5;datestr;$DAY;F6L7;time . str;$HOUR;F0L1;timestr;$MINUTE;F2L3;timestr;$SECOND;F4L5 Example 9-41. Examples for the timestamp instruction The second sample is a definition for a table having two columns for the timestamp: the datestr column containing the date in the format YYYYMMDD and timestr column with the time in the format HHMMSS. (E.g.: datestr= 20020415 and timestr=084933 for 8:49:33 on april 15th 2002) mssqlread and oracleread command line parameters These parameter can be set in the field supplementary parameters parameter example description -D -D Sets the name of a remote databserver. . foo.bar.com -H -H Changes the value of the hostname field in the thud.grunt.netcreated events. default value localhost tcp/ip hostname -L -L 60 Set the timeout for database login (in seconds). 60 -E -E Skip old events: Discard all events found when opening the database the first time. disabled 81 Chapter 9. Component-specific configuration jdbcread The jdbc reader implements a generic database reader component similar to mssqlread and oracleread. It uses the java jdbc interface to access the database and can therefore be used to access any database for which a jdbc driver is available. jdbcread specific parameters and their setting in the configuration file The configuration of the jdbcread is very similar to the configuration of mssqlread and oracleread. There are few differences: • Additional command line parameters for the java virtual machine (e.g. classpath settings) can be passed before the component parameters are given. This parameters must at least add the calaJNI.jar file to the classpath. End the java parameters with two dashes. • The database name consists of three parts: • the JDBCRead java classname (de/cenit/eb/sm/cala/jdbc/reader/DefaultCalaJDBCRead by default) • the jdcb driver class • the jdbc database URL These parts are separated by colons. 001 ... de/cenit/eb/sm/cala/jdbc/reader/DefaultCalaJDBCRead:com.mysql.jdbc.Drive . r:jdbc:mysql://localhost/fndsdb Example 9-42. An example database string for jdbcread 001 ... ... 002 003 ... ... ... ... ... ... jdbcread=run!jdbcread -Djava.class.path=calaJNI.jar;mysql-connector.jar . -- -P 11011 -AB 127.0.0.1,port!11011,targets!jdncread,db_log_types!log_ . audit_fndsdb,conf!port;run;targets;db_log_types log_audit_fndsdb=table!AUDIT_LOG,database!de/cenit/eb/sm/cala/jdbc/read er/DefaultCalaJDBCRead:com.mysql.jdbc.Driver:jdbc:mysql://localhost/fnd sdb,db_entry_id!AL_DATETIME;ASCE,map!AL_PROCESSID;pid;AL_STATUS;AL_STAT US;AL_WORKSTATN_ADDR;workstation;AL_EVENT_PARAM1;msg;AL_EVENT_PARAM2;PA RAM2;AL_EVENT_PARAM3;PARAM3;AL_EVENT_PARAM4;PARAM4;AL_USER;USER,copy_un mapped!0,timestamp!AL_DATETIME,type!tec;FNDS_MSSQL,defaultclass!FNDS_AU DITLOG_Error,classmap!ds_class.map;AL_EVENT_ID,pollinterval!30 . . . . . . Example 9-43. An example jdbread configuration using a mysql connector for connection to the local database fndsdb 82 Chapter 9. Component-specific configuration The jdbc reader needs a java 1.4 or higher virtual machine to be in the library path. For further configuration information refer to section mssqlread and oracleread. 83 Chapter 9. Component-specific configuration msgclsfsrv The processing server msgclsfsrv is the core component of CALA. Due to its central function it features some specific configuration parameters, which are explained in the following chapter. To provide better understanding of these causal relationships, a few of the key terms and their functions are explained first. Comment: Several msgclsfsrv processes can be implemented in parallel manner on a system. Since version 2.1, this no longer requires a unique name for the program binary. Definition MessageMap File General: The value/information defined in a Message Map file is used to manipulate/process events (FIRs). This means that a kind of linear control function is implemented with the Message Map definitions. The structure of a Message Map file is defined with a Message Classification Type (MCT), i.e. the format is not firmly specified. Definition RulesMap File General: A Rules Map file is a description of rules for event handling. Rules are used to handle correlations between correlating events and to perform actions/modifications on events in dependency of previous by received events. The structure of a Rules Map file is defined with a Rules Map Type (RMT), i.e. the format is not firmly specified. The large number of possible parameters available for the Message Classification Server (msgclsfsrv) configuration makes it necessary to have a hierarchical structure in the configuration window. 84 Chapter 9. Component-specific configuration The basic msgclsfsrv window The Settings of msgclsfsrv dialog General settings for msgclsfsrv, such as logical name and ports can be configured in the basic window.Further configuration can be set in the sub-windows. The Message Map Types window In order to set up MCT definitions (Message Classification Types), the appropriate field must be selected. At this point, the Message Classification Type (MCT) window opens. 85 Chapter 9. Component-specific configuration The MessageMap Types dialog Definition of MessageMap Classification Type (MCT) A message map classification type (MCT) is a logical unit for message mapping. It is used to process data streams (specified by primary and secondary data type) for a list of emitters. All configured MCTs are working parallel, this means, that each incoming event is put into every MCT which is defined for its stream. The MCTs will process the event and send it to the emitters defined on it. If no MCT is found for an event, the event is not modified and sent to all targets of msgclsfsrv. A MCT definition can use one or more message map files which are given in the table Corresponding message maps . For more information about message map files see The Message Map definition window. MCT parameters and their setting in the configuration file Each MCT is configured in a line of its own within the configuration file logctlsrv.conf . The message classification servers configuration line only contains references to the MCT definitions. 86 Chapter 9. Component-specific configuration 001 ... ... 002 ... msgclsfsrv=run!msgclsfsrv -P 11010,port!11010,targets!tecfmtemit;cmdemit . ; calaproxy;reportemit;snmpemit;smtpemit,types!map_sev,conf!port;run; t . argets;types map_sev=type!v2;fn_log,handledby!calaproxy;tecfmtemit,msgmaps!fn_severi . ty.map;fn_severity,eventframe!7200,dupekey!$CLASS;L;ORIGIN;L Example 9-44. These are the configuration lines created for the message map classification type: types instruction The types instruction contains a list of all message map classification types to be used by the message classification server 001 types! {; } Figure 9-35. Format of types instruction 001 types!map_sev Example 9-45. Example types instruction MCT configuration line 001 ... ... ... =type! ,handledby! {; . }, msgmaps! {; ; },eventframe! , dupekey! {; } Figure 9-36. Format of mct configuration line 001 ... map_sev=type!v2;fn_log,handledby!calaproxy;tecfmtemit,msgmaps!fn_severit . y.map;fn_severity,eventframe!7200,dupekey!$CLASS;L;ORIGIN;L Example 9-46. Example mct configurationline 87 Chapter 9. Component-specific configuration MCT configuration parameters type instruction This describes the data type combination (primary / secondary) which is applied to this MCT definition. These parameters are set by the entry fields Primary data type and Secondary data type 001 type! ; Figure 9-37. Format of mct type instruction 001 type!v2;fn_log Example 9-47. Example mct type instruction handledby instruction The handledby instruction describes the following component(s) which contain(s) results of this type as a target. If no handledby parameter is defined, all FIRs are propagated to all defined emitters. The emitters for the MCT are taken from the textbox Emitter 001 handledby! {; } Figure 9-38. Format of mct handledby instruction 001 handledby!calaproxy;tecfmtemit Example 9-48. Example mct handledby instruction Explanation: The FIRs of this type are propagated to calaproxy and tecfmtemit . msgmaps instruction The msgmaps instruction gives a list of message map file to be used from this MCT. Each message map file must be specified by its filename and a logical identifier, which is used to define the file s format later. In the settings window the message map files are set in the table corresponding message maps 001 ... msgmaps! ; {; ; } Figure 9-39. Format of mct msgmaps instruction 88 Chapter 9. Component-specific configuration 001 msgmaps!fn_severity.map;fn_severity Example 9-49. Example mct msgmaps instruction Explanation: The above example defines the Message Map fn_severity whose Message Map data are stored in the fn_severity.map file. eventframe instruction The event frame describes a time frame. Events received during this time frame are checked for duplicates. The default value is 3600 seconds = 1hr. Event frame settings can be indicated separately for every MCT (Message Classification Type). If no event frame is given, or event frame is set to 0 seconds, duplicate detection is disabled. 001 eventframe! Figure 9-40. Format of mct eventframe instruction 001 eventframe!7200 Example 9-50. Example mct eventframe instruction Example explanation: This event frame setting defines a time interval of 2 hours (7200 seconds) for duplicate detection purposes. dupekey instruction A dupekey is a key which is used to recognize duplicate events. It is created from one or more parts of an event. The event fields used within this key are defined in the table Duplicate keys (dupekey definition) 001 dupekey! ; {; ; } Figure 9-41. Format of mct dupekey instruction 001 dupekey!$CLASS;L;ORIGIN;L Example 9-51. Example dupekey instruction The dupekey instruction defines the field names (slots) which should be used for duplication detection purposes. Sub-parameter text-position defines which part of the key field should be used for duplication detection purposes.Further information on the text position can be found in chapter msgclsfsrv Text Formatting. The example defines a combination of internal slot $CLASS and the slot ORIGIN as a duplication detection key. L means the entire field contents (starting from the left) are used. 89 Chapter 9. Component-specific configuration If the dupekey is omitted, duplicate detection is disabled unless the $DUPEKEY field is mapped in the FIR during the classification process. In this case, the key referenced by the $DUPEKEY is used. This must be entered in the configuration list and in the AUXKEYS list (see Auxkeys definition window). 001 ... mqlogmct=type!v2;MQ,handledby!tecfmtemit,msgmaps!suppressed.map;suppress . map1;mq.map;mqmap1;severity.map;redefsevmap1,dupekey!errcode;L;msg;L Example 9-52. Another example of a complete MCT definition Explanation of the example: • One MCT definition is defined with the name mqlogmct • Primary type is v2, secondary type is MQ. • Data is sent to tecfmtemit for further processing. (handleby sub-parameter) • The Message Map definitions suppressmap1 (described by the file suppressed.map), mqmap1 (described by the file mq.map) and redefsefmap1 (described by the file severity.map). • The slots errcode and msg are used for this MCT type for duplicate detection and designator designates the use of the complete string (from left) for duplicate detection 90 Chapter 9. Component-specific configuration The Message Map definition window Once the MCT definition window has been closed by pressing the OK button, the Message Map file relating to the MCT must be defined. This configuration is performed using sub-menu Message maps The message map definition dialog This window defines the format of the message map files. To edit the message map file contents, press the Edit button. Message Map parameters and their setting in the configuration file A format description must exist for every Message Map file stored in the configuration file logctlsrv.conf . 001 ... =key! ; {; ; }, fields! {; } Figure 9-42. Format of message map definition 001 fn_severity=key!severity;L,fields!severity Example 9-53. Example message map definition The slot names following the keyword key (at least 1 slot) are related to the MessageMap assignment. The slot names which follow the keyword fields are mapped using the values for the relevant line/column.This means that all slot names can be indicated as key and as field as well. Explanation of the example 91 Chapter 9. Component-specific configuration The format of the Message Map definition with the name fn_severity contains the key severity and the severity field is also used as a mapable field. Note: This Message Map definition is used for implementing non-Tivoli severity values on severity values used by Tivoli. The file which describes this format possesses has following format: 001 002 003 004 005 006 INFO HARMLESS WARN WARNING ERROR CRITICAL Error CRITICAL Notice HARMLESS Warning WARNING Example 9-54. An example mapfile for the above map definition The content of the slot severity for all events manipulated with this Message Map is processed in accordance with this conversion table. E.g.: If the severity field on an event being processed contains the value Notice , the contents of the severity slot will be mapped to HARMLESS after processing. The CALA-processing server defines a few internal slots which are required for internal processing, but which can also be forwarded as external slots (to the T/EC). The most important internal slot is $CLASS which represents the event class name of the T/EC ( Class ). Default Mapping If the map file contains a line with the special string __DEFAULT_KEY__ as key, the mapping defined in this line is used for events which do not match for other mappings. Note: The default entry can be used anywhere in the map file, if several occurences are found, the latest one is used. Deleting slots Event fields can be deleted by setting it s value to *. To delete a complete event, map it s class to **. This works with message maps, rules maps and re-mapping. Special slots for duplicate detection To implement duplicate detection, 5 reserved internal field names (internal slots) are implemented, $SEVFLD, $ESCAT, $ESCLEV, $CLASS and $DUPEKEY. These slots have the following meaning: Chapter 9. Component-specific configuration 92 field name description $SEVFLD Describes the name of the field which implements the severity field. $ESCAT Describes the number of identical events which are suppressed using duplicate detection. $ESCLEV Describes the value of the severity level $ESCAT for identical events and escalates the $ESCAT+1 event to this level. $CLASS The class of the event (FIR) $DUPEKEY Name of a duplicate key, see also section Auxkeys definition window $ESCCNT Escalation counter holds the no. of escalations that occurred (can be used in further processing). Another example for a complete message map definition 001 mqmap1=key!errcode;L,fields!$CLASS;$SEVFLD;severity;$ESCAT;$ESCLEV Example 9-55. Another example for a complete message map definition Explanation of the example • The Map Definition mqmap1 message uses the errcode field for duplicate detection. The entire contents of this field are considered (text position L) • The first field in the field instruction $CLASS (internal slot, see above) is mapped with this Message Map in relation to the errcodes (key field is errcode ). • The second field, $SEVFLD, describes the name of the severity field (the severity field can be any field within the FIR). • The third field is an assignment for the field severity. In this example the field severity is also used for escalation, so the mapping is overwritten if an escalation occurs. • The fourth field, $ESCAT, contains the number of identical events (identical in terms of the defined dupekey fields in the MCT definition) which have to be suppressed. • The last defined field $ESCLEV defines the value of the field with which the field described in $ESCFLD is mapped if an escalation occurs. An escalation occurs, if (within one event frame) more than in $ESCAT defined events of the same type arrived. 001 AMQ9001 AMQ_CALA_AMQ9001 severity MINOR 10 CRITICAL Figure 9-43. Excerpt from the related message map file Using this message map, events are processed as follows: • If a FIR is received, whose errcode slot contains the value AMQ9001, the Eventclass ($CLASS slot) is mapped to AMQ_CALA_AMQ9001. 93 Chapter 9. Component-specific configuration • The field which describes the severity is called severity (T/EC standard). • Default severity is mapped to MINOR • The 2 nd to 9 th event with errcode set to AMQ9001 will be suppressed. • If 10 events are identified as identical within the defined period of time (refer to event frame instruction), the severity level is increased to CRITICAL and a new event is initiated (FIR). 94 Chapter 9. Component-specific configuration Operations on FIR fields per Message Maps A normal message map contains just strings, which were assigned to the specified FIR fields. For some applications, it may be useful to do some arithmetic operations on fields. If the mapping value has one of the following pre- or postfixes, a special action is performed. Prefix/Postfix Action + string concatenation ++ addition -- subtraction ** multiplication // division All these operations can be used as postfix or prefix. Usage as prefix (e.g. ++5) means while the operator given as postfix means E.g. if the value of the field counter is 5, and the mapping value for this field is --3, the resulting value will be 2 (= 5-3). If the mapping value is 8--, the result will be 3 (= 8-5). All operators except the string concatenation operator (+), only work with numeric values. If any operation value is not numeric or a division by zero occurs, the field will be set to the string NaN (Not a Number). Instead of provoding a fixed value, it is also possible to give a reference to any other field of the FIR. A reference is given with the prefix & followed by the field’s name. If no operation is defined, the reference just copies the value of the referenced field. If any of these operators appears in a string, which should be assigned to the field, this string must be written in quotes. If a quote appears in a quoted string, it has to be escaped with backslashes (\). Some examples: • ++5 increases the field’s value by 5. • 5++ increases the field’s value by 5 (same as above). • --1 decreases the field’s value by 1. • 10-- sets the field’s value to 10 minus its old value. • 2** doubles the field’s value. • ++&count increases the field by the value stored in the field named count. • &count is replaced by the value of the field count. • PREFIX+ adds the string PREFIX to the beginning of the field’s value. • +POSTFIX appends the string POSTFIX to the end of the field’s value. • "--&count" sets the field’s value to the value of the count field decreased by one. Chapter 9. Component-specific configuration 95 The Rules definition window The rules definition dialog Definition of Rules Map Type (RMT) A rules map type is a logical unit to handle correlation between events. Like message map types, rules map types are used to process data streams for one or more emitters, but a RMT can also handle several data streams having different primary and secondary data types. RMT parameters and their setting in the configuration file The configuration of rules map types is similar to configuration of message map types. This is the configuration line of the example above: 001 ... ... 002 msgclsfsrv=run!msgclsfsrv -P 11010,port!11010,targets!tecfmtemit;cmdemit . ;calaproxy;reportemit;snmpemit;smtpemit,types!map_sev,rules!test_rule,c . onf!port;run;targets;types;rules test_rule=for!reportemit,type!tec;cala_test,corrkey!$CLASS;L;sub_source . 96 ... ;L,rulesmaps!tr_map.rmp;tr_map Example 9-56. An example msgclsfsrv configuration with a rules map type definition rules instruction The rules instruction lists the RMTs used by the message classification server. 001 rules! {; } Figure 9-44. Format of rmt rules instruction 001 rules!test_rule Example 9-57. Example Format of rmt rules instruction RMT configuration line 001 ... ... ... =for! {; },type! ; {; ; },rulesmaps! ; {;rulesmap filename>; }, corrkey! ;