Sensor Settings > Logging > Manual Setup (Config File)

The EtherSensor Watcher service configuration is stored in the watcher.xml file, located in the common configuration directory, Microolap EtherSensor [INSTALLDIR]\config.

A sample watcher.xml configuration file:

<?xml version="1.0" encoding="utf-8"?>
<WatcherConfig version="4.1">
<Syslog>
<LogRule
output="file://icap.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel
name="ICAP"
loglevels="all" />
</LogRule>
<LogRule
output="file://icap.err.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel
name="ICAP"
loglevels="error, warning, criterr" />
</LogRule>
<LogRule output="file://ethcap.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="ETHCAP"
loglevels="all" />
</LogRule>
<LogRule output="file://ethcap.err.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="ETHCAP"
loglevels="error, warning, criterr" />
</LogRule>
<LogRule output="file://caperrors.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="CAPERR"
loglevels="all" />
</LogRule>
<LogRule output="file://capture.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="CAPMAIN"
loglevels="all" />
</LogRule>
<LogRule output="file://analyser.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="ANALYSER"
loglevels="all" />
</LogRule>
<LogRule output="file://analyser.err.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="ANALYSER"
loglevels="error, warning, criterr" />
</LogRule>
<LogRule output="file://filter.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="FILTER"
loglevels="all" />
</LogRule>
<LogRule output="file://transfer.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="TRANSFER"
loglevels="all" />
</LogRule>
<LogRule output="file://transfer.err.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="TRANSFER"
loglevels="error, warning, criterr" />
</LogRule>
<LogRule output="file://watcher.log"
maxsize="10Mb"
encoding="utf-8" endline="CR,LF">
<Channel name="WATCHER"
loglevels="all" />
</LogRule>
<LogRule output="file://watcher.err.log"
maxsize="10Mb"
encoding="utf-8"
endline="CR,LF">
<Channel name="WATCHER"
loglevels="error, warning, criterr" />
</LogRule>
<LogRule output="file://squid-access.log"
maxsize="10Mb"
encoding="us-ascii"
endline="LF">
<Channel name="SQUID-ACCESS"
loglevels="all" />
</LogRule>
</Syslog>
<Statistics daysnumber="7">
<Detection>
<Sessions>true</Sessions>
<Hosts>true</Hosts>
<Detectors>true</Detectors>
<Messages>false</Messages>
<Squeries>false</Squeries>
<TransportProfiles>true</TransportProfiles>
</Detection>
<Counters>
<Performance>true</Performance>
<Icap>true</Icap>
<Interfaces>true</Interfaces>
<Parsers>true</Parsers>
<Cache>true</Cache>
<Analysers>true</Analysers>
<Filters>true</Filters>
<Quotas>true</Quotas>
<Transports>true</Transports>
</Counters>
</Statistics>
</WatcherConfig>

Description of tags used in the watcher.xml configuration file:

WatcherConfig tag

This is the root tag of the service configuration. The version attribute specifies the configuration version.

Syslog tag

This is the root tag for configuring access to syslog servers.

LogRule tag

Defines log file configuration. The output attribute specifies the log file name, and the maxsize attribute specifies the maximum file size. When the maximum file size is reached, the log file is moved to the backup directory and a new log file with the same name and the same parameters is created. The encoding attribute specifies the encoding in which messages are to be saved, and the endline attribute specifies the format for defining the end of a line in a message.

Channel tag

The Channel tag is nested within the LogRule tag. It contains the name of the service message channel, messages from which are to be saved in a file named output.

The channel is a label that indicates a message belongs to a service. Sometimes it also provides details on the service module that was running when the message appeared.

The name attribute of the Channel tag specifies the name of the channel. In the current version of Microolap EtherSensor (5.1.0.13519), the following channel names are available:

For the EtherSensor EtherCAP service:

ETHCAP
The main message channel for the service

CAPERR
The channel for traffic analysis error messages

CAPMAIN
The channel for messages from the connections being processed.

For the EtherSensor ICAP service:

ICAP
The main message channel for the service

ICAP-REQUEST
The channel for logging HTTP requests (responses) received from ICAP clients

For the EtherSensor LotusTXN service:

LOTUSTXN
The main message channel for the service

For the EtherSensor Analyser service:

ANALYSER
The main message channel for the service

FILTER
The channel for messages from the service data filtering service EtherSensor Analyser

For the EtherSensor Transfer service:

TRANSFER
The main message channel for the service

For the EtherSensor Watcher service:

WATCHER
The main message channel for the service

If EtherSensor Watcher has received a message with a channel name that is not defined in the configuration, the message will be saved to unknown.log.

The loglevel attribute specifies levels for messages that may be logged in the file.

Available values:

all
All messages are logged

criterr
Critical errors

error
Non-critical errors/external resource (a file, a connection) unavailability

warning
Warnings: invalid data formats, exceeding quotas, etc.

info
Regular messages issued during normal operation

debug
Debug messages

detdebug
Detailed debugging messages

DbChangeInterval tag

Defines the interval (in minutes) for the rotation of the database files of Microolap EtherSensor work counters.

Statistics tag

This is the root tag to configure the gathering of statistics. The daysnumber attribute of this tag specifies a number of days over which statistics will be gathered. Possible values are between 0 and 62 days. 0 means that the gathering of statistics has been suspended.

Detection tag

The Detection tag is nested within the Statistics tag. It contains the settings for gathering statistics on message detection results.

Sessions tag

The Sessions tag is nested within the Statistics tag. It contains the flag for gathering statistics on TCP connections. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\sessions directory and includes the following parameters for the connections being monitored:

▪the MAC address of the interface where the connection has been captured;

▪the connection creation and termination time;

▪IP addresses and ports of the connection;

▪the amount of data send from the client to the server and vice versa, the protocol used over TCP/IP (HTTP, ICQ, SMTP, POP3).

Hosts tag

The Hosts tag is nested within the Statistics tag. It contains the flag for gathering statistics on HTTP connections. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\hosts directories and includes the following parameters for the connections being monitored:

▪the MAC address of the interface where the connection has been captured;

▪IP addresses and ports of the connection;

▪the DNS name of the server to which the connection has been established.

Detectors tag

The Detectors tag is nested within the Statistics tag. It contains the flag for gathering statistics on the results of message detectors operation. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\detectors directories and includes the following parameters:

▪Event timestamp.

▪Detector name.

▪Detection status.

▪The number of messages detected.

Messages tag

The Messages tag is nested within the Statistics tag. It contains the flag for gathering message metadata. Such statistics are gathered in an XML file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\messages directories and includes the captured messages metadata:

▪Message service headers sent according to the protocol used.

▪Metadata created by Microolap EtherSensor during message processing (X-Sensor-... headers).

▪ From, To, Cc, Bcc, Subject headers.

Squeries tag

The Squeries tag is nested within the Statistics tag. It contains the flag for gathering search requests. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\squeries directories and includes the following parameters:

▪Event timestamp.

▪IP address and port of the search request sender.

▪the DNS name of the search service.

▪Search request phrase.

TransportProfiles tag

The TransportProfiles tag is nested within the Statistics tag. It contains the flag for gathering statistics on the results of delivery profiles operation. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\transports directories and includes the following parameters:

▪Event timestamp.

▪Delivery profile name.

▪Protocol used to send the message.

▪Message sending status.

Counters tag

The Counters tag is nested within the Statistics tag. It contains the settings for gathering Microolap EtherSensor performance counter values.

Performance tag

The Performance tag is nested within the Counters tag. It contains the flag for gathering the machine resource usage counter values. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\performance directories and includes the following parameters:

▪Event timestamp.

▪CPU usage (current, average and peak values).

▪Memory usage (current, average and peak values).

▪System thread usage.

▪System object (files, events, etc.) descriptor usage.

▪General OS load indicators (CPU, memory).

Icap tag

The Icap tag is nested within the Counters tag. It contains the flag for gathering ICAP server counter values. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\icap directories and includes the values of the EtherSensor ICAP service counters:

▪Event timestamp.

▪The number of connections to the ICAP server.

▪The number of requests (GET, POST, PUT) processed.

Interfaces tag

The Interfaces tag is nested within the Counters tag. It contains the flag for gathering the counter values for traffic processing on interface adapters. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\interfaces directories and includes the following parameters:

▪Event timestamp.

▪Processed packet counter.

▪Counters of processed TCP connections.

Parsers tag

The Parsers tag is nested within the Counters tag. It contains the flag for gathering the counter values for detected connections by protocol. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\parsers directories and includes the following parameters:

▪Event timestamp.

▪Counters of processed TCP connections by protocol (SMTP, POP3, HTTP, FTP).

Cache tag

The Cache tag is nested within the Counters tag. It contains the flag for gathering the counter values for the analyzer cache. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\cache directories and includes the following parameters:

▪Event timestamp.

▪Processed analyzer cache object counters.

Analysers tag

The Analysers tag is nested within the Counters tag. It contains the flag for gathering the counter values for detected messages. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\analyser directories and includes the following parameters:

▪Event timestamp.

▪Analyzer message detection counters.

Filters tag

The Filters tag is nested within the Counters tag. It contains the flag for gathering the counter values for the HTTP request filter and the message filter. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\filters directories and includes the following parameters:

▪Event timestamp.

▪Analyzer filters counters (HTTP request RAW filter, message filter).

Quotas tag

The Quotas tag is nested within the Counters tag. It contains the flag for gathering the counter values for disk quotas. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\quotas directories and includes the following parameters:

▪Event timestamp.

▪Disk quota usage counters.

Transports tag

The Transports tag is nested within the Counters tag. It contains the flag for gathering the counter values for delivered messages. Such statistics are gathered in a CSV file located in the [INSTALLDIR]\data\statistics\YYYY-MM-DD\transports directories and includes the following parameters:

▪Event timestamp.

▪Messages delivery counters.

Sample Configurations of EtherSensor Watcher

Example 1:

This configuration will create a file named example.log to which messages will be saved for the following channels:

▪The EtherSensor EtherCAP service - messages from the main ETHCAP channel with the following logging level: critical errors.

▪The EtherSensor ICAP service - messages from the main ICAP channel with the following logging level: non-critical service errors.

▪The EtherSensor Analyser service - messages from the main ANALYSER channel with the following logging level: warnings.

▪The EtherSensor Watcher service - messages from the main WATCHER channel with the following logging level: all messages.

Example 2:

This configuration will create a file named error.log to save messages from all Microolap EtherSensor services with the logging level of "non-critical errors".

Example 3:

In this case, the file will include messages with the following channel tags: ETHCAP - all messages from the service, CAPERR - only capture errors, CAPMAIN - only message processing errors.

Log File Format

Log files are XML files that look approximately as follows:

<Message time="2010-07-12T15:22:27.5390000" level="info">
<Client channelname="WATCHER"
processname="watcher.exe"
modulename="watcher.exe"
processId="5252" />
<Text>Start of the application work.</Text>
</Message>
<Message time="2010-07-12T15:22:27.5390000" level="info">
<Client channelname="WATCHER"
processname="watcher.exe"
modulename="watcher.exe"
processId="5252" />
</Message>
<Message time="2010-07-12T15:28:52.4670000" level="info">
<Client channelname="WATCHER"
processname="watcher.exe"
modulename="watcher.exe"
processId="5252" />
<Text>Finish of the application work.</Text>
</Message>

Message tag

It is the root directory of a message stored in a log file. It has the following attributes: time - specifies the time when the message has been sent, level - the priority with which the message was sent (for example, info means an information message, error means an error message).

Client tag

This tag describes the message sender. It has the following attributes: channelname - the message channel name, processname - the sender process name, modulename - the module name within the process that created the message, processId - the process ID in the runtime environment OS Microolap EtherSensor.

Text tag

The text of the message.

Additionally, in critical situations, the following tags may be used:

AdditionalDotNet tag

Describes dotNet error messages (including the information about exceptions thrown during Microolap EtherSensor work in .NET Framework).

AdditionalWin32 tag

Describes Win32 error messages (including information about exceptions thrown during Microolap EtherSensor in the native code).

Watcher Service Manual Setup (Config File)

Watcher Service Manual Setup (Config File)