Event and Object Analysis > Messages Created

Microolap EtherSensor generates messages from application-level objects (messages) extracted from the traffic.Microolap EtherSensor delivers such messages for further processing to consumer systems via the results delivery service.

The following ways and methods are used to deliver reconstructed communications data:

1. The captured message is converted into an XML passport, which contains all known details of captured data, senders, recipients, etc. The collected data are added to the passport during the processing.

2. After the object processing is completed by Microolap EtherSensor the object is passed to the results delivery service in the form of passport and other data files for delivery.

3. Depending on the required transport type and its settings, the results delivery service uses the passport to generate the object for delivery.

For example, in case of SMTP delivery, an email message is generated where the sender and the recipient addresses (FROM, TO, CC, BCC) are populated with captured data for the source and destination of the message (if such data is available). The text of the message is used as the text of the email message, and transmitted files are represented as attachments to the message. Other data collected by Microolap EtherSensor during the message processing are saved as MIME headers of the message.

Message format for the delivery of results

Messages transmitted by the results delivery service should meet the RFC requirements to SMTP messages. Long headers may be split and encoded; UTF-8 is used as the primary encoding. Object types are determined based on processed protocol headers.

Please note:
1. For certain web services, multiple messages may be captured for a single user operation in the user interface: this is how Microolap EtherSensor processes drafts and attachments sent to the service.
2. Sometimes the text and the attachment may be delivered as separate messages.
3. Sometimes a single container with the contents of several messages may be delivered (this mainly happens in case of IM protocols to minimize traffic and related overheads).

Service MIME headers of Microolap EtherSensor with sample values

X-Sensor-Version: 5.1.0.13519

Text string. Identifies the current version of Microolap EtherSensor.

X-Sensor-Id: sensor-01

Text string. You can use sensor ID to distinguish messages from different sensors or group them by sensors. It can be any ASCII ID, and is assigned UHID by default.

X-Sensor-Session-Id: 6612456

Integer. Internal ID of connections processed by Microolap EtherSensor. It is assigned by the message source - EtherSensor ICAP or EtherSensor EtherCAP service. It is then recorded to the capture.log file.

X-Sensor-Net-Interface-Id: 00-21-28-10-58-80

Text string. If the message is intercepted by the EtherSensor EtherCAP service, it is assigned the MAC address of the interface or capdrop - the virtual ID of the PCAP file parsing driver.

For the ICAP server, the ID is taken from the ICAP server configuration. You can use this header to trace the source of the message: the interface or service from which the data came for processing.

X-Sensor-Session-Level: 0

Integer. Specifies the number of protocols processed in order to reach the message. The list of protocols may include HTTP connection, HTTP proxy connection over it, and GRE encapsulation.

X-Sensor-Src-Address: 10.31.90.22:47016

X-Sensor-Dst-Address: 193.203.100.139:8080

Connection IP addresses and ports (outgoing and destination). They may be not defined when ICAP traffic without X-Client-IP and X-Server-IP headers is processed. An example of ICAP headers: X-Client-IP: 192.168.3.67, X-Server-IP: 123.45.67.89.

X-Sensor-Src-Host: pc-test.msk.su

X-Sensor-Dst-Host: nns-team.ru

Text strings. Host names corresponding to the X-Sensor-Src-Address and X-Sensor-Dst-Address headers. Many networks use DHCP to assign internal addresses, and that is why the host name should only be determined when the message is captured. To do so, Microolap EtherSensor sends a return DNS query to the DNS server specified in the EtherSensor Analyser service configuration. If the value cannot be recognized, it is set to .

X-Sensor-Protocol: HTTP

Text string. The name of the detector used to parse data. Possible options are SMTP, ICQ, MRA, etc.

X-Sensor-Detector: phpBB

Text string. The name of the Microolap EtherSensor detector which detected the presence of data in the connection.

X-Sensor-Attachments-Count: 0

Integer. The number of files in the intercepted message.

X-Sensor-Object-Date: Fri, 17 Sep 2010 17:30:24 +0400

Text string. The time when the message was intercepted (object creation date in Microolap EtherSensor). The time zone is based on the sensor OS settings.

X-Sensor-Object-Size: 2735

Integer. The size of the intercepted object in bytes before processing.

X-Sensor-Object-MD5Hash: c326230de58279229862b18e818a3912

Text string, the md5 hash of the captured object. If messages contain identical text but have different size, hash, object date or source and destination addresses and ports, they are not duplicates but rather two very similar objects. Such objects normally do not have identical md5 hash. If they do, then Microolap EtherSensor processes the same connection multiple times (a loop).

X-Sensor-Via: 1.1 off:1080 (squid/2.6.STABLE18)

X-Sensor-Forwarded-For: 10.255.241.31

Text strings. Headers from an HTTP connection; headers values are populated (if available). You can use these values to determine if the connection client is behind a proxy server, or sometimes find out the client's address at the moment of message registration.

X-Sensor-Icap-Client-Username: user1

X-Sensor-Icap-Subscriber-Id: mike.smith@mycompany.com

X-Sensor-Icap-Authenticated-User: TERBUDovLzE5Mi4xNjguMTIuMTAwL289bXljb21wYW55LCBvdT1lbmdpbmVlcmluZywgY249bWlrZS5zbWl0aA==

X-Sensor-Icap-Authenticated-Group: TERBUDovLzE5Mi4xNjguMTIuMTAwL289bXljb21wYW55LCBvdT1lbmdpbmVlcmluZw==

The values of these headers are generated when ICAP traffic is processed. Corresponding ICAP protocol headers are used for that (X-Client-Username, X-Subscriber-ID, X-Authenticated-User, X-Authenticated-Groups).

X-Sensor-Filter-Name: TEST

X-Sensor-Tags: Filtered=1

X-Sensor-Labels: filter-begin-time="2010-09-17T17:30:24.6318125+04:00",

dns-begin="2010-09-17T17:30:24.6318125+04:00",

dns-end="2010-09-17T17:30:24.6318125+04:00",

Filtered="true",

filter-end-time="2010-09-17T17:30:24.6318125+04:00"

Service headers of the EtherSensor Analyser service. You can use them to determine the filter that was triggered, the nature of the message content, tags and labels set for it, and track how the message is processed by the filter. Resulting headers may vary significantly depending on the filter policy.

Date: Fri, 17 Sep 2010 17:30:24 +0400

The X-Sensor-Object-Date value is copied to this header.

From: anonymous@nns-team.ru

To: forum@nns-team.ru

CC, BCC and other headers

Subject: Re: World of Tanks

Sender and receiver headers are populated with user addresses or IDs extracted from messages, query headers, etc. when possible. These may be not defined, and they depend on the detector: e.g. if a protocol does not set the message subject, it may be populated with the information from the sensor.

X-Sensor-RawSource-Type: LotusMail

All the messages are labeled with the "X-Sensor-RawSource-Type" header to specify the data source of the final message.

This header in the current version of Microolap EtherSensor (5.1.0.13519) may have the following values:

HttpGetRequest

Specifies a GET HTTP query as the primary data source.

HttpPostRequest

Specifies a POST HTTP query as the primary data source.

HttpPutRequest

Specifies a PUT HTTP query as the primary data source.

FtpFile

Specifies an FTP file as the primary data source.

SmtpEml

Specifies an SMTP message in EML format as the primary data source.

Pop3Eml

Specifies a POP3 message in EML format as the primary data source.

IcqContactList

Specifies an ICQ contact list as the primary data source.

IcqMessageList

Specifies an ICQ message list as the primary data source.

IcqFile

Specifies a file transmitted between ICQ clients as the primary data source.

IcqLoginInfo

Specifies ICQ user details as the primary data source.

MraUserInfo

Specifies MRA user details as the primary data source.

MraContactList

Specifies an MRA contact list as the primary data source.

MraMessageList

Specifies an MRA message list as the primary data source.

MraFile

Specifies a file transmitted between MRA clients as the primary data source.

MsnContactList

Specifies an MSN contact list as the primary data source.

MsnMessageList

Specifies an MSN message list as the primary data source.

MsnFile

Specifies a file transmitted between MSN clients as the primary data source.

XmppContactList

Specifies an XMPP contact list as the primary data source.

XmppMessageList

Specifies an XMPP message list as the primary data source.

XmppFile

Specifies a file transmitted between XMPP clients as the primary data source.

IrcMessageList

Specifies an IRC message list as the primary data source.

IrcFile

Specifies a file transmitted between IRC clients as the primary data source.

SkypeVersionRequest

Specifies a latest version request to ui.skype.com as the primary data source.

SslSessionsList

Specifies an SSL session list as the primary data source.

LotusMail

Specifies a LOTUS protocol message list as the primary data source.

LotusAttachment

Specifies an attachment file of a LOTUS message as the primary data source.

X-Sensor-LicOption: Lotus

All the messages are labeled with the "X-Sensor-LicOption" header to specify the module used to process this message. This header in the current version of Microolap EtherSensor (5.1.0.13519) may have the following values:

WebMail

Indicates that the message was processed by the module with the "Web mail" licensed option.

WebSocial

Indicates that the message was processed by the module with the "Social networks" licensed option.

Indicates that the message was processed by the module with the "E-mail" licensed option.

Indicates that the message was processed by the module with the "Instant messages" licensed option.

Indicates that the message was processed by the module with the "File transfer" licensed option.

WebMailRead

Indicates that the message was processed by the module with the "Reading incoming web mail" licensed option.

Lotus

Indicates that the message was processed by the module with the "Interception of Lotus Notes messages" licensed option.

LotusTxn

Indicates that the message was processed by the module with the "Extraction of messages from Lotus Notes Transaction Log" licensed option.

X-Sensor-UID: 0e515c8c-61eb-11e1-a529-000c29ff0707

This header is generated when the Microolap EtherSensor server communicates with EtherSensor Agent instances installed on network users workstations. The header value uniquely identifies the company user on a specific computer within the company.

X-Sensor-UID-UserName: CN=Administrator,CN=Users,DC=bigbrother,DC=foo

This header is generated when the Microolap EtherSensor server communicates with EtherSensor Agent instances installed at workstations of network users. The header value uniquely identifies the company user within the company.

X-Sensor-UID-UserSID: S-1-5-21-86032015-1269853868-1024056280-1001

X-Sensor-UID-ComputerName: WS325-LOCK.bigbrother.foo

X-Sensor-UID-AdapterType: if_type_ethernet_csmacd

This header is generated when the Microolap EtherSensor server communicates with EtherSensor Agent instances installed at workstations of network users. The header value contains the type of the network adapter used to send the message. See the complete list of possible values of this field on Microsoft website.

X-Sensor-UID-MacAddress: 00-1F-C6-2D-EA-40

This header is generated when the Microolap EtherSensor server communicates with EtherSensor Agent instances installed at workstations of network users. The header value contains the MAC address of the network adapter used to send the message.

X-Sensor-UHID: UO2D-RNVO-JRN7-R1EN-91C0-61TA-1HP7-YRVF

Contains a unique ID for each Microolap EtherSensor instance and equipment set (Unique Hardware Identifier).

X-Sensor-Lotus-MessageId: <OF4D026078.B21F0C4F-ON44257C15.002E1625-44257C15.002E2225@LocalDomain>

Contains the unique ID of the message transmitted over the Lotus Notes protocol.

X-Sensor-Lotus-Form: Reply

Contains the name of the form of the message transmitted over the Lotus protocol.

X-Sensor-Lotus-Mailer: Lotus Notes Release 8.5.2FP2 SHF236 October 24, 2011

Contains a string which identifies the type of the Lotus Notes user.

X-Sensor-Lotus-INetPrincipal: UserName/OU/O@ServerName.Domain.com

Contains extended details of the message sender.

X-Sensor-Lotus-RouteServers: CN=lotus1/O=Company

Contains the list of Lotus servers which participated in message transmission.

X-Sensor-Lotus-References: <OF02B9DAC2.33CDF581-ON44257C15.002DF8CD@LocalDomain>

Contains the list of message IDs referred by the current message.

Generated messages

Generated messages