Collect Microsoft Exchange logs
This document explains how to ingest Microsoft Exchange logs to Google Security Operations using Bindplane. The parser first attempts to interpret the incoming "message" field as JSON. If this fails, it applies a series of Grok patterns and CSV parsing techniques to extract fields from the raw message string, handling various Microsoft Exchange log formats and populating a standardized Unified Data Model (UDM) schema for security analysis.
Before you begin
Make sure you have the following prerequisites:
- Google SecOps instance
- Windows 2016 or later, or a Linux host with
systemd
- If running behind a proxy, firewall ports are open
- Installed Exchange Services
- Privileged access to Microsoft Windows Exchange
Get Google SecOps ingestion authentication file
- Sign in to the Google SecOps console.
- Go to SIEM Settings > Collection Agents.
- Download the Ingestion Authentication File. Save the file securely on the system where Bindplane will be installed.
Get Google SecOps customer ID
- Sign in to the Google SecOps console.
- Go to SIEM Settings > Profile.
- Copy and save the Customer ID from the Organization Details section.
Install the Bindplane agent
Windows installation
- Open the Command Prompt or PowerShell as an administrator.
Run the following command:
msiexec /i "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" /quiet
Linux installation
- Open a terminal with root or sudo privileges.
Run the following command:
sudo sh -c "$(curl -fsSlL https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh)" install_unix.sh
Additional installation resources
For additional installation options, consult the installation guide.
Configure the Bindplane agent to ingest Syslog and send to Google SecOps
- Access the configuration file:
- Locate the
config.yaml
file. Typically, it's in the/etc/bindplane-agent/
directory on Linux or in the installation directory on Windows. - Open the file using a text editor (for example,
nano
,vi
, or Notepad).
- Locate the
Edit the
config.yaml
file as follows:receivers: udplog: # Replace the port and IP address as required listen_address: "0.0.0.0:514" exporters: chronicle/chronicle_w_labels: compression: gzip # Adjust the path to the credentials file you downloaded in Step 1 creds: '/path/to/ingestion-authentication-file.json' # Replace with your actual customer ID from Step 2 customer_id: <customer_id> endpoint: malachiteingestion-pa.googleapis.com # Add optional ingestion labels for better organization ingestion_labels: log_type: 'EXCHANGE_MAIL' raw_log_field: body service: pipelines: logs/source0__chronicle_w_labels-0: receivers: - udplog exporters: - chronicle/chronicle_w_labels
Replace the port and IP address as required in your infrastructure.
Replace
<customer_id>
with the actual Customer ID.Update
/path/to/ingestion-authentication-file.json
to the path where the authentication file was saved in the Get Google SecOps ingestion authentication file section.
Restart the Bindplane agent to apply the changes
To restart the Bindplane agent in Linux, run the following command:
sudo systemctl restart bindplane-agent
To restart the Bindplane agent in Windows, you can either use the Services console or enter the following command:
net stop BindPlaneAgent && net start BindPlaneAgent
Configure MS Exchange events collection and export
- Sign in to the MS Exchange server machine.
- Download and Install NXlog on a Windows Server.
- Go to the NXlog installation path and open the config file.
- Open the
nxlog.conf
file located in the conf directory, typically atC:\Program Files\nxlog\conf (Default Installation Path
using a text editor. Add the following configuration code:
<Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0" Path="Application"> <Select Path="Application"> *[System[(Level=1 or Level=2 or Level=3)]] </Select> <Select Path="System"> *[System[(Level=1 or Level=2 or Level=3)]] </Select> <Select Path="MSExchange Management">*</Select> </Query> </QueryList> </QueryXML> </Input> <Output syslog> Module om_udp Host <bindplane-agent-ip> Port <bindplane-agent-port> Exec to_syslog_bsd(); </Output> <Route exchange_to_syslog> Path eventlog => syslog </Route>
Replace
<bindplane-agent-ip>
and<bindplane-agent-port>
with Bindplane configured IP address and port.Start the NXLog service:
- Open Services (services.msc) on the Windows Server.
- Find the NXLog service in the list of services.
- Right-click on NXLog and select Start.
Optional: Automate NXLog Startup:
- Open Services (services.msc).
- Find NXLog in the list.
- Right-click and select Properties.
- Set the Startup type to Automatic.
UDM mapping table
Log field | UDM mapping | Logic |
---|---|---|
c-ip | read_only_udm.target.asset.ip | Value taken from c-ip field |
c-ip | read_only_udm.target.ip | Value taken from c-ip field |
client-hostname | read_only_udm.principal.asset.hostname | Value taken from client-hostname field |
client-hostname | read_only_udm.principal.hostname | Value taken from client-hostname field |
client-ip | read_only_udm.principal.asset.ip | Value taken from client-ip field |
client-ip | read_only_udm.principal.ip | Value taken from client-ip field |
column1 | read_only_udm.metadata.event_timestamp | Value taken from column1 field |
column10 | read_only_udm.intermediary.resource.attribute.labels.value | Value taken from column10 field |
column11 | read_only_udm.network.email.mail_id | Value taken from column11 field |
column12 | read_only_udm.additional.fields.value.string_value | Value taken from column12 field |
column13 | read_only_udm.network.email.to | Value taken from column13 field |
column13 | read_only_udm.target.user.email_addresses | Value taken from column13 field |
column15 | read_only_udm.additional.fields.value.string_value | Value taken from column15 field |
column16 | read_only_udm.target.resource.attribute.labels.value | Value taken from column16 field |
column19 | read_only_udm.network.email.subject | Value taken from column19 field |
column2 | read_only_udm.principal.asset.ip | Value taken from column2 field |
column2 | read_only_udm.principal.ip | Value taken from column2 field |
column20 | read_only_udm.network.email.from | Value taken from column20 field |
column20 | read_only_udm.principal.user.email_addresses | Value taken from column20 field |
column21 | read_only_udm.security_result.detection_fields.value | Value taken from column21 field |
column22 | read_only_udm.security_result.description | Value taken from column22 field |
column24 | read_only_udm.additional.fields.value.string_value | Value taken from column24 field |
column25 | read_only_udm.principal.asset.ip | Value taken from column25 field |
column25 | read_only_udm.principal.ip | Value taken from column25 field |
column26 | read_only_udm.target.asset.ip | Value taken from column26 field |
column26 | read_only_udm.target.ip | Value taken from column26 field |
column27 | read_only_udm.security_result.detection_fields.value | Value taken from column27 field |
column28 | read_only_udm.additional.fields.value.string_value | Value taken from column28 field |
column29 | read_only_udm.metadata.product_log_id | Value taken from column29 field |
column3 | read_only_udm.principal.asset.hostname | Value taken from column3 field |
column3 | read_only_udm.principal.hostname | Value taken from column3 field |
column30 | read_only_udm.metadata.product_version | Value taken from column30 field |
column4 | read_only_udm.target.asset.ip | Value taken from column4 field |
column4 | read_only_udm.target.ip | Value taken from column4 field |
column5 | read_only_udm.target.asset.hostname | Value taken from column5 field |
column5 | read_only_udm.target.hostname | Value taken from column5 field |
column6 | read_only_udm.metadata.event_timestamp | Value taken from column6 field |
column6 | read_only_udm.network.http.response_code | Value taken from column6 field |
column6 | read_only_udm.network.session_id | Value taken from column6 field |
column6 | read_only_udm.metadata.description | Value taken from column6 field |
column7 | read_only_udm.additional.fields.value.string_value | Value taken from column7 field |
column8 | read_only_udm.additional.fields.value.string_value | Value taken from column8 field |
column9 | read_only_udm.metadata.product_event_type | Value taken from column9 field |
connector_id | read_only_udm.additional.fields.value.string_value | Value taken from connector-id field |
cs-method | read_only_udm.network.http.method | Value taken from cs-method field |
cs-uri-query | read_only_udm.target.url | Value taken from cs-uri-query field |
cs-uri-stem | read_only_udm.target.url | Value taken from cs-uri-stem field |
csReferer | read_only_udm.network.http.referral_url | Value taken from csReferer field |
csUser-Agent | read_only_udm.network.http.user_agent | Value taken from csUser-Agent field |
cs-username | read_only_udm.principal.user.userid | Value taken from cs-username field |
custom-data | read_only_udm.security_result.detection_fields.value | Value taken from custom-data field |
data | read_only_udm.security_result.about.labels.value | Value taken from data field |
data | read_only_udm.security_result.description | Value taken from data field |
data | read_only_udm.network.email.from | Value taken from data field |
data | read_only_udm.network.email.to | Value taken from data field |
data | read_only_udm.target.hostname | Value taken from data field |
data | read_only_udm.security_result.description | Value taken from data field |
data | read_only_udm.network.sent_bytes | Value taken from data field |
data | read_only_udm.target.user.email_addresses | Value taken from data field |
date | read_only_udm.metadata.event_timestamp | Value taken from date and time fields |
date-time | read_only_udm.metadata.event_timestamp | Value taken from date-time field |
DeliveryLatency | read_only_udm.security_result.detection_fields.value | Value taken from DeliveryLatency field in custom-data or message-info |
DeliveryPriority | read_only_udm.security_result.detection_fields.value | Value taken from DeliveryPriority field in custom-data or column21 field |
DeliveryPriority | read_only_udm.security_result.priority | If DeliveryPriority is Low or Normal then LOW_PRIORITY , if DeliveryPriority is Medium then MEDIUM_PRIORITY , if DeliveryPriority is High then HIGH_PRIORITY |
directionality | read_only_udm.network.direction | If directionality is Incoming then INBOUND , if directionality is Originating then OUTBOUND |
E2ELatency | read_only_udm.security_result.detection_fields.value | Value taken from E2ELatency field in custom-data or message-info |
event | read_only_udm.metadata.product_event_type | If event is + then Connect , if event is - then Disconnect , if event is * then Information , if event is > then Send , if event is < then Receive |
event | read_only_udm.network.direction | If event is > then OUTBOUND , if event is < then INBOUND |
EventID | read_only_udm.security_result.detection_fields.value | Value taken from EventID field |
EventReceivedTime | read_only_udm.metadata.collected_timestamp | Value taken from EventReceivedTime field |
EventReceivedTime | read_only_udm.metadata.event_timestamp | Value taken from EventReceivedTime field in column6 |
FirstForestHop | read_only_udm.security_result.detection_fields.value | Value taken from FirstForestHop field in custom-data |
FromEntity | read_only_udm.security_result.detection_fields.value | Value taken from FromEntity field in custom-data or message-info |
guid | read_only_udm.metadata.product_log_id | Value taken from guid field |
Hostname | read_only_udm.principal.asset.hostname | Value taken from Hostname field |
Hostname | read_only_udm.principal.hostname | Value taken from Hostname field |
IncludeInSla | read_only_udm.security_result.detection_fields.value | Value taken from IncludeInSla field in custom-data or message-info |
internal-message-id | read_only_udm.intermediary.resource.attribute.labels.value | Value taken from internal-message-id field |
IsProbe | read_only_udm.security_result.detection_fields.value | Value taken from IsProbe field in custom-data or column21 field |
Keywords | read_only_udm.security_result.detection_fields.value | Value taken from Keywords field |
local-endpoint | read_only_udm.principal.asset.ip | Value taken from local-endpoint field |
local-endpoint | read_only_udm.principal.ip | Value taken from local-endpoint field |
local-endpoint | read_only_udm.principal.port | Value taken from local-endpoint field |
Mailboxes | read_only_udm.security_result.detection_fields.value | Value taken from Mailboxes field in custom-data or message-info |
MailboxDatabaseGuid | read_only_udm.security_result.detection_fields.value | Value taken from MailboxDatabaseGuid field in custom-data or message-info |
MAIL FROM | read_only_udm.network.email.from | Value taken from MAIL FROM field in data |
MAIL FROM | read_only_udm.principal.user.email_addresses | Value taken from MAIL FROM field in data |
MAIL From | read_only_udm.network.email.from | Value taken from MAIL From field in data |
MAIL From | read_only_udm.principal.user.email_addresses | Value taken from MAIL From field in data |
message-id | read_only_udm.network.email.mail_id | Value taken from message-id field |
message-info | read_only_udm.security_result.detection_fields.value | Value taken from message-info field |
message-info | read_only_udm.security_result.description | Value taken from message-info field |
MessageValue | read_only_udm.security_result.detection_fields.value | Value taken from MessageValue field in custom-data |
message-subject | read_only_udm.network.email.subject | Value taken from message-subject field |
method | read_only_udm.network.http.method | Value taken from method field |
Microsoft_Exchange_Transport_MailRecipient_RequiredTlsAuthLevel | read_only_udm.security_result.detection_fields.value | Value taken from Microsoft_Exchange_Transport_MailRecipient_RequiredTlsAuthLevel field in custom-data |
MsgRecipCount | read_only_udm.security_result.detection_fields.value | Value taken from MsgRecipCount field in custom-data or message-info |
network-message-id | read_only_udm.additional.fields.value.string_value | Value taken from network-message-id field |
OriginalFromAddress | read_only_udm.principal.user.email_addresses | Value taken from OriginalFromAddress field in custom-data or column21 field |
P2RecipStat | read_only_udm.security_result.detection_fields.value | Value taken from P2RecipStat field in custom-data or message-info |
PersistProbeTrace | read_only_udm.security_result.detection_fields.value | Value taken from PersistProbeTrace field in custom-data or column21 field |
PrioritizationReason | read_only_udm.security_result.detection_fields.value | Value taken from PrioritizationReason field in custom-data |
ProbeType | read_only_udm.security_result.detection_fields.value | Value taken from ProbeType field in custom-data or column21 field |
ProcessID | read_only_udm.principal.process.pid | Value taken from ProcessID field |
ProxiedClientHostname | read_only_udm.intermediary.hostname | Value taken from ProxiedClientHostname field in custom-data |
ProxiedClientIPAddress | read_only_udm.intermediary.asset.ip | Value taken from ProxiedClientIPAddress field in custom-data |
ProxiedClientIPAddress | read_only_udm.intermediary.ip | Value taken from ProxiedClientIPAddress field in custom-data |
ProxyHop1 | read_only_udm.security_result.detection_fields.value | Value taken from ProxyHop1 field in custom-data |
RCPT TO | read_only_udm.network.email.to | Value taken from RCPT TO field in data |
RCPT TO | read_only_udm.target.user.email_addresses | Value taken from RCPT TO field in data |
RCPT To | read_only_udm.network.email.to | Value taken from RCPT To field in data |
RCPT To | read_only_udm.target.user.email_addresses | Value taken from RCPT To field in data |
recipient-address | read_only_udm.target.user.email_addresses | Value taken from recipient-address field |
recipient-count | read_only_udm.target.resource.attribute.labels.value | Value taken from recipient-count field |
recipient-status | read_only_udm.target.resource.attribute.labels.value | Value taken from recipient-status field |
remote-endpoint | read_only_udm.target.asset.ip | Value taken from remote-endpoint field |
remote-endpoint | read_only_udm.target.ip | Value taken from remote-endpoint field |
remote-endpoint | read_only_udm.target.port | Value taken from remote-endpoint field |
res_code | read_only_udm.network.http.response_code | Value taken from res_code field |
s-ip | read_only_udm.principal.asset.ip | Value taken from s-ip field |
s-ip | read_only_udm.principal.ip | Value taken from s-ip field |
s-port | read_only_udm.principal.port | Value taken from s-port field |
sc-status | read_only_udm.network.http.response_code | Value taken from sc-status field |
sc-substatus | read_only_udm.additional.fields.value.string_value | Value taken from sc-substatus field |
sender-address | read_only_udm.network.email.from | Value taken from sender-address field |
sender-address | read_only_udm.principal.user.email_addresses | Value taken from sender-address field |
sequence-number | read_only_udm.additional.fields.value.number_value | Value taken from sequence-number field |
server-hostname | read_only_udm.target.asset.hostname | Value taken from server-hostname field |
server-hostname | read_only_udm.target.hostname | Value taken from server-hostname field |
server-ip | read_only_udm.target.asset.ip | Value taken from server-ip field |
server-ip | read_only_udm.target.ip | Value taken from server-ip field |
session-id | read_only_udm.network.session_id | Value taken from session-id field |
sessionid | read_only_udm.network.session_id | Value taken from sessionid field |
Severity | read_only_udm.security_result.severity | If Severity contains Info then INFORMATIONAL , if Severity contains Error then ERROR , if Severity contains Warning then MEDIUM , else UNKNOWN_SEVERITY |
SeverityValue | read_only_udm.security_result.severity_details | Value taken from SeverityValue field |
SlaExclusionReason | read_only_udm.security_result.detection_fields.value | Value taken from SlaExclusionReason field in custom-data |
source | read_only_udm.additional.fields.value.string_value | Value taken from source field |
SourceModuleName | read_only_udm.principal.resource.name | Value taken from SourceModuleName field |
SourceModuleType | read_only_udm.principal.resource.type | Value taken from SourceModuleType field |
SourceName | read_only_udm.principal.resource.attribute.labels.value | Value taken from SourceName field |
StoreObjectIds | read_only_udm.security_result.detection_fields.value | Value taken from StoreObjectIds field in custom-data or message-info |
Task | read_only_udm.security_result.detection_fields.value | Value taken from Task field |
ThreadID | read_only_udm.security_result.detection_fields.value | Value taken from ThreadID field |
time | read_only_udm.metadata.event_timestamp | Value taken from date and time fields |
ToEntity | read_only_udm.security_result.detection_fields.value | Value taken from ToEntity field in custom-data or message-info |
total-bytes | read_only_udm.additional.fields.value.string_value | Value taken from total-bytes field |
TransportTrafficSubType | read_only_udm.security_result.detection_fields.value | Value taken from TransportTrafficSubType field in custom-data |
TransportTrafficSubType | read_only_udm.metadata.product_version | Value taken from TransportTrafficSubType field in custom-data |
ts | read_only_udm.metadata.event_timestamp | Value taken from ts field |
u_agent | read_only_udm.network.http.user_agent | Value taken from u_agent field |
u_param | read_only_udm.target.url | Value taken from u_param field |
u_path | read_only_udm.target.url | Value taken from u_path field |
u_path | read_only_udm.target.url | Value taken from u_path and u_param fields |
user | read_only_udm.target.user.userid | Value taken from user field |
user | read_only_udm.target.user.email_addresses | Value taken from user field |
metadata.event_type | read_only_udm.metadata.event_type | If has_principal_email is true and has_target_email is true then EMAIL_TRANSACTION , if event_type is GENERIC_EVENT and principal_hostname or s_ip or host is not empty or has_principal is true then STATUS_UPDATE , if event_type is GENERIC_EVENT and has_principal_email is true or has_target_email is true then USER_UNCATEGORIZED , else value taken from event_type field |
metadata.log_type | read_only_udm.metadata.log_type | Hardcoded value EXCHANGE_MAIL |
metadata.product_name | read_only_udm.metadata.product_name | Hardcoded value Exchange Mail |
metadata.vendor_name | read_only_udm.metadata.vendor_name | Hardcoded value Microsoft |
network.application_protocol | read_only_udm.network.application_protocol | If app_protocol is SMTP or HTTP or HTTPS then value taken from app_protocol field, if app_protocol contains SMTP then SMTP |
network.direction | read_only_udm.network.direction | If s_ip is not empty then INBOUND |
network.email.from | read_only_udm.network.email.from | Value taken from from_mail field |
network.email.mail_id | read_only_udm.network.email.mail_id | Value taken from msg_id field |
network.email.subject | read_only_udm.network.email.subject | Value taken from column19 field |
network.email.to | read_only_udm.network.email.to | Value taken from to_mail field |
network.http.method | read_only_udm.network.http.method | Value taken from method field |
network.http.response_code | read_only_udm.network.http.response_code | Value taken from res_code field |
network.http.user_agent | read_only_udm.network.http.user_agent | Value taken from u_agent field |
network.sent_bytes | read_only_udm.network.sent_bytes | Value taken from sent_bytes field |
network.session_id | read_only_udm.network.session_id | Value taken from sessionid field |
principal.asset.hostname | read_only_udm.principal.asset.hostname | Value taken from principal_hostname field |
principal.asset.hostname | read_only_udm.principal.asset.hostname | Value taken from host field |
principal.asset.hostname | read_only_udm.principal.asset.hostname | Value taken from column3 field |
principal.asset.ip | read_only_udm.principal.asset.ip | Value taken from column2 field |
principal.asset.ip | read_only_udm.principal.asset.ip | Value taken from column25 field |
principal.asset.ip | read_only_udm.principal.asset.ip | Value taken from s_ip field |
principal.hostname | read_only_udm.principal.hostname | Value taken from principal_hostname field |
principal.hostname | read_only_udm.principal.hostname | Value taken from host field |
principal.hostname | read_only_udm.principal.hostname | Value taken from column3 field |
principal.ip | read_only_udm.principal.ip | Value taken from column2 field |
principal.ip | read_only_udm.principal.ip | Value taken from column25 field |
principal.ip | read_only_udm.principal.ip | Value taken from s_ip field |
principal.port | read_only_udm.principal.port | Value taken from s-port field |
principal.user.email_addresses | read_only_udm.principal.user.email_addresses | Value taken from mail field |
principal.user.email_addresses | read_only_udm.principal.user.email_addresses | Value taken from email_address field |
principal.user.userid | read_only_udm.principal.user.userid | Value taken from cs-username field |
security_result.about.labels.key | read_only_udm.security_result.about.labels.key | Hardcoded value Response Code |
security_result.description | read_only_udm.security_result.description | Value taken from context field |
security_result.description | read_only_udm.security_result.description | Value taken from column22 field |
security_result.priority | read_only_udm.security_result.priority | If severity is 1 or 2 or 3 then LOW , if severity is 4 or 5 or 6 then MEDIUM , if severity is 7 or 8 or 9 then HIGH |
security_result.severity | read_only_udm.security_result.severity | If Severity contains Info then INFORMATIONAL , if Severity contains Error then ERROR , if Severity contains Warning then MEDIUM , else UNKNOWN_SEVERITY |
target.administrative_domain | read_only_udm.target.administrative_domain | Value taken from domain field |
target.asset.hostname | read_only_udm.target.asset.hostname | Value taken from column5 field |
target.asset.hostname | read_only_udm.target.asset.hostname | Value taken from target_host field |
target.asset.ip | read_only_udm.target.asset.ip | Value taken from column4 field |
target.asset.ip | read_only_udm.target.asset.ip | Value taken from column26 field |
target.asset.ip | read_only_udm.target.asset.ip | Value taken from c-ip field |
target.hostname | read_only_udm.target.hostname | Value taken from column5 field |
target.hostname | read_only_udm.target.hostname | Value taken from target_host field |
target.ip | read_only_udm.target.ip | Value taken from column4 field |
target.ip | read_only_udm.target.ip | Value taken from column26 field |
target.ip | read_only_udm.target.ip | Value taken from c-ip field |
target.port | read_only_udm.target.port | Value taken from c_port field |
target.resource.attribute.labels.key | read_only_udm.target.resource.attribute.labels.key | Hardcoded value Recipients Count |
target.user.email_addresses | read_only_udm.target.user.email_addresses | Value taken from user field |
target.user.user_display_name | read_only_udm.target.user.user_display_name | Value taken from username field |
target.user.userid | read_only_udm.target.user.userid | Value taken from user field |
target.url | read_only_udm.target.url | Value taken from u_path field |
Need more help? Get answers from Community members and Google SecOps professionals.