Collect BeyondTrust Endpoint Privilege Management logs

This document explains how to ingest BeyondTrust Endpoint Privilege Management (EPM) logs to Google Security Operations using AWS S3. The parser focuses on transforming raw JSON log data from BeyondTrust Endpoint into a structured format conforming to the Unified Data Model (UDM). It first initializes default values for various fields and then parses the JSON payload, subsequently mapping specific fields from the raw log into corresponding UDM fields within the event.idm.read_only_udm object.

Before you begin

Make sure you have the following prerequisites:

• Google SecOps instance
• Privileged access to AWS
• Privileged access to BeyondTrust Endpoint Privilege Management

Configure AWS IAM for Google SecOps ingestion

1. Create a User following this user guide: Creating an IAM user.
2. Select the created User.
3. Select the Security Credentials tab.
4. Click Create Access Key in the Access Keys section.
5. Select Third-party service as Use case.
6. Click Next.
7. Optional: Add a description tag.
8. Click Create access key.
9. Click Download CSV file to save the Access Key and Secret Access Key for future reference.
10. Click Done.
11. Select the Permissions tab.
12. Click Add permissions in the Permissions policies section.
13. Select Add permissions.
14. Select Attach policies directly.
15. Search for and select the AmazonS3FullAccess policy.
16. Click Next.
17. Click Add permissions.

Configure BeyondTrust EPM for API access

1. Sign in to the BeyondTrust Privilege Management web console as an administrator.
2. Go to System Configuration > REST API > Tokens.
3. Click Add Token.
4. Provide the following configuration details:
   • Name: Enter Google SecOps Collector.
   • Scopes: Select Audit:Read and other scopes as required.
5. Save and Copy the token value (this will be your BPT_API_TOKEN).
6. Copy your API base URL; it is typically https://<your-epm-server>/api/v3 or /api/v2, depending on your version (you'll use this as BPT_API_URL).

Create an AWS S3 Bucket

1. Sign in to the AWS Management Console.
2. Go to AWS Console > Services > S3 > Create bucket.
3. Provide the following configuration details:
   • Bucket name: my-beyondtrust-logs.
   • Region: [your choice] > Create.

Create an IAM Role for EC2

1. Sign in to the AWS Management Console.
2. Go to AWS Console > Services > IAM > Roles > Create role.
3. Provide the following configuration details:
   • Trusted entity: AWS service > EC2 > Next.
   • Attach permission: AmazonS3FullAccess (or a scoped policy to your bucket) > Next.
   • Role name: EC2-S3-BPT-Writer > Create role.

Optional: Launch and configure your EC2 Collector VM

1. Sign in to the AWS Managmenet Console.
2. Go to Services.
3. In the search bar, type EC2 and select it.
4. In the EC2 dashboard, click Instances.
5. Click Launch instances.
6. Provide the following configuration details:
   • Name: Enter BPT-Log-Collector
   • AMI: Select Ubuntu Server 22.04 LTS
   • Instance type: t3.micro (or larger), and_then click Next.
   • Network: Make sure the Network setting is set to your default VPC.
   • IAM role: Select the EC2-S3-BPT-Writer IAM role from the menu.
   • Auto-assign Public IP: Enable (or make sure you can reach it using VPN) > Next.
   • Add Storage: Leave the default storage configuration (8 GiB), and then click Next.
   • Select Create a new security group.
   • Inbound rule: Click Add Rule.
   • Type: Select SSH.
   • Port: 22.
   • Source: your IP
   • Click Review and Launch.
   • Select or create a key pair.
   • Click Download Key Pair.
   • Save the downloaded PEM file. You will need this file to connect to your instance using SSH.

7. Connect to your Virtual Machine (VM) using SSH:

   chmod 400 ~/Downloads/your-key.pem
   ssh -i ~/Downloads/your-key.pem ubuntu@<EC2_PUBLIC_IP>
   

Install collector prerequisites

1. Run the following command:

   # Update OS
   sudo apt update && sudo apt upgrade -y
   
   # Install Python, Git
   sudo apt install -y python3 python3-venv python3-pip git
   
   # Create & activate virtualenv
   python3 -m venv ~/bpt-venv
   source ~/bpt-venv/bin/activate
   
   # Install libraries
   pip install requests boto3
   

2. Create a directory & state file:

   sudo mkdir -p /var/lib/bpt-collector
   sudo touch /var/lib/bpt-collector/last_run.txt
   sudo chown ubuntu:ubuntu /var/lib/bpt-collector/last_run.txt
   

3. Initialize it (for example, to 1 hour ago):

   echo "$(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ)" > /var/lib/bpt-collector/last_run.txt
   

Deploy the Armis Collector Script

1. Create a project folder:

   mkdir ~/bpt-collector && cd ~/bpt-collector
   

2. Export the required environment variables (for example, in ~/.bashrc):

   export BPT_API_URL="https://<your-subdomain>-services.pm.beyondtrustcloud.com"
   export BPT_CLIENT_ID="your-client-id"
   export BPT_CLIENT_SECRET="your-client-secret"
   export S3_BUCKET="my-bpt-logs"
   export S3_PREFIX="bpt/"
   export STATE_FILE="/var/lib/bpt-collector/last_run.txt"
   export PAGE_SIZE="100"
   

3. Create collector_bpt.py and enter the following code:

   #!/usr/bin/env python3
   import os, sys, json, boto3, requests
   from datetime import datetime, timezone, timedelta
   
   # ── UTILS ─────────────────────────────────────────────────────────────────
   def must_env(var):
       val = os.getenv(var)
       if not val:
           print(f"ERROR: environment variable {var} is required", file=sys.stderr)
           sys.exit(1)
       return val
   
   def ensure_state_file(path):
       d = os.path.dirname(path)
       if not os.path.isdir(d):
           os.makedirs(d, exist_ok=True)
       if not os.path.isfile(path):
           ts = (datetime.now(timezone.utc) - timedelta(hours=1))\
               .strftime("%Y-%m-%dT%H:%M:%SZ")
           with open(path, "w") as f:
               f.write(ts)
   
   # ── CONFIG ─────────────────────────────────────────────────────────────────
   BPT_API_URL    = must_env("BPT_API_URL")       # for example, https://subdomain-services.pm.beyondtrustcloud.com
   CLIENT_ID      = must_env("BPT_CLIENT_ID")
   CLIENT_SECRET  = must_env("BPT_CLIENT_SECRET")
   S3_BUCKET      = must_env("S3_BUCKET")
   S3_PREFIX      = os.getenv("S3_PREFIX", "")    # for example, "bpt/"
   STATE_FILE     = os.getenv("STATE_FILE", "/var/lib/bpt-collector/last_run.txt")
   PAGE_SIZE      = int(os.getenv("PAGE_SIZE", "100"))
   # ── END CONFIG ─────────────────────────────────────────────────────────────
   
   ensure_state_file(STATE_FILE)
   
   def read_last_run():
       with open(STATE_FILE, "r") as f:
           ts = f.read().strip()
       return datetime.fromisoformat(ts.replace("Z", "+00:00"))
   
   def write_last_run(dt):
       with open(STATE_FILE, "w") as f:
           f.write(dt.strftime("%Y-%m-%dT%H:%M:%SZ"))
   
   def get_oauth_token():
       resp = requests.post(
           f"{BPT_API_URL}/oauth/connect/token",
           headers={"Content-Type": "application/x-www-form-urlencoded"},
           data={
               "grant_type":    "client_credentials",
               "client_id":     CLIENT_ID,
               "client_secret": CLIENT_SECRET
           }
       )
       resp.raise_for_status()
       return resp.json()["access_token"]
   
   def fetch_events(token, start, end):
       headers = {"Authorization": f"Bearer {token}"}
       offset = 0
       while True:
           params = {
               "startTime": start,
               "endTime":   end,
               "limit":     PAGE_SIZE,
               "offset":    offset
           }
           resp = requests.get(
               f"{BPT_API_URL}/management-api/v1/Audit/Events",
               headers=headers, params=params
           )
           resp.raise_for_status()
           events = resp.json().get("events", [])
           if not events:
               break
           for e in events:
               yield e
           offset += PAGE_SIZE
   
   def upload_to_s3(obj, key):
       boto3.client("s3").put_object(
           Bucket=S3_BUCKET, Key=key,
           Body=json.dumps(obj).encode("utf-8")
       )
   
   def main():
       # 1) determine window
       start_dt = read_last_run()
       end_dt   = datetime.now(timezone.utc)
       START = start_dt.strftime("%Y-%m-%dT%H:%M:%SZ")
       END   = end_dt.strftime("%Y-%m-%dT%H:%M:%SZ")
       print(f"Fetching events from {START} to {END}")
   
       # 2) authenticate and fetch
       token = get_oauth_token()
       count = 0
       for idx, evt in enumerate(fetch_events(token, START, END), start=1):
           key = f"{S3_PREFIX}{end_dt.strftime('%Y/%m/%d')}/evt_{int(end_dt.timestamp())}_{idx}.json"
           upload_to_s3(evt, key)
           count += 1
       print(f"Uploaded {count} events")
   
       # 3) persist state
       write_last_run(end_dt)
   
   if __name__ == "__main__":
       main()
   

4. Make it executable:

   chmod +x collector_bpt.py
   

Schedule Daily with Cron

1. Run the following command:

   crontab -e
   

2. Add the daily job at midnight UTC:

   0 0 * * * cd ~/bpt-collector && source ~/bpt-venv/bin/activate && ./collector_bpt.py >> ~/bpt-collector/bpt.log 2>&1
   

Configure a feed in Google SecOps to ingest Beyondtrust EPM logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, BeyondTrust EPM Logs).
4. Select Amazon S3 as the Source type.
5. Select Beyondtrust Endpoint Privilege Management as the Log type.
6. Click Next.

7. Specify values for the following input parameters:

   • Region: The region where the Amazon S3 bucket is located.
   • S3 URI: The bucket URI (the format should be: s3://your-log-bucket-name/). Replace the following:
     • your-log-bucket-name: the name of the bucket.
   • URI is a: Select Directory or Directory which includes subdirectories.
   • Source deletion options: select the deletion option according to your preference.
   • Access Key ID: the User access key with access to the S3 bucket.
   • Secret Access Key: the User secret key with access to the S3 bucket.
   • Asset namespace: the asset namespace.
   • Ingestion labels: the label to be applied to the events from this feed.

8. Click Next.

9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM mapping table

Need more help? Get answers from Community members and Google SecOps professionals.