In the following guide we will discuss how to configure a SIEM using ELK Stack.

How to configure a SIEM at free of cost or the least expenses? How to configure a SIEM without any shackles of license cost?

How to configure elasticsearch, kibana and logstash as a SIEM?

1. Ubuntu Host Setup

The following section will guide you to prepare an ubuntu server for ELK setup:

Essential package installation:

• docker: How To Install and Use Docker on Ubuntu 20.04 | DigitalOcean
• python: How to Install Python Pip on Ubuntu 20.04 | Linuxize
• vm-tools: $ apt install open-vm-tools

Hardening of the host:

• Disabled root ssh login
• Enable ssh login with ssh key
• Enable ufw firewall and allow only required ports

Mount additional SSD for storage:

• Find the mount name: $ lsblk
• Find UUID of the mount: $ blkid /dev/sdXX
• Partition and format if UUID is not assigned:
     $ fdisk /dev/sdXX
     $ mkdf.ext4 /dev/sdXX
• Add the details to fstab for persistant hard disk:
    $ sudo nano /etc/fstad
  _{/dev/sdb1 /disk1 ext4 default 1 2}

2. Installation of ELK (elasticsearch, logstash, kibana)

In the following section, we will discuss on the configuration of elasticsearch, logstash and kibana. We will layout the arious configuration changes to be done in the config files. The version we are working on currently is 7.12.

• Elasticsearch
  – Installation instructions: https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
  – Configuration changes on /etc/elasticsearch/elasticsearch.yml:

cluster.name: nullcluster network.host: 0.0.0.0 http.port: 9200 node.name: elasticsrv01 path:     data:            /mnt/hdd1/elasticsearch

yml file configuration

• Kibana
  – Installation instructions: https://www.elastic.co/guide/en/kibana/current/deb.html
  – We will forward the traffic to port https/443 from the default kibana port 5601:
    – Ref: https://blog.petermartyniak.com/changing-kibana-port-to-80-without-running-it-as-root/
    – Ref: https://gist.github.com/kimus/9315140
  
  $ sudo ufw enable
  $ sudo ufw allow 443
  $ sudo ufw allow 5601
  $ sudo nano /etc/ufw/before.rules

*nat :PREROUTING ACCEPT [0:0] :POSTROUTING ACCEPT [0:0] -A PREROUTING -p tcp –dport 443 -j REDIRECT –to-port 5601 COMMIT

• Logstash
  – Installation instructions: https://www.elastic.co/downloads/logstash
  – configuration for metricbeat to forward the logs through logstash:

$ sudo nano /etc/logstash/conf.d/20_metricbeat.yml

input {   beats {     port => 5056     ssl => true     ssl_key => ‘/etc/logstash/config/certs/logstash.pkcs8.key’     ssl_certificate => ‘/etc/logstash/config/certs/logstash01.crt’   } } output {   elasticsearch {     hosts => [“https://<domain name>:9200”]     cacert => ‘/etc/logstash/config/certs/ca.crt’     user => ‘logstash_internal’     password => ‘***’     index => “%{[@metadata][beat]}-%{[@metadata][version]}”   } }

• Configuration of heartbeat through logstash:
  
  $ sudo nano /etc/logstash/conf.d/01_heartbeat.conf

input {   beats {     port => 5055     ssl => true     ssl_key => ‘/etc/logstash/config/certs/logstash.pkcs8.key’     ssl_certificate => ‘/etc/logstash/config/certs/azlogstash01.crt’   } } output {   elasticsearch {     hosts => [“https://<domain name>:9200”]     cacert => ‘/etc/logstash/config/certs/ca.crt’     user => ‘logstash_internal’     password => ‘***’     index => “%{[@metadata][beat]}-%{[@metadata][version]}”   } }

• Heartbeat
  Installation Instructions: https://www.elastic.co/guide/en/beats/heartbeat/current/heartbeat-installation-configuration.html   
  
• Metricbeat
  – Installation Instructions: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-installation-configuration.html
  

3. Enabling security

In the following section, we will discuss how to accomplish secured access of kibana, elasticsearch and to ensure encrypted transmission of logs from beats agent to the log parsers.

• Minimal Security (passwords): https://www.elastic.co/guide/en/elasticsearch/reference/7.12/security-minimal-setup.html
  
  $ /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

$ sudo nano /etc/elasticsearch/elasticsearch.yml

“`
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

“`

• Transmission security using certificates: SSL/TLS – https://www.elastic.co/blog/configuring-ssl-tls-and-https-to-secure-elasticsearch-kibana-beats-and-logstash
  
  – Elasticsearch configuration updates:

xpack.security.http.ssl.enabled: true xpack.security.http.ssl.key: certs/elasticsrv01.key xpack.security.http.ssl.certificate: certs/elasticsrv01.crt xpack.security.http.ssl.certificate_authorities: certs/ca.crt xpack.security.transport.ssl.key: certs/elasticsrv01.key xpack.security.transport.ssl.certificate: certs/elasticsrv01.crt xpack.security.transport.ssl.certificate_authorities: certs/ca.crt xpack.security.transport.ssl.verification_mode: certificate

• HTTPS for Kibana:
  
  $ /usr/share/elasticsearch/bin/elasticsearch-certutil csr -name elasticsrv01 -dns <domain name 1>,< domain name 2>
  
  – Encrypt traffic between Kibana and Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup-https.html
    
  $ /usr/share/elasticsearch/bin/elasticsearch-certutil http
  
• Monitoring of the stack:
  
  •  Logstash Security:
    – Creating a logstash_write role: https://www.elastic.co/guide/en/logstash/current/ls-security.html
    – logstash certificate gen:
      $ openssl pkcs8 -in azlogstash04.key -topk8 -nocrypt -out azlogstash04.pkcs8.key
    – update permissions:
       $ chown root:logstash /etc/logstash/config/certs/azlogstash04.pkcs8.key
       $ chmod 640 /etc/logstash/config/certs/azlogstash04.pkcs8.key
    

4. Configuration of observability module of elastic

• Uptime
  – csv to yaml: https://onlineyamltools.com/convert-csv-to-yaml
  – Update yaml files in /etc/heartbeat/monitors.d/

• Network Devices
  • SNMP
  • Temperature
  • Last restarted time
  • Server devices
    • Metrics collection
      
• Elastiflow:
  • Official Guide: https://github.com/robcowart/elastiflow/blob/master/INSTALL.md
  • Medium post: https://medium.com/@ronaldbartels/a-guide-to-installing-elastiflow-53c915250df8

5. Alerting of elastic incidents using free module

• ElastAlert: https://github.com/johnsusek/elastalert-server
  
  persistent docker: guide- https://stackoverflow.com/questions/43671482/how-to-run-docker-compose-up-d-at-system-start-up
  $sudo systemctl enable docker
  `restart: always` to be added to docker-compose.yml
  $docker-compose up -d
  
  

6. Forwarding checkpoint firewall logs to elasticsearch

• Checkpoint:
  – Filebeat with checkpoint module is utilized
  – Checkpoint log exporter: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk122323
  – Filebeat configuration for different indices: https://alexmarquardt.com/2021/03/15/driving-filebeat-data-into-separate-indices-uses-legacy-index-templates/
  
  – Filebeat.yml configuration details:

output.elasticsearch:   # Array of hosts to connect to.   hosts: [“https://<domain name>:9200”]   username: “elastic”   password: “***”   ssl:    certificate_authorities: [“/etc/filebeat/config/certs/ca.crt”]    verification_mode: “certificate”   indices:    – index: “filebeat-%{[agent.version]}-%{[event.module]:other}” setup.template.enabled: true setup.template.name: “filebeat-%{[agent.version]}” setup.template.pattern: “filebeat-%{[agent.version]}-*” setup.template.fields: “fields.yml” setup.template.overwrite: false setup.ilm.enabled: false # we handle ILM in the cluster, so not defined here

• Creating Alias

PUT filebeat-7.12.0-checkpoint-000001{  “aliases”: {    “filebeat-7.12.0-checkpoint”: {      “is_write_index”: true    }  }}

• Define ILM policy `filebeat-checkpoint-policy`:
• Create high-order template and assign to the ILM policy:

PUT _template/filebeat-7.12.0-checkpoint-ilm{  “order”: 50,  “index_patterns”: [    “filebeat-7.12.0-checkpoint-*”  ],  “settings”: {    “index”: {      “lifecycle”: {        “name”: “filebeat-checkpoint-policy”,        “rollover_alias”: “filebeat-7.12.0-checkpoint”      }    }  }}

• Checkpoint log field description: https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk144192#SECURITY_LOGS
  

cp_log_export add name elasticlive_filebeat_exporter \ target-server 10.8.9.153 \ target-port 9520 \ protocol udp \ format syslog \ –apply-now

7. WARM HOST Configuration:

The Warm host can be utilized when we have data that are to be archived and are not required immediately. A general rule of thumb is to transfer data older than 3 days or more to a host with lesser configuration WARM host.

• Configuration changes:

node.name: extranode01 discovery.seed_hosts: [“<domain name 1>”,] cluster.initial_master_nodes: [“elasticsrv01”] node.attr.box_type: warm

• Renaming/Re-indexing pre-existing indexes:

PUT filebeat-7.12.0-checkpoint-backup {     “settings” : {         “index” : {             “number_of_shards” : 1,             “number_of_replicas” : 0         }     } } PUT filebeat-7.12.0-checkpoint-backup/_settings { “index.routing.allocation.require.temp”: “warm” }     POST /_reindex {   “source”: {     “index”: “filebeat-7.12.0-checkpoint”   },   “dest”: {     “index”: “filebeat-7.12.0-checkpoint-backup”   } }

8. Disk allocation changes:

1. Configuration:
   Explanation: https://www.elastic.co/guide/en/elasticsearch/reference/6.8/disk-allocator.html
   
   To obtain the current configuration details
   GET _cluster/settings?include_defaults&flat_settings

To change the disk allocation:

PUT _cluster/settings {   “transient”: {     “cluster.routing.allocation.disk.watermark.high”: “95%”   } }

ANNEXURE A: COMMON COMMANDS FOR TROUBLESHOOTING

• Linux
  – to monitor last logs:
  $ tail -f /var/log/elasticsearch/elk.log
  
• Metricbeat windows
  > .\metricbeat.exe setup -e -d “*”
  > .\metricbeat.exe modules list
  
• Elastiflow
  $ netstat -su
  $ tcpdump port 2055
  
• Heartbeat
  $ heartbeat -c heartbeat.yml -e -v
  $ heartbeat -e                                   //to send the output to standard