Blog

In this chapter, we will discuss the security and monitoring aspects of Logstash.

Monitoring

Logstash is a very good tool to monitor the servers and services in production environments. Applications in production environment produces different kinds of log data like access Logs, Error Logs, etc. Logstash can count or analyze the number of errors, accesses or other events using filter plugins. This analysis and counting can be used for monitoring different servers and their services.

Logstash offers plugins like HTTP Poller to monitor the website status monitoring. Here, we are monitoring a website named mysite hosted on a local Apache Tomcat Server.

logstash.conf

In this config file, the http_poller plugin is used to hit the site specified in the plugin after a time interval specified in interval setting. Finally, it writes the status of the site to a standard output.

input {
   http_poller {
      urls => {
         site => "http://localhost:8080/mysite"
      }
      request_timeout => 20
      interval => 30
      metadata_target => "http_poller_metadata"
   }
}
output {
   if [http_poller_metadata][code] == 200 {
      stdout {
         codec => line{format => "%{http_poller_metadata[response_message]}"}
      }
   }
   if [http_poller_metadata][code] != 200 {
      stdout {
         codec => line{format => "down"}
      }
   }
}

Run logstash

We can run Logstash with the following command.

>logstash –f logstash.conf

stdout

If the site is up, then the output will be −

Ok

If we stop the site by using the Manager App of Tomcat, the output will change to −

down

Security

Logstash provides plenty of features for secure communication with external systems and supports authentication mechanism. All Logstash plugins support authentication and encryption over HTTP connections.

Security with HTTP protocol

There are settings like user and password for authentication purposes in various plugins offered by Logstash like in the Elasticsearch plugin.

elasticsearch {
   user => <username>
   password => <password>
}

The other authentication is PKI (public key infrastructure) for Elasticsearch. The developer needs to define two settings in the Elasticsearch output plugin to enable the PKI authentication.

elasticsearch {
   keystore => <string_value>
   keystore_password => <password>
}

In the HTTPS protocol, a developer can use the authority’s certificate for SSL/TLS.

elasticsearch {
   ssl => true
   cacert => <path to .pem file>
}

Security with Transport Protocol

To use the transport protocol with Elasticsearch, users need to set protocol setting to transport. This avoids un-marshalling of JSON objects and leads to more efficiency.

The basic authentication is same as performed in http protocol in Elasticsearch output protocol.

elasticsearch {
   protocol => “transport”
   user => <username>
   password => <password>
}

The PKI authentication also needs the SSL sets to be true with other settings in the Elasticsearch output protocol −

elasticsearch {
   protocol => “transport”
   ssl => true
   keystore => <string_value>
   keystore_password => <password>
}

Finally, the SSL security requires a little with more settings than other security methods in communication.

elasticsearch {
   ssl => true
   ssl => true
   keystore => <string_value>
   keystore_password => <password>
   truststore => 
   truststore_password => <password>
}

Other Security Benefits from Logstash

Logstash can help input system sources to prevent against attacks like denial of service attacks. The monitoring of logs and analyzing the different events in those logs can help system administrators to check the variation in the incoming connections and errors. These analyses can help to see if the attack is happening or going to happen on the servers.

Other products of the Elasticsearch Company such as x-pack and filebeat provides some functionality to communicate securely with Logstash.

Learn More : Click Here