This is Part 2 of my tutorial series on ELK on CentOS 7

  • Part 1 - Operating System, Java and Tweaks
  • Part 2 (This Site) - Elasticsearch
  • Part 3 - Kibana
  • Part 4 - Logstash
  • Part 5 - Filebeat
  • Part 6 - Securing and Clean Up
  • Part 7 - Extending the cluster

If you have followed part one of the tutorial, you should have CentOS 7 with Oracle Java 8 installed. You also should have configured a sudo user, lsof and nano.

Unless you have good reasons to do otherwise, I would recommend to use their repository since that allows you to get regular updates. For further reference and alternate installation options, check out Elastic's documentation on this.

Install Elasticsearch

First, we will need to import Elastic's GPG Key:

$ sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Next, we need to create a file called elasticsearch.repo in the folder /etc/yum.repos.d/. (This is for all RedHat based distributions such as CentOS, Fedora etc. For OpenSUSE it's /etc/zypp/repos.d/).

$ sudo nano /etc/yum.repos.d/elasticsearch.repo

Copy the following content to the file, save and exit:

[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

On CentOS and RedHat use this command to install:

$ sudo yum install elasticsearch -y

For Fedora:

$ sudo dnf install elasticsearch

For OpenSUSE:

$ sudo zypper install elasticsearch

Next, you want to configure Elasticsearch to start as a daemon on system startup:

CentOS uses systemd to do that. So the steps are:

$ sudo systemctl daemon-reload
$ sudo systemctl enable elasticsearch.service

Now start the service:

$ sudo systemctl start elasticsearch.service

By default, Elasticsearch does not tell you, if there's any problem with the daemon in syslog. This information instead goes to log files in /var/log/elasticsearch/.

You also won't be able to access the folder /var/log/elasticsearch/ properly with the sudo user. To double check the logs you might want to login as root or switch over to root with the command su -.

To make sure, elasticseacrch is running, first check the network-socket:

$ sudo lsof -Pni|grep elastic

You should see something like:

java     11779 elasticsearch  197u  IPv6 128929      0t0  TCP [::1]:9300 (LISTEN)
java     11779 elasticsearch  198u  IPv6 128938      0t0  TCP 127.0.0.1:9300 (LISTEN)
java     11779 elasticsearch  211u  IPv6 128957      0t0  TCP 127.0.0.1:9200 (LISTEN)
java     11779 elasticsearch  212u  IPv6 127904      0t0  TCP [::1]:9200 (LISTEN)

You may notice that Elasticsearch is only listening on localhost. We will keep it this way for now (this will change in cluster configurations).

Another test is to query the interface:

$ curl -X GET "localhost:9200/"

This should result in something like:

{
  "name" : "Pb3EVXQ",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Ex84vyPXRiqdoHIEgqDo2Q",
  "version" : {
    "number" : "6.4.2",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "04711c2",
    "build_date" : "2018-09-26T13:34:09.098244Z",
    "build_snapshot" : false,
    "lucene_version" : "7.4.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

As long as we're running the whole stack on one host, there's nothing really to configure. However I would recommend to already set a few variables such as the clustername and other options.

Open the file /etc/elasticsearch/elasticsearch.yml and change the following lines (note that the lines are spread around, so the configuration file will not look as condensed as I show here. Use whatever you think for the names, but of course they should make sense in your environment.

cluster.name: elastic_cluster
node.name: ${HOSTNAME}

Once you start using Elasticsearch in a production cluster, there will be a few more options to be considered, which I will cover in one of the next tutorials.

Disable Swapping

Swapping is really terrible for Elasticsearch, so we will disable it completely:

$ sudo swapoff -a

Next, we need to switch swapping off in fstab. Open up /etc/fstab. Be careful what you do here, in doubt dismiss any changes, close the file and reopen.

$ sudo nano /etc/fstab

If you see any lines with the word swap in them, simply comment them out by adding a # in front of the line.

Save and exit.

Tuning

In general, Elastic's default configuration is fine for our testing environment. The only setting you should adjust is the JVM heap size. This should roughly be half of your system memory.

To find out your available system memory, type

$ sudo free -m -h

Next, open the file /etc/elasticsearch/jvm.options:

$ sudo nano /etc/elasticsearch/jvm.options

Find the lines:

-Xms512m
-Xmx512m

Note that the number is probably set to accommodate your system RAM size upon installation, it could also read something like:

-Xms1G
-Xmx1G

Set it to half of your system RAM. Save, close and restart elasticsearch:

$ sudo systemctl restart elasticsearch.service

Conclusion

We now have a running Elasticsearch core. Next step will be to install a Kibana frontend.