In this article, I will take you through the steps to Install Graylog on Ubuntu 20.04 LTS. In a Distributed Architecture, one of the main components that needs to be created is a robust Logging System. Without this, it would be very difficult to  diagnose any problem by trying to piece together logs from multiple sources. So a logging system built for Distributed Computing would be most ideal to use here. There are many logging solutions available in the open source world. One of the more popular choice is Graylog. Setting up a Graylog server is a simple process, requiring a MongoDB database and an Elasticsearch database to support it. More on Graylog Official Documentation.

Why Graylog Server

Graylog defines a JSON format called GELF for sending log data to its servers, and accepts a very flexible set of keys. Graylog servers can accept log streams from multiple sources and you can define post-processing actions as well, such as reformatting data and sending alerts based on user-defined rules.

How to Install Graylog on Ubuntu 20.04 LTS

Also Read: How to Install Grafana PCP Plugin on CentOS 8/Fedora 35

Step 1: Prerequisites

a) You should have a running Ubuntu 20.04 LTS Server.

b) You should have sudo or root access to run privileged commands.

c) You should have apt-get, apt-key and wget utility available in the System.

d) You should also have tee and pwgen utility available in your System.

 

Step 2: Install OpenJDK

Since Elasticsearch has a dependency of Java Platform so you need to first install OpenJDK and other required packages using below apt-get install command as shown below.

root@localhost:~# apt-get install apt-transport-https openjdk-8-jre-headless uuid-runtime pwgen
Reading package lists... Done
Building dependency tree
Reading state information... Done
uuid-runtime is already the newest version (2.34-0.1ubuntu9.1).
uuid-runtime set to manually installed.
The following package was automatically installed and is no longer required:
libllvm11
Use 'apt autoremove' to remove it.
The following additional packages will be installed:
ca-certificates-java java-common
Suggested packages:
default-jre fonts-dejavu-extra fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei fonts-wqy-zenhei
The following NEW packages will be installed:
apt-transport-https ca-certificates-java java-common openjdk-8-jre-headless pwgen
0 upgraded, 5 newly installed, 0 to remove and 47 not upgraded.
Need to get 28.2 MB of archives.
After this operation, 104 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
.......................................................

 

Step 3: Install MongoDB

In the next step, you need to install MongoDB Server. MongoDB is the most favorable choice for storing configuration data. It stores metadata information like User's Information or Stream Configuration.

a) Import Signed Key

First you need to import the signed key using below apt-key command.

root@localhost:~# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 9DA31620334BD75D9DCB49F368818C72E52529D4
Executing: /tmp/apt-key-gpghome.cI3bXFAZ0f/gpg.1.sh --keyserver hkp://keyserver.ubuntu.com:80 --recv 9DA31620334BD75D9DCB49F368818C72E52529D4
gpg: key 68818C72E52529D4: public key "MongoDB 4.0 Release Signing Key <packaging@mongodb.com>" imported
gpg: Total number processed: 1
gpg: imported: 1

b) Add Repository

Then add the MongoDB Repository using below command.

root@localhost:~# echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-4.0.list
deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 multiverse

c) Update the System

After adding the repository information, you need to update the package cache with all the package information from recently added repository using apt-get update command.

root@localhost:~# apt-get update
Ign:1 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 InRelease
Hit:2 http://in.archive.ubuntu.com/ubuntu focal InRelease
Hit:3 http://in.archive.ubuntu.com/ubuntu focal-updates InRelease
Get:4 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 Release [2,989 B]
Hit:5 http://in.archive.ubuntu.com/ubuntu focal-backports InRelease
Get:6 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 Release.gpg [801 B]
Get:7 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:8 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0/multiverse amd64 Packages [17.6 kB]
Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 DEP-11 Metadata [35.7 kB]
Get:10 http://security.ubuntu.com/ubuntu focal-security/universe amd64 DEP-11 Metadata [64.5 kB]
Get:11 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 DEP-11 Metadata [2,464 B]
Fetched 238 kB in 3s (77.6 kB/s)
Reading package lists... Done

d) Install MongoDB

In the next step, install MongoDB packages along with its dependencies using apt-get install -y mongodb-org command as shown below.

root@localhost:~# apt-get install -y mongodb-org
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
libllvm11
Use 'apt autoremove' to remove it.
The following additional packages will be installed:
mongodb-org-mongos mongodb-org-server mongodb-org-shell mongodb-org-tools
The following NEW packages will be installed:
mongodb-org mongodb-org-mongos mongodb-org-server mongodb-org-shell mongodb-org-tools
0 upgraded, 5 newly installed, 0 to remove and 47 not upgraded.
Need to get 73.8 MB of archives.
After this operation, 269 MB of additional disk space will be used.

e) Enable MongoDB Service

Then enable the MongoDB Service by using systemctl enable mongod.service command.

root@localhost:~# systemctl enable mongod.service
Created symlink /etc/systemd/system/multi-user.target.wants/mongod.service → /lib/systemd/system/mongod.service.

f) Restart MongoDB Service

Finally restart the service by using systemctl restart mongod command. You can verify the service status by using systemctl status mongod command.

root@localhost:~# systemctl restart mongod
root@localhost:~# systemctl status mongod
● mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2021-11-26 00:04:39 IST; 3s ago
Docs: https://docs.mongodb.org/manual
Main PID: 7414 (mongod)
Memory: 43.4M
CGroup: /system.slice/mongod.service
└─7414 /usr/bin/mongod --config /etc/mongod.conf

Nov 26 00:04:39 localhost systemd[1]: Started MongoDB Database Server.

 

Step 4: Install Elasticsearch

Next, you need to install Elasticsearch. Graylog uses Elasticsearch to store all the log data efficiently. The data gets stored in Apache Lucene indices as an inverted index, which makes it faster to search and hence an ideal solution for searching and analysis.

a) Download GPG Key

Here also you need to first download the Secure GPG Key from Elasticsearch Artifacts Page using below wget command.

root@localhost:~# wget -q https://artifacts.elastic.co/GPG-KEY-elasticsearch -O myKey

b) Add GPG Key

Then, add the GPG Key using apt-key add myKey command. If you see the status as OK on the output then it is added successfully.

root@localhost:~# apt-key add myKey
OK

c)  Add Repository

Like MongoDB, Elasticsearch is also not available from default Ubuntu Repository so you need to add separate repository to download the Elasticsearch packages using a package manager.

root@localhost:~# echo "deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main

d) Update Server

Now update the package cache with all the packages information from newly added repository using apt-get update command as shown below.

root@localhost:~# apt-get update
Hit:1 http://in.archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://in.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:3 http://in.archive.ubuntu.com/ubuntu focal-backports InRelease
Get:4 https://artifacts.elastic.co/packages/oss-7.x/apt stable InRelease [10.4 kB]
Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease
Ign:6 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 InRelease
Hit:7 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 Release
Get:8 https://artifacts.elastic.co/packages/oss-7.x/apt stable/main amd64 Packages [69.3 kB]
Get:10 https://artifacts.elastic.co/packages/oss-7.x/apt stable/main i386 Packages [56.4 kB]
Fetched 136 kB in 2s (54.4 kB/s)
Reading package lists... Done

e) Install Elasticsearch

Now the next logical thing to do is to install Elasticsearch by using apt-get install elasticsearch-oss command as shown below.

root@localhost:~# apt-get install elasticsearch-oss
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
libllvm11
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
elasticsearch-oss
0 upgraded, 1 newly installed, 0 to remove and 47 not upgraded.
Need to get 231 MB of archives.
After this operation, 420 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/oss-7.x/apt stable/main amd64 elasticsearch-oss amd64 7.10.2 [231 MB]

f) Configure Elasticsearch Configuration

After successful installation, you need to configure elasticsearch where you need to add below lines.

root@localhost:~# tee -a /etc/elasticsearch/elasticsearch.yml > /dev/null <<EOT
> cluster.name: graylog
> action.auto_create_index: false
> EOT

g) Reload Daemon

Then reload the daemon to take new configuration using systemctl daemon-reload command as shown below.

root@localhost:~# systemctl daemon-reload

h) Enable Elasticsearch Service

Then enable the service by using systemctl enable elasticsearch command as shown below.

root@localhost:~# systemctl enable elasticsearch
Synchronizing state of elasticsearch.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable elasticsearch
Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /lib/systemd/system/elasticsearch.service.

i) Restart Elasticsearch Service

Finally restart the service by using systemctl restart elasticsearch command. You can verify the status by using systemctl status elasticsearch command.

root@localhost:~# systemctl restart elasticsearch
root@localhost:~# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2021-11-26 00:20:21 IST; 4min 0s ago
Docs: https://www.elastic.co
Main PID: 8802 (java)
Tasks: 30 (limit: 2299)
Memory: 1.1G
CGroup: /system.slice/elasticsearch.service
└─8802 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+Alway>

 

Step 5: Install Graylog

Since Graylog is not available in the default Ubuntu Repository so first you need to download and install the Graylog repository package from where you can download the Graylog packages directly using a package Manager.

a) Download Graylog Repository

You can use wget or curl to download the latest Graylog repository .deb package. Here we are using wget utility to download the package in our local System.

root@localhost:~# wget https://packages.graylog2.org/repo/packages/graylog-4.2-repository_latest.deb
--2021-11-26 00:27:18-- https://packages.graylog2.org/repo/packages/graylog-4.2-repository_latest.deb
Resolving packages.graylog2.org (packages.graylog2.org)... 54.157.4.65, 54.91.6.89, 54.196.16.164, ...
Connecting to packages.graylog2.org (packages.graylog2.org)|54.157.4.65|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://graylog-package-repository.s3.eu-west-1.amazonaws.com/packages/graylog-4.2-repository_latest.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20211125T185720Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=AKIAIJSI6MCSPXFVDPIA%2F20211125%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Signature=c8c2c90fc79cff59d0d1a47aea77a405172c51b381c3739e3ce7ffcbf959ce06 [following]
--2021-11-26 00:27:20-- https://graylog-package-repository.s3.eu-west-1.amazonaws.com/packages/graylog-4.2-repository_latest.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20211125T185720Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=AKIAIJSI6MCSPXFVDPIA%2F20211125%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Signature=c8c2c90fc79cff59d0d1a47aea77a405172c51b381c3739e3ce7ffcbf959ce06
Resolving graylog-package-repository.s3.eu-west-1.amazonaws.com (graylog-package-repository.s3.eu-west-1.amazonaws.com)... 52.218.122.138
Connecting to graylog-package-repository.s3.eu-west-1.amazonaws.com (graylog-package-repository.s3.eu-west-1.amazonaws.com)|52.218.122.138|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2086 (2.0K) [application/vnd.debian.binary-package]
Saving to: ‘graylog-4.2-repository_latest.deb’

graylog-4.2-repository_latest.deb 100%[============================================================================>] 2.04K --.-KB/s in 0.006s

2021-11-26 00:27:21 (369 KB/s) - ‘graylog-4.2-repository_latest.deb’ saved [2086/2086]

b) Install Repository

Once downloaded, you can then install the package by using dpkg -i graylog-4.2-repository_latest.deb command as shown below.

root@localhost:~# dpkg -i graylog-4.2-repository_latest.deb
Selecting previously unselected package graylog-4.2-repository.
(Reading database ... 187495 files and directories currently installed.)
Preparing to unpack graylog-4.2-repository_latest.deb ...
Unpacking graylog-4.2-repository (1-4) ...
Setting up graylog-4.2-repository (1-4) ...

c) Update Server

Then update the package cache with all the package information from Graylog Repository. Without this step, package manager will not able to locate the package.

root@localhost:~# apt-get update
Hit:1 http://in.archive.ubuntu.com/ubuntu focal InRelease
Get:2 http://in.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:3 http://in.archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:5 https://artifacts.elastic.co/packages/oss-7.x/apt stable InRelease
Ign:6 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 InRelease
Hit:8 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 Release
Get:9 http://in.archive.ubuntu.com/ubuntu focal-updates/main amd64 DEP-11 Metadata [277 kB]
Get:10 http://in.archive.ubuntu.com/ubuntu focal-updates/universe amd64 DEP-11 Metadata [356 kB]
Get:11 http://in.archive.ubuntu.com/ubuntu focal-updates/universe DEP-11 64x64 Icons [383 kB]
Get:12 http://in.archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 DEP-11 Metadata [944 B]
Get:7 https://packages.graylog2.org/repo/debian stable InRelease [31.8 kB]
Get:13 http://in.archive.ubuntu.com/ubuntu focal-backports/universe amd64 DEP-11 Metadata [10.4 kB]
Get:15 https://packages.graylog2.org/repo/debian stable/4.2 i386 Packages [4,838 B]
Get:16 https://packages.graylog2.org/repo/debian stable/4.2 amd64 Packages [4,838 B]
Fetched 1,284 kB in 7s (188 kB/s)
Reading package lists... Done

d) Install Graylog Server

Once the repository information is fetched, you can install the package by using apt-get install graylog-server command as shown below.

root@localhost:~# apt-get install graylog-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
libllvm11
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
graylog-server
.....................
Unpacking graylog-server (4.2.1-1) ...
Setting up graylog-server (4.2.1-1) ...
################################################################################
Graylog does NOT start automatically!

Please run the following commands if you want to start Graylog automatically on system boot:

    sudo systemctl enable graylog-server.service

    sudo systemctl start graylog-server.service

################################################################################
Processing triggers for systemd (245.4-4ubuntu3.13) ...

e) Configure Graylog Server

You must set a secret to secure/pepper the stored user passwords. To generate one, use command pwgen -N 1 -s 96 as shown below.

root@localhost:~# pwgen -N 1 -s 96
ZiPyhwoT7sHiOWabJu4LQG3AIXMfo3uXaT9qdUXFVFSPkteca0tjCsqth0z9Vs4UpqIvJBAGo9znysQ7W5kvwf95HOBaYkqC

Then you must specify a hash password for the root user using below command. Just for the demo, here we are using password Test@123$. You are free to choose any strong password.

root@localhost:~# echo -n "Enter Password: " && head -1 </dev/stdin | tr -d '\n' | sha256sum | cut -d" " -f1
Enter Password: Test@123$
8eba3de05b01544fbcac2c412d053c9e602c680d53e78ddecb74017aeac93ae5

After generating both the password, open server.conf file using nano /etc/graylog/server/server.conf command and set below parameter.

root@localhost:~# nano /etc/graylog/server/server.conf
.................................................
password_secret = ZiPyhwoT7sHiOWabJu4LQG3AIXMfo3uXaT9qdUXFVFSPkteca0tjCsqth0z9Vs4UpqIvJBAGo9znysQ7W5kvwf95HOBaYkqC
..................................................
root_password_sha2 = 8eba3de05b01544fbcac2c412d053c9e602c680d53e78ddecb74017aeac93ae5
...................................................
http_bind_address = 0.0.0.0:9000

f) Reload Daemon

Then reload the daemon to take new configuration using systemctl daemon-reload command as shown below.

root@localhost:~# systemctl daemon-reload

g) Enable Graylog Server

If you want to start Graylog Server automatically after a crash or reboot then enable the service by using systemctl enable graylog-server command as shown below.

root@localhost:~# systemctl enable graylog-server
Synchronizing state of graylog-server.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable graylog-server
Created symlink /etc/systemd/system/multi-user.target.wants/graylog-server.service → /lib/systemd/system/graylog-server.service.

h) Start Graylog Server

Once all done, you can now start the service by using systemctl start graylog-server command. You can then verify the status using systemctl status graylog-server command. If it shows active and running then you are all good.

root@localhost:~# systemctl start graylog-server
root@localhost:~# systemctl status graylog-server
● graylog-server.service - Graylog server
Loaded: loaded (/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2021-11-26 00:39:46 IST; 5s ago
Docs: http://docs.graylog.org/
Main PID: 9976 (graylog-server)
Tasks: 14 (limit: 2299)
Memory: 62.9M
CGroup: /system.slice/graylog-server.service
├─9976 /bin/sh /usr/share/graylog-server/bin/graylog-server
└─9999 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:-OmitStackTraceInFastThrow -Djdk.tls.acknowledgeCloseNotify=true >

Nov 26 00:39:46 localhost systemd[1]: Started Graylog server.

 

Step 6: Open Graylog GUI

Go to your Favorite Browser and use URL http://<local_server_ip>:9000 to open the Graylog Login Page. In my case, local Server IP Address is 192.168.29.110 so I will use http://192.168.29.110:9000 URL in the browser. Once opened, it should show like below where it will ask to provide username and password. Username will be admin and password will be Test@123$ which you have set earlier. Then Click on Sign In.

Once Signed In, you should see a Search Page like below. This confirms the successful installation and working of Graylog Server. Now you can go ahead and configure the Server as per your requirement.