Prometheus Overview and Setup

4 min readMar 25, 2018

Prometheus is an opensource monitoring solution that gathers time series based numerical data. It is a project which was started by Google’s ex-employees at SoundCloud.

To monitor your services and infra with Prometheus your service needs to expose an endpoint in the form of port or URL. For example:- {{localhost:9090}}. The endpoint is an HTTP interface that exposes the metrics.

For some platforms such as Kubernetes and skyDNS Prometheus act as directly instrumented software that means you don’t have to install any kind of exporters to monitor these platforms. It can directly monitor by Prometheus.

One of the best thing about Prometheus is that it uses Time Series Database(TSDB) because of that you can use mathematical operations, queries to analyze them. Prometheus uses sq-lite as a database but it keeps the monitoring data in volumes.

Pre-requisites

A CentOS VM or Machine
A non-root sudo user, preferably one named Prometheus.

Installing Prometheus Server

First, create a new directory to store all the files you download in this tutorial and move to it.

mkdir /opt/prometheus-setup
cd /opt/prometheus-setup

Create a user named “Prometheus”

useradd prometheus

Use wget to download the latest build of the Prometheus server and time-series database from GitHub.

wget https://github.com/prometheus/prometheus/releases/download/v2.0.0/prometheus-2.0.0.linux-amd64.tar.gz

The Prometheus monitoring system consists of several components, each of which needs to be installed separately.

Use tar to extract prometheus-2.0.0.linux-amd64.tar.gz:

tar -xvzf ~/opt/prometheus-setup/prometheus-2.0.0.linux-amd64.tar.gz .

Place your executable file somewhere in your PATH variable, or add them into the path for easy access.

mv prometheus-2.0.0.linux-amd64  prometheus

sudo mv  prometheus/prometheus  /usr/bin/

sudo chown prometheus:prometheus /usr/bin/prometheussudo chown -R prometheus:prometheus /etc/prometheussudo chown -R prometheus:prometheus /opt/prometheus-setup

Another thing I would recommend you to create a directory of Prometheus in /etc

mkdir /etc/prometheus
cp prometheus/prometheus.yml /etc/prometheus/

After this, you can execute a simple command to check Prometheus status

prometheus --version

You should see the following message on your screen:

prometheus,       version 2.0.0 (branch: HEAD, revision:       0a74f98628a0463dddc90528220c94de5032d1a0)
build user:       root@615b82cb36b6
build date:       20171108-07:11:59
go version:       go1.9.2

Create a service for Prometheus

sudo vi /etc/systemd/system/prometheus.service

Now add these lines to the files

[Unit]
Description=Prometheus

[Service]
User=prometheus
ExecStart=/usr/bin/prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /opt/prometheus-setup/

[Install]
WantedBy=multi-user.target

After this, you can reload the systemd daemon and start the Prometheus service.

systemctl daemon-reload

systemctl start prometheus

systemctl enable prometheus

Installing Node Exporter

Prometheus was developed for the purpose of monitoring web services. In order to monitor the metrics of your server, you should install a tool called Node Exporter. Node Exporter, as its name suggests, exports lots of metrics (such as disk I/O statistics, CPU load, memory usage, network statistics, and more) in a format Prometheus understands. Enter the Downloads directory and use wget to download the latest build of Node Exporter which is available on GitHub.

Node exporter is a binary which is written in go which monitors the resources such as CPU, ram, and filesystem.

wget https://github.com/prometheus/node_exporter/releases/download/v0.15.1/node_exporter-0.15.1.linux-amd64.tar.gztar -xvzf node_exporter-0.15.1.linux-amd64.tar.gz .mv node_exporter-0.15.1.linux-amd64 node-exporter

Now move the node_exporter binary file to the binary executable path

mv node-exporter/node_exporter /usr/bin/

Running Node Exporter as a Service

Create a user named “Prometheus” on the machine on which you are going to create node exporter service.

useradd prometheus

To make it easy to start and stop the Node Exporter, let us now convert it into a service. Use vi or any other text editor to create a unit configuration file called node_exporter.service.

sudo vi /etc/systemd/system/node_exporter.service

This file should contain the path of the node_exporter executable, and also specify which user should run the executable. Accordingly, add the following code:

[Unit]
Description=Node Exporter

[Service]
User=prometheus
ExecStart=/usr/bin/node_exporter

[Install]
WantedBy=default.target

Save the file and exit the text editor. Reload systemd so that it reads the configuration file you just created.

sudo systemctl daemon-reload

At this point, Node Exporter is available as a service which can be managed using the systemctl command. Enable it so that it starts automatically at boot time.

sudo systemctl enable node_exporter.service

You can now either reboot your server or use the following command to start the service manually:

sudo systemctl start node_exporter.service

Once it starts, use a browser to view Node Exporter’s web interface, which is available at http://your_server_ip:9100/metrics. You should see a page with a lot of text:

Starting Prometheus Server with a new node

Before you start Prometheus, you must first edit a configuration file for it called prometheus.yml.

vim /etc/prometheus/prometheus.yml

Copy the following code into the file.

Example Configuration File for Prometheus

After adding configuration in prometheus.yml. We should restart the service by

systemctl restart prometheus

This creates a scrape_configs section and defines a job called a node. It includes the URL of your Node Exporter’s web interface in its array of targets. The scrape_interval is set to 15 seconds so that Prometheus scrapes the metrics once every fifteen seconds. You could name your job anything you want, but calling it “node” allows you to use the default console templates of Node Exporter.

Use a browser to visit Prometheus’s homepage available at http://your_server_ip:9090. You’ll see the following homepage. Visit http://your_server_ip:9090/consoles/node.html to access the Node Console and click on your server, localhost:9100, to view its metrics.