Skip to content

GitLab

Last edited by Andrei Pavel
Page history

Install Prometheus Grafana

Prometheus and Grafana installation

NOTE: These installation instructions describe a very early experiment the Stork team did with Prometheus and Grafana. Integration with both is planned for Stork 0.8.

The following page describes how to install Prometheus with Grafana and its dependencies on Ubuntu 18.04. Since Stork is in very early stages, it currently supports Ubuntu 18.04 only. We do not want to get distracted with portability. The list will be expanded some time in the future once the project becomes a bit more mature.

1. Prometheus

$ sudo apt install prometheus
$ sudo systemctl enable prometheus
$ sudo systemctl start prometheus

The config file is located in: /etc/prometheus/prometheus.yml. When started, by default Prometheus listens on localhost:9090.

Connect Prometheus to exporters

Define connection to Prometheus by editing /etc/prometheus/prometheus.yml:

apiVersion: 1

datasources:

# This reports statistics about Prometheus (how long it took to scrape data etc.)
# This is built into Prometheus itself.
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://localhost:9090

# Node exporter reports hardware/OS statistics for the node (the host bind & kea are run on)
# Requires node-exporter to run
  - job_name: 'node'
    static_configs:
      - targets: ['localhost:9100']

# Gathers statistics from Kea, requires kea-exporter
  - job_name: 'kea'
    static_configs:
      - targets: ['localhost:9547']

# Gathers statistics from bind9, requires bind-exporter
  - job_name: 'bind9'
    static_configs:
    - targets: ['localhost:9119']

Node exporter

To install Node exporter you need a go compiler version >= 1.11. Get resources:

$ go get github.com/prometheus/node_exporter

go to node_exporter directory:

$ cd ${GOPATH-$HOME/go}/src/github.com/prometheus/node_exporter

and build:

$ make

To start node exporter:

$ ./node_exporter

You should see the output like that:

INFO[0000] Starting node_exporter (version=0.16.0, branch=HEAD, revision=d42bd70f4363dced6b77d8fc311ea57b63387e4f)  source="node_exporter.go:82"
INFO[0000] Build context (go=go1.9.6, user=root@a67a9bc13a69, date=20180515-15:53:28)  source="node_exporter.go:83"
INFO[0000] Enabled collectors:                           source="node_exporter.go:90"
INFO[0000]  - boottime                                   source="node_exporter.go:97"
...
INFO[0000] Listening on :9100                            source="node_exporter.go:111"

By default exporter exposes metrics on localhost:9100. If the Node exporter is running you can verify that metrics are being exported by cURLing the /metrics endpoint:

$ curl http://localhost:9100/metrics

To manage statistics via Prometheus you must change the Prometheus configuration file, /etc/prometheus/prometheus.yml, by adding these lines to scrape_configs section:

- job_name: 'node'
  static_configs:
  - targets: ['localhost:9100']

BIND exporter

There is a BIND 9 exporter for Prometheus available publicly here. The Github page explains how to install from source:

go get github.com/digitalocean/bind_exporter
cd $GOPATH/src/github.com/digitalocean/bind_exporter
make

Note, this depends on prometheus/promu that requires at least Go version 1.11.

Run BIND 9. Make sure that named is built with libxml2 support.

named -V | grep libxml2

The named.conf configuration file (/etc/bind9/named.conf in case of Ubuntu) should at least be configured to emit statistics:

statistics-channels {
  inet 127.0.0.1 port 8053 allow { 127.0.0.1; };
};

After starting BIND 9, start the bind_exporter and it will collect statistics from the configured channel.

./bind_exporter
curl http://localhost:9119/metrics

Scrape with Prometheus

Add a new scrape job configuration for BIND 9:

scrape_configs:
  - job_name: 'bind9'
    static_configs:
    - targets: ['localhost:9119']

And restart Prometheus to see the BIND 9 metrics.

Kea exporter

Prometheus exporter capability is now provided natively by stork agent. Please install stork-agent.

2. Grafana

Installation from deb packages

# apt-get install -y software-properties-common
# wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
# add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
# apt-get update
# apt-get install grafana

To start the server: systemctl start grafana-server. If you want grafana to start automatically after system restart, also do systemctl enable grafana-server. The config is in /etc/grafana/grafana.ini. By default, grafana listens on localhost:3000.

More detailed instructions: https://grafana.com/docs/installation/debian/

Use Prometheus as datasource in Grafana

  1. Got to http://localhost:3000
  2. Login (if this is your first login, the default is admin/admin, please change your password!)
  3. Click Prometheus, use http://localhost:9090 as URL.

Add Stork dashboards

Stork provides several Grafana templates. Those can be easily imported into Grafana:

  1. Navigate to your Grafana UI and click Home button.
  2. Click Import dashboard
  3. Get the Stork templates. If you have Stork sources on your disk, the templates are in grafana/ directory. Alternatively, you can download them from stork gitlab.
  4. Click on upload .JSON file and upload the file.
  5. Some versions of Grafana require selecting the data source. Select prometheus.

3. Final set-up checks

You should have your environment ready. Here's a list of checks you can do to ensure your environment is sane:

  1. Make sure your prometheus is running. Go to http://stork.lab.isc.org:9090/targets and make sure you see bind 9, kea and node on the list and all of them are marked as up.
  2. Make sure your Grafana is running. Go to http://stork.lab.isc.org:3000, click home, click on Kea dashboard, or BIND9 dashboard.

You should see some statistics. If Stork is monitoring your live network - great, you're all set. If this is a test environment, you should probably go ahead an generate some simulated traffic. Head to page about generating some data.