Monitoring logs

Centralized Logs with Loki and Promtail

Loki install

1 - Download loki

wget https://github.com/grafana/loki/releases/download/v2.8.0/loki-linux-amd64.zip

2 - Download Loki example configuration file

wget https://raw.githubusercontent.com/grafana/loki/master/cmd/loki/loki-local-config.yaml

3 - Create directory for Loki application

sudo mkdir /usr/local/bin/loki

4 - Unzip loki

unzip loki-linux-amd64.zip

5 - Copy files to Loki application directory

sudo cp loki-linux-amd64 /usr/local/bin/loki
sudo cp loki-local-config.yaml /usr/local/bin/loki

6 - Create service file

sudo vi /etc/systemd/system/loki.service

Put this content in the file and save

[Unit]
Description=Loki Service
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/loki/loki-linux-amd64 -config.file=/usr/local/bin/loki/loki-local-config.yaml

[Install]
WantedBy=multi-user.target

7 - Reload daemons

sudo systemctl daemon-reload

8 - Execute the loki

sudo systemctl start loki

9 - Checking if loki is really running

sudo systemctl status loki

10 - Enabling service in boot

sudo systemctl enable loki

Promtail install

1 - Download Promtail

wget https://github.com/grafana/loki/releases/download/v2.4.2/promtail-linux-amd64.zip

2 - Download Promtail example configuration file

wget https://raw.githubusercontent.com/grafana/loki/main/clients/cmd/promtail/promtail-local-config.yaml

3 - Create directory for Promtail application

sudo mkdir /usr/local/bin/promtail

4 - Unzip promtail

unzip promtail-linux-amd64.zip

5 - Copy files to Promtail application directory

sudo cp promtail-linux-amd64 /usr/local/bin/promtail
sudo cp promtail-local-config.yaml /usr/local/bin/promtail

6 - Configuring the Promtail configuration file

Open file

sudo nano /usr/local/bin/promtail/promtail-local-config.yaml

Set the address of the Loki Server that will receive the Log stream, and the path of the files that will be sent.

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: <LABEL NAME OF YOUR JOB>
      __path__: <PATH OF YOUR LOG FILE>

7 - Create service file

sudo vi /etc/systemd/system/promtail.service

Put this content in the file and save

[Unit]
Description=Promtail Service
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/promtail/promtail-linux-amd64 -config.file=/usr/local/bin/promtail/promtail-local-config.yaml

[Install]
WantedBy=multi-user.target

8 - Reload daemons

sudo systemctl daemon-reload

9 - Execute the promtail

sudo systemctl start promtail

10 - Checking if promtail is really running

sudo systemctl status promtail

11 - Enabling service in boot

sudo systemctl enable promtail

Dashboards Loki query based

1 - In "Dashboards" menu click in "Browse"

2 - Click in "New Dashboard" button or click on an existing dashboard

3 - Click in "Add new panel" button

4 - Select Loki datasource

5 - Define the content filter in Loki query

6 - Define title and description of panel and click in save button