# Monitoring Grafana
\
Grafana is an open-source platform for data analysis and monitoring. It enables you to query, visualize, and comprehend metrics from various systems, such as databases, servers, cloud services, and many others.
## Prometheus Install
#### 1 - Download latest software version
```
wget https://github.com/prometheus/prometheus/releases/download/v2.30.2/prometheus-2.30.2.linux-amd64
```
#### 2 - Unzip
```
tar -xvf prometheus-2.30.2.linux-amd64.tar.gz
```
#### 3 - Copying contents from the prometheus directory to the appropriate directory
```
sudo cp -r ./prometheus-2.30.2.linux-amd64 /usr/local/bin/prometheus
```
#### 4 - Remove downloaded files
```
rm prometheus-2.30.2.linux-amd64.tar.gz
rm -r prometheus-2.30.2.linux-amd64
```
### 5 - Creating service to run prometheus
#### 5.1 - Create Prometheus Service File
Create a file:
```
sudo vi /etc/systemd/system/prometheus.service
```
Put the following content inside it:
```
[Unit]
Description=Prometheus Service
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/prometheus/prometheus --config.file /usr/local/bin/prometheus/prometheus.yml
[Install]
WantedBy=multi-user.target
```
#### 5.2 - Reload daemons
```
sudo systemctl daemon-reload
```
#### 6 - Execute the prometheus
```
sudo systemctl start prometheus
```
#### 7 - Checking if prometheus is really running
```
sudo systemctl status prometheus
```
Something like this will appear:
```
● prometheus.service - Prometheus Service
Loaded: loaded (/etc/systemd/system/prometheus.service; disabled; vendor preset: enabled)
Active: active (running) since Sun 2021-10-03 04:55:46 UTC; 7min ago
Main PID: 2388 (prometheus)
Tasks: 6 (limit: 1071)
Memory: 22.2M
CGroup: /system.slice/prometheus.service
└─2388 /usr/local/bin/prometheus/prometheus --config.file /usr/local/bin/prometheus/prometheus.yml
```
#### 8 - Enabling service in boot
```
sudo systemctl enable prometheus
```
## Node exporter install
#### 1 - Download latest software version
```
wget https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz
```
#### 2 - Unzip
```
tar -xvf node_exporter-1.3.1.linux-amd64.tar.gz
```
#### 3 - Copying contents from the node\_exporter directory to the appropriate directory
```
sudo cp -r ./node_exporter-1.3.1.linux-amd64 /usr/local/bin/node_exporter
```
#### 4 - Remove downloaded files
```
rm node_exporter-1.3.1.linux-amd64.tar.gz
rm -r node_exporter-1.3.1.linux-amd64
```
### 5 - Creating service to run node\_exporter
#### 5.1 - Create Node exporter Service File
Create a file:
```
sudo vi /etc/systemd/system/node_exporter.service
```
Put the following content inside it:
```
[Unit]
Description=Node Exporter Service
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/node_exporter/node_exporter
[Install]
WantedBy=multi-user.target
```
#### 5.2 - Reload daemons
```
sudo systemctl daemon-reload
```
#### 6 - Execute the node\_exporter
```
sudo systemctl start node_exporter
```
#### 7 - Checking if node\_exporter is really running
```
sudo systemctl status node_exporter
```
Something like this will appear:
```
sudo systemctl status node_exporter
● node_exporter.service - Node Exporter Service
Loaded: loaded (/etc/systemd/system/node_exporter.service; disabled; vendor preset: enabled)
Active: active (running) since Sun 2021-10-03 05:23:18 UTC; 14min ago
Main PID: 2808 (node_exporter)
Tasks: 3 (limit: 1071)
Memory: 2.0M
CGroup: /system.slice/node_exporter.service
└─2808 /usr/local/bin/node_exporter/node_exporter
```
#### 8 - Enabling service in boot
```
sudo systemctl enable node_exporter
```
## Grafana Install
#### 1 - Download latest software version and install
```
sudo apt-get install -y adduser libfontconfig1
wget https://dl.grafana.com/oss/release/grafana_8.1.5_amd64.deb
sudo dpkg -i grafana_8.1.5_amd64.deb
```
#### 2 - Remove downloaded package
```
rm grafana_8.1.5_amd64.deb
```
#### 3 - Start grafana-server service
```
sudo systemctl start grafana-server
```
#### 4 - Checking if grafana-server is really running
```
sudo systemctl status grafana-server
```
Something like this will appear:
```
● grafana-server.service - Grafana instance
Loaded: loaded (/lib/systemd/system/grafana-server.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2021-10-03 23:40:30 UTC; 2h 41min ago
Docs: http://docs.grafana.org
Main PID: 7047 (grafana-server)
Tasks: 11 (limit: 4705)
Memory: 32.7M
CGroup: /system.slice/grafana-server.service
└─7047 /usr/sbin/grafana-server --config=/etc/grafana/grafana.ini --pidfile=/var/run/grafana/grafana-server.pid --packaging=deb
cfg:default.paths.logs=/var/log/grafana cfg:default.paths.data=/var/lib/grafana cfg:default.paths.plugins=/var/lib/grafana/plugins cfg:defaul
t.paths.provisioning=/etc/grafana/provisioning
```
#### 5 - Enabling service in boot
```
sudo systemctl enable grafana-server
```
#### 6 - Access Grafana
Open Grafana at the following URL:
* Note: Open port 3000 in your firewall
* Grafana's default credentials are:
* username: admin password: admin
Note: a good security practice is to change the admin user password after the first login.
#### 7 - Define Data Source
Select Data source option in Configuration section in left panel.
Select Prometheus datasource type.
Define Prometheus name and URL.
URL: [http://localhost:9090](http://localhost:9090/)
Click in Save and test button.
#### 8 - Import Dashboard
Upload file and click in load button.
\\
Define name of Dashboard, select folder and click in import button.
If everything happens normally, your dashboard will appear on the screen.
## Grafana Alerts
#### 1 - Access the Alert menu and then click Admin
#### 2 - Click the "Add Alertmanager" button
#### 3 - Set the URL where your external alertmanager instance is installed and click on the "Add alertmanagers" button
#### 4 - Access the panel inside the Dashboard that you want to add the alert, right-click on the panel and then click "Edit"
#### 5 - Click the "Alert" tab and click the "Create alert rule from this panel" button to create an alert with a direct link to this panel.
#### 6 - Define the name of the alert, the folder, the function, select the query and the limit that will be used for the alert.
#### 7 - Set the field "evaluate every" and "for", if necessary add a custom label and then click "Save and exit"
## Nginx Install and enable HTTPS connections with Let's Encrypt
```
sudo apt install nginx
```
Enable Nginx in boot:
```
sudo systemctl enable nginx
```
Certbot snap version install:
```
sudo snap install core; sudo snap refresh core
sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot
sudo certbot --nginx
```
Configure /etc/nginx/sites-enabled/default :
```
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
proxy_pass http://localhost:3000;
}
location /api/live {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $http_host;
proxy_pass http://localhost:3000/;
}
location /grafana/ {
proxy_pass http://localhost:3000/;
proxy_set_header Host $Host;
proxy_max_temp_file_size 0;
}
```
Configure /etc/grafana/grafana.ini :
```
[server]
root_url = %(protocol)s://%(domain)s:/grafana
serve_from_sub_path = true
```
Restart nginx and grafana-server services :
```
sudo systemctl restart grafana-server
sudo systemctl restart nginx
```
## Configuring names in hosts file
#### 1 - Edit /etc/hosts file
Define the names that will be used for each host, and put them in the file /etc/hosts, follow exemple:
```
sudo nano /etc/hosts
nameofhost1 10.0.0.1
nameofhost2 10.0.0.2
nameofhost3 10.0.0.3
```
#### 2 - Test configuration with ping command
```
ping nameofhost
```
Something like this will appear:
```
PING localhost (10.0.0.1) 56(84) bytes of data.
64 bytes from ip6-localhost (10.0.0.1): icmp_seq=1 ttl=64 time=0.012 ms
64 bytes from ip6-localhost (10.0.0.1): icmp_seq=2 ttl=64 time=0.042 ms
```
if the IP address that returns after the command matches the name you put in the /etc/hosts file, the configuration is correct
Now in the configuration of prometheus you can configure the host name instead of the IP
```
- job_name: "Validators"
static_configs:
- targets: ["nameofhost1:9100","nameofhost2:9100","nameofhost3:9100"]
```
## GitHub OAuth2 Authentication
#### 1 - Register your application with GitHub
To enable the GitHub OAuth2 you must register your application with GitHub. GitHub will generate a client ID and secret key for you to use. When you create the application you will need to specify a callback URL. Specify this as callback:
```
https://myrealdomain.com/grafana/login/github
```
Access your organization
Access Settings of your organization
Access Developer section and select OAuth App
Click in "New OAuth App"
Define OAuth app info
Get Client ID and generate new client secret
Store your Client ID and Client secret for alertmanager configuration
#### 2 - Set root\_url in grafana.ini file
```
sudo nano /etc/grafana/grafana.ini
[server]
root_url = https://myrealdomain.com/grafana
serve_from_sub_path = true
```
#### 3 - Enable GitHub in Grafana in grafana.ini file
```
sudo nano /etc/grafana/grafana.ini
[auth.github]
enabled = true
allow_sign_up = true
client_id = YOUR_GITHUB_APP_CLIENT_ID
client_secret = YOUR_GITHUB_APP_CLIENT_SECRET
scopes = user:email,read:org
auth_url = https://github.com/login/oauth/authorize
token_url = https://github.com/login/oauth/access_token
api_url = https://api.github.com/user
# space-delimited organization names
allowed_organizations = organization1 organization2
```
#### 4 - Restart grafana-server service
```
sudo systemctl restart grafana-server
```
#### 5 - After logout we have a login option with github
