Deploying Grafana Loki for Centralized Logging on VPS: Promtail, Nginx, Systemd
TL;DR
In this detailed guide, we will step-by-step set up a centralized log collection and analysis system on your VPS using the Grafana Loki stack. You will learn how to install and configure Loki for log storage, Promtail for collecting logs from your applications, and Nginx as a reverse proxy with HTTPS for secure access. As a result, you will get a powerful and scalable solution for efficient monitoring of events on your server and applications.
- Installing Grafana Loki and Promtail as Systemd services.
- Configuring Nginx as a reverse proxy for Loki with HTTPS support via Certbot.
- Configuring Promtail to collect logs from various sources and send them to Loki.
- Ensuring secure access to the Loki UI via Nginx and TLS.
- Step-by-step instructions for server preparation, installation, configuration, backups, and troubleshooting.
What we are setting up and why
In today's world, where applications and services generate huge volumes of data, centralized logging becomes not just a convenience, but a necessity. We will be deploying Grafana Loki — a horizontally scalable, highly available, multi-tenant log aggregation system developed by Grafana Labs. Loki differs from other logging systems in that it indexes only log metadata (labels), while storing the logs themselves in a compressed form. This makes it very efficient in terms of resource usage and storage cost.
As part of this tutorial, we will set up a complete stack for centralized collection, storage, and access to logs:
- Grafana Loki: The core of the system, which accepts logs, indexes them by labels, and stores them.
- Promtail: An agent installed on each server where logs are generated. It monitors specified log files, applies labels to them, and sends them to Loki.
- Nginx: Will act as a reverse proxy for Loki, providing secure access via HTTPS and the ability to host Loki on standard ports.
- Systemd: We will use Systemd to manage Loki and Promtail services, ensuring their automatic startup and monitoring.
Ultimately, the reader will get a fully functional centralized logging system. This will allow collecting logs from all your applications and services on a VPS (or multiple VPSs), viewing them in a convenient Grafana interface (which can be connected to Loki), quickly finding necessary events, debugging problems, and monitoring system status. This significantly simplifies infrastructure management, especially when you have multiple services or microservices deployed on one or different servers.
Alternatives: Cloud-managed vs. Self-hosted on VPS
There are several approaches to centralized logging:
- Cloud-managed services (SaaS): Platforms such as Datadog, Splunk Cloud, Logz.io, AWS CloudWatch, Google Cloud Logging, Azure Monitor. They offer ready-made solutions with minimal setup, high availability, scalability, and rich functionality. Pros: Ease of use, no need to manage infrastructure, powerful analytical tools. Cons: High cost, especially with large log volumes; potential data privacy concerns, as logs are stored with a third-party provider; less control over the infrastructure.
- Self-hosted solutions on a dedicated server/VPS: This is the approach we are implementing with Loki. You have complete control over the entire system, from infrastructure to configuration. Pros: Full control over data and security, potentially significantly lower cost for large log volumes (paying only for VPS/dedicated), flexibility in customization for specific needs. Cons: Requires technical knowledge for deployment and maintenance, responsibility for availability and scaling lies with you.
Choosing a self-hosted solution on a VPS with Loki is particularly relevant for VPS owners, developers, solo founders, and enthusiasts who want full control over their infrastructure, optimize costs, and not depend on third-party cloud providers. Loki, thanks to its design, is an excellent choice for this task, as it is less resource-intensive compared to other systems like the ELK stack (Elasticsearch, Logstash, Kibana), especially in terms of RAM and disk space consumption due to indexing only metadata.
What VPS configuration is needed for this task
Choosing the right VPS for Grafana Loki depends on the volume of logs you plan to collect and their retention period. Loki efficiently compresses logs, but still requires sufficient disk space and moderate CPU/RAM resources for indexing and queries.
Minimum requirements for Loki and Promtail (for small projects, up to 100 GB logs per month)
- CPU: 2 cores. Loki and Promtail do not heavily load the processor in idle mode, but peak loads during indexing or complex queries may require more.
- RAM: 4 GB. Loki can be quite memory-hungry with large query volumes or long-term index storage in memory. Promtail consumes little.
- Disk: NVMe SSD from 100 GB. Disk speed is critical for writing logs and executing queries. Volume depends on log volume and retention policy. For 100 GB of logs per month and 3 months of storage, about 300 GB will be needed.
- Network: 100 Mbps. For transferring logs from Promtail to Loki and accessing the UI.
Recommended VPS plan for medium projects (up to 500 GB logs per month, multiple sources)
For more serious tasks, where you collect logs from several applications or servers, and expect up to 500 GB of logs per month, you should consider a more powerful configuration:
- CPU: 4 cores.
- RAM: 8 GB.
- Disk: NVMe SSD from 500 GB to 1 TB.
- Network: 1 Gbps.
Such a plan will provide sufficient performance for processing queries and storing a larger volume of data. You can consider a VPS with the specified characteristics for deploying your centralized logging system.
When a dedicated server is needed, not a VPS
A dedicated server becomes preferable when:
- Very large log volumes: If you plan to collect terabytes of logs per month (e.g., from a large Kubernetes cluster or many microservices).
- High query performance: Instant access to logs and execution of complex analytical queries on huge data arrays are required.
- Guaranteed resources: You need 100% guaranteed CPU, RAM, and disk resources, without the "noisy neighbor" effect sometimes found on VPS.
- Strict storage requirements: Need for RAID arrays to increase disk subsystem reliability or specific disk type requirements (e.g., high-performance HDDs for archival log storage, together with SSDs for hot data).
- Physical isolation: To comply with strict security standards and compliance (e.g., PCI DSS, HIPAA), where complete physical isolation is required.
For most individual developers, solo founders, and small teams, a VPS will be more than sufficient and more cost-effective. However, if you see that your log volume exceeds 1-2 TB per month, or VPS performance begins to suffer, transitioning to a suitable dedicated server will be justified.
VPS Location: What it affects
The choice of VPS location affects several key aspects:
- Latency: The closer the VPS is to the log sources (your other servers, application clients), the lower the latency for log transmission. This is important for operational monitoring and reducing network load.
- Legal compliance: Some jurisdictions have strict data storage requirements, including logs. Ensure that the chosen location complies with applicable data privacy laws (e.g., GDPR in Europe).
- Cost: VPS prices can vary depending on the region.
- Availability: Choose regions with good network infrastructure and high data center availability.
Ideally, choose a location that is geographically close to your main log-generating servers to minimize network latency and ensure fast data transfer.
Server preparation
Server Preparation
Before proceeding with the installation of Loki and Promtail, you need to perform basic setup of your fresh VPS. This will enhance security and usability.
It is assumed that you are using Ubuntu Server 22.04 LTS or 24.04 LTS distribution, which is a popular choice for VPS.
1. SSH Connection and System Update
Connect to your server via SSH using the credentials provided by your provider. This is usually the root user and password.
ssh root@YOUR_IP_ADDRESS
After successful login, update the package list and installed packages to their latest versions:
sudo apt update && sudo apt upgrade -y
2. Creating a New User with Sudo Privileges
Working as the root user is insecure. Let's create a new user and grant them sudo privileges.
adduser lokiadmin
usermod -aG sudo lokiadmin
Now, exit the root session and log in as the new user:
exit
ssh lokiadmin@YOUR_IP_ADDRESS
3. Configuring SSH Key Authentication
To enhance security, it is recommended to use SSH keys instead of passwords. If you don't have keys yet, generate them on your local machine:
ssh-keygen -t rsa -b 4096
Copy the public key to the server (replace lokiadmin and YOUR_IP_ADDRESS):
ssh-copy-id lokiadmin@YOUR_IP_ADDRESS
Now you can disable password authentication for SSH. Edit the file /etc/ssh/sshd_config:
sudo nano /etc/ssh/sshd_config
Find and change the following lines:
#PasswordAuthentication yes
PasswordAuthentication no
#PermitRootLogin yes
PermitRootLogin no
Save changes (Ctrl+O, Enter) and exit (Ctrl+X). Restart the SSH service:
sudo systemctl restart sshd
After this, ensure that you can log in with your SSH key before closing the current session.
4. Firewall Configuration (UFW)
UFW (Uncomplicated Firewall) is a convenient wrapper for iptables. Install and configure it:
sudo apt install ufw -y
sudo ufw allow OpenSSH
sudo ufw allow http
sudo ufw allow https
sudo ufw enable
sudo ufw status
Check the status. Ports 22 (SSH), 80 (HTTP), and 443 (HTTPS) should be allowed. We will add a port for Loki later.
5. Installing Fail2Ban
Fail2Ban helps protect against brute-force attacks by blocking IP addresses that make too many failed login attempts.
sudo apt install fail2ban -y
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
Fail2Ban protects SSH by default. You can configure it by editing the file /etc/fail2ban/jail.local (create it if it doesn't exist) and adding your rules.
6. Installing Basic Utilities
Install several useful utilities that will come in handy during setup and debugging:
sudo apt install curl wget git nano htop unzip -y
Your server is now ready for deploying Grafana Loki and Promtail.
Software Installation — Step-by-step
We will install Loki and Promtail by downloading binary files from official GitHub repositories and configuring them as Systemd services. Nginx will be installed from the APT repository.
The actual software versions for 2026 will be conditional, but we will use the latest stable releases available at the time of generation.
Installing Grafana Loki
Let's create directories for Loki and download the current binary file.
# 1. Create directories for Loki
sudo mkdir -p /etc/loki
sudo mkdir -p /var/lib/loki
sudo chown -R lokiadmin:lokiadmin /var/lib/loki # Grant write permissions to the current user
# 2. Change to a temporary directory
cd /tmp
# 3. Download the latest stable version of Loki (e.g., v3.1.0 for 2026)
# Always check the latest version on GitHub: https://github.com/grafana/loki/releases
LOKI_VERSION="3.1.0" # Specify the current version
wget "https://github.com/grafana/loki/releases/download/v${LOKI_VERSION}/loki-linux-amd64.zip"
# 4. Unzip the archive
unzip "loki-linux-amd64.zip"
# 5. Move the binary file to /usr/local/bin
sudo mv loki-linux-amd64 /usr/local/bin/loki
# 6. Ensure the file is executable
sudo chmod a+x /usr/local/bin/loki
# 7. Check the version
loki --version
Now, let's create a loki user, under whom the service will run, to ensure the principle of least privilege.
# 1. Create a system user loki without a home directory and shell
sudo useradd --system --no-create-home --shell /bin/false loki
# 2. Transfer ownership of the Loki data directory to the loki user
sudo chown -R loki:loki /var/lib/loki
Installing Promtail
Similar to Loki, let's download and install Promtail.
# 1. Create a directory for Promtail
sudo mkdir -p /etc/promtail
# 2. Change to a temporary directory
cd /tmp
# 3. Download the latest stable version of Promtail (e.g., v3.1.0 for 2026)
# Always check the latest version on GitHub: https://github.com/grafana/loki/releases
PROMTAIL_VERSION="3.1.0" # Specify the current version
wget "https://github.com/grafana/loki/releases/download/v${PROMTAIL_VERSION}/promtail-linux-amd64.zip"
# 4. Unzip the archive
unzip "promtail-linux-amd64.zip"
# 5. Move the binary file to /usr/local/bin
sudo mv promtail-linux-amd64 /usr/local/bin/promtail
# 6. Ensure the file is executable
sudo chmod a+x /usr/local/bin/promtail
# 7. Check the version
promtail --version
Let's create a promtail user to run the service.
# 1. Create a system user promtail without a home directory and shell
sudo useradd --system --no-create-home --shell /bin/false promtail
Installing Nginx and Certbot
Nginx will act as a reverse proxy and provide HTTPS access to Loki. Certbot will help automate the issuance and renewal of TLS certificates from Let's Encrypt.
# 1. Install Nginx from Ubuntu repositories
sudo apt install nginx -y
# 2. Start Nginx and enable it to start on boot
sudo systemctl start nginx
sudo systemctl enable nginx
# 3. Install Certbot and the Nginx plugin
sudo apt install certbot python3-certbot-nginx -y
At this stage, the main components are installed. Now let's proceed to their configuration.
Configuration
Proper configuration is key to stable and efficient system operation. We will configure Loki, Promtail, Nginx, and Certbot.
Loki Configuration
Let's create a configuration file for Loki /etc/loki/config.yaml. This file defines how Loki will store logs, which ports to listen on, and other parameters.
sudo nano /etc/loki/config.yaml
Insert the following content:
auth_enabled: false # For simplicity, authentication is disabled. For production, consider basic_auth or OIDC
server:
http_listen_port: 3100 # Port on which Loki will listen for HTTP requests
grpc_listen_port: 9095 # Port for gRPC (used by Promtail)
common:
path_prefix: /var/lib/loki/wal # Path for Write-Ahead Log
replication_factor: 1 # Number of replicas for data storage
ring:
instance_addr: 127.0.0.1
kvstore:
store: inmemory # For a simple installation, we use inmemory; for production, consider Consul/Etcd
replication_factor: 1
schema_config:
configs:
- from: 2020-10-24 # Start date for using this schema
store: boltdb-shipper # Use boltdb-shipper for index storage
object_store: filesystem # Store log chunks on the filesystem
schema: v12
index:
prefix: index_
period: 24h
storage_config:
boltdb_shipper:
active_index_directory: /var/lib/loki/boltdb-shipper-active # Directory for active indexes
cache_location: /var/lib/loki/boltdb-shipper-cache # Directory for index cache
resync_interval: 5s
shared_store: filesystem
filesystem:
directory: /var/lib/loki/chunks # Directory for storing log chunks
compactor:
working_directory: /var/lib/loki/compactor # Directory for the compactor
shared_store: filesystem
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h # Reject samples older than 7 days
max_query_lookback: 720h # Maximum period for queries (30 days)
max_query_series: 5000 # Maximum number of series in a query
max_query_length: 720h # Maximum query duration
max_global_streams_per_user: 5000000
max_streams_per_user: 100000
ruler:
storage:
type: local
local:
directory: /var/lib/loki/rules
rule_path: /var/lib/loki/rules-temp
alertmanager_url: http://localhost:9093 # Example, if you will use Alertmanager
ring:
kvstore:
store: inmemory
# Query settings (query-frontend, query-scheduler)
query_range:
align_queries_with_step: true
# Ingester settings (component that receives logs)
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
max_transfer_retries: 0
Save the file. Now, let's create a Systemd service for Loki.
sudo nano /etc/systemd/system/loki.service
Insert the following content:
[Unit]
Description=Loki log aggregation system
After=network.target
[Service]
User=loki
Group=loki
Type=simple
ExecStart=/usr/local/bin/loki -config.file /etc/loki/config.yaml
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
Save the file, reload Systemd, and start Loki:
sudo systemctl daemon-reload
sudo systemctl start loki
sudo systemctl enable loki
sudo ufw allow 3100/tcp # Open Loki port
sudo ufw reload
Check Loki's status:
sudo systemctl status loki
It should show active (running).
Promtail Configuration
Now, let's configure Promtail to collect logs. Create the file /etc/promtail/config.yaml.
sudo nano /etc/promtail/config.yaml
Insert the following content. This example will collect syslog and Nginx logs.
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /var/lib/promtail/positions.yaml # File for tracking log positions
clients:
- url: http://localhost:3100/loki/api/v1/push # URL for sending logs to Loki
scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log # Collect all .log files from /var/log
host: YOUR_HOSTNAME # Replace with the actual hostname of your VPS
pipeline_stages:
- docker: {} # Example, if Promtail were in Docker, other parsers could be used here
- match:
selector: '{job="varlogs"}'
stages:
- regex:
expression: '^(?P
IMPORTANT: Replace YOUR_HOSTNAME with the actual hostname of your VPS. Also, ensure that the log paths (/var/log/*log, /var/log/nginx/access.log, /var/log/nginx/error.log) match your system.
Save the file. Now, let's create a Systemd service for Promtail.
sudo nano /etc/systemd/system/promtail.service
Insert the following content:
[Unit]
Description=Promtail log collector
After=network.target
[Service]
User=promtail
Group=promtail
Type=simple
ExecStart=/usr/local/bin/promtail -config.file /etc/promtail/config.yaml
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
Save the file, reload Systemd, and start Promtail:
sudo systemctl daemon-reload
sudo systemctl start promtail
sudo systemctl enable promtail
Check Promtail's status:
sudo systemctl status promtail
It should show active (running).
Nginx Configuration
Nginx will act as a reverse proxy for Loki. First, let's remove the default Nginx configuration:
sudo rm /etc/nginx/sites-enabled/default
Let's create a new configuration file for Loki, for example, /etc/nginx/sites-available/loki.conf.
sudo nano /etc/nginx/sites-available/loki.conf
Insert the following content. Replace your.domain.com with your domain, which will point to your VPS.
server {
listen 80;
listen [::]:80;
server_name your.domain.com; # Replace with your domain
location / {
proxy_pass http://localhost:3100; # Proxy requests to Loki
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s; # Increase timeout for large requests
}
}
Save the file. Now, activate this configuration by creating a symbolic link:
sudo ln -s /etc/nginx/sites-available/loki.conf /etc/nginx/sites-enabled/
Check the Nginx configuration for errors and restart it:
sudo nginx -t
sudo systemctl restart nginx
If there are no errors, Nginx should restart successfully.
Setting up HTTPS with Certbot
For secure access to Loki via Nginx, we will use Certbot to obtain a TLS certificate from Let's Encrypt.
sudo certbot --nginx -d your.domain.com
Certbot will guide you through the process. You will need to enter an email address for notifications and agree to the terms of service. It will automatically detect your Nginx server block for your.domain.com and modify it, adding HTTPS settings and HTTP to HTTPS redirection.
After successful Certbot setup, your /etc/nginx/sites-available/loki.conf file will look approximately like this (Certbot will add its directives):
server {
listen 80;
listen [::]:80;
server_name your.domain.com;
# Certbot will add HTTPS redirection here
return 301 https://$host$request_uri;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name your.domain.com;
ssl_certificate /etc/letsencrypt/live/your.domain.com/fullchain.pem; # Path to certificate
ssl_certificate_key /etc/letsencrypt/live/your.domain.com/privkey.pem; # Path to private key
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
location / {
proxy_pass http://localhost:3100;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
}
}
Certbot will also configure automatic certificate renewal. You can check this:
sudo systemctl status certbot.timer
sudo certbot renew --dry-run
Verifying Functionality
Let's ensure that all components are working correctly.
- Loki Check:
curl -v http://localhost:3100/readyIt should return a
200 OKstatus and the messageLoki is ready. - Promtail Check:
sudo journalctl -u promtail.service -fYou will see Promtail logs. Ensure there are no errors when sending logs to Loki.
- Nginx and HTTPS Check:
Open
https://your.domain.comin your browser. You should see the Loki interface. If you see a404or502 Bad Gatewayerror, check Nginx logs (/var/log/nginx/error.log) and Loki logs. - Loki Log Check:
From the command line, you can make a request to Loki via
curlto ensure that logs are being received:curl -G "https://your.domain.com/loki/api/v1/query_range" --data-urlencode 'query={job="varlogs"}' | jq .You should see a JSON response with your logs. If
jqis not installed, install it:sudo apt install jq -y.For more convenient log viewing, you can connect Loki to Grafana. Install Grafana on a separate server or as a Docker container and add Loki as a data source. Then you can use the LogQL query language in Grafana Explore.