Setting Up Postfix, Dovecot

What We Configure and Why

In the modern world, email remains one of the most important means of communication. While there are many free and paid email services, such as Gmail, Outlook, or Yandex.Mail, they have their limitations: lack of full data control, potential censorship, dependence on provider policies, and sometimes insufficient flexibility in configuration. Setting up your own mail server on a VPS allows you to take complete control over your correspondence.

In this guide, we will set up a full-fledged email system consisting of three key components:

• Postfix: This is a Mail Transfer Agent (MTA) that will be responsible for sending and receiving email via the SMTP protocol. Postfix is known for its security, reliability, and performance.
• Dovecot: This is a Mail Delivery Agent (MDA) and IMAP/POP3 server. Dovecot allows mail clients (both desktop and mobile) to access and manage your mailboxes. It supports secure connections (SSL/TLS).
• Roundcube: This is a modern webmail client that will provide you and your users with a convenient interface for working with mail directly from a browser. It is easy to use, feature-rich, and easily integrates with Postfix and Dovecot.

Ultimately, you will get a fully functional, secure, and scalable mail server that can serve domains and users of your choice. This is an ideal solution for developers who need their own mail service for projects, SaaS solo-founders who value independence, or any other users who value privacy and complete control over their data.

Alternatives: Cloud-Managed vs. Self-Hosted

There are two main approaches to email management:

• Cloud-Managed (managed cloud services): These are solutions like Google Workspace, Microsoft 365, Zoho Mail, or Proton Mail.
  • Pros: Easy setup (often "turnkey"), high availability, automatic updates and backups, professional support.
  • Cons: High cost per user, lack of full data control, dependence on provider policy, limited customization options.
• Self-Hosted (own server on VPS): Setting up a mail server independently on a rented VPS or dedicated server.
  • Pros: Full control over data and configuration, high degree of privacy, economies of scale (especially for a large number of users), possibility of integration with your other services.
  • Cons: Requires technical knowledge and time for setup/maintenance, responsibility for security and backups lies with you, potential issues with email deliverability if not configured correctly.

Choosing self-hosted on a VPS is justified if you value complete control, are willing to invest time in setup and maintenance, and want to get the most flexible and private solution for your needs.

What VPS Configuration is Needed for This Task

VPS requirements for a mail server can vary depending on the expected load (number of users, traffic volume). However, for most personal and small team projects, you can aim for the following minimum characteristics:

Minimum Requirements

• CPU: 1 core. Modern processors are powerful enough to handle email traffic.
• RAM: 2 GB. Postfix, Dovecot, Nginx, PHP-FPM, and the database (MariaDB) can consume a significant amount of memory. 2 GB will allow for comfortable operation.
• Disk: 50-80 GB SSD. SSD will significantly speed up I/O operations, which is critical for working with mail storage and databases. Disk space depends on the number and size of mailboxes, as well as the duration of archive storage.
• Network: 100 Mbps or 1 Gbps. Channel speed is important for fast delivery and reception of emails, but for typical use, 100 Mbps is quite sufficient.
• Operating System: Debian 12 (Bookworm) or Ubuntu 24.04 LTS (Noble Numbat). These distributions are well-supported, have up-to-date packages, and extensive documentation.
• Dedicated IP address: Mandatory. Mail servers cannot operate with shared IP addresses due to reputation issues.

Recommended VPS Plan for Starting

For comfortable operation of a mail server supporting multiple domains and up to 50 active users, the following configuration is recommended:

• CPU: 2 cores
• RAM: 4 GB
• Disk: 100-200 GB SSD
• Network: 1 Gbps

Such a configuration will be sufficient for processing emails, operating the Roundcube web interface, and maintaining high performance. You can get a VPS with the specified characteristics, which will provide the necessary performance and reliability for your mail server.

When a Dedicated Server is Needed, Not a VPS

A dedicated server may be required if:

• You have a very large number of users (hundreds or thousands).
• You plan to store huge volumes of mail (terabytes).
• You need maximum performance and resource isolation beyond what a VPS offers.
• Specific hardware configurations not available on a VPS are required.

For most mail server tasks, a VPS is the optimal choice in terms of price/performance ratio.

Location: What It Affects

Choosing a VPS location has several important aspects:

• Latency: The closer the server is to your primary audience (mail users), the lower the latency when accessing the Roundcube web interface or synchronizing mail clients.
• Jurisdiction and Legislation: Depending on the server's country of location, local data storage and privacy laws apply to it. This can be critical for some projects.
• Cost: VPS prices can differ in various countries.

For a mail server, it is generally recommended to choose a location that is geographically closer to most of your users to ensure minimal latency.

Server Preparation

Before installing the mail software, you need to perform basic setup and security hardening of your new VPS. We will use the Debian 12 (Bookworm) distribution as an example. The steps for Ubuntu will be similar.

1. System Update

First, update the package list and installed packages to their latest versions.

sudo apt update          # Update package list
sudo apt upgrade -y      # Upgrade installed packages
sudo apt autoremove -y   # Remove unnecessary packages

2. Creating a New User with Sudo Privileges

Working as the root user is unsafe. Create a new user and grant them sudo privileges.

sudo adduser username          # Replace 'username' with your desired username
sudo usermod -aG sudo username # Add user to sudo group

Log out of the current root session and log in as the new user.

exit

3. SSH Key Setup (Recommended)

To enhance SSH access security, it is recommended to use SSH keys instead of passwords. First, generate a key on your local machine if you don't have one yet:

ssh-keygen -t rsa -b 4096

Then, copy the public key to the server:

ssh-copy-id username@your_vps_ip # Replace 'username' and 'your_vps_ip'

After verifying key-based login, disable password login for root and regular users in the file /etc/ssh/sshd_config:

sudo nano /etc/ssh/sshd_config

Find and change the following lines:

PermitRootLogin no
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

Restart the SSH server:

sudo systemctl restart sshd

4. Installing and Configuring Fail2Ban

Fail2Ban scans logs and blocks IP addresses that show signs of malicious activity (e.g., multiple failed login attempts). This is critically important for protecting SSH and mail services.

sudo apt install fail2ban -y
sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Create a local configuration file for Fail2Ban so that your changes are not overwritten during package updates:

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit /etc/fail2ban/jail.local to enable protection for SSH and other services, and configure blocking parameters (bantime, findtime, maxretry). Ensure that enabled = true for [sshd] and other mail services that we will configure later ([postfix], [dovecot]).

sudo nano /etc/fail2ban/jail.local

Example section for SSH (ensure enabled = true):

[sshd]
enabled = true
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s

Restart Fail2Ban after changes:

sudo systemctl restart fail2ban

5. Firewall Configuration (UFW)

Uncomplicated Firewall (UFW) is an easy-to-use interface for iptables. We will configure it to allow only the necessary ports.

sudo apt install ufw -y

Allow SSH (port 22), HTTP (port 80), HTTPS (port 443), and standard mail ports:

sudo ufw allow OpenSSH
sudo ufw allow http
sudo ufw allow https
sudo ufw allow 25/tcp  # SMTP
sudo ufw allow 587/tcp # SMTPS (Submission)
sudo ufw allow 465/tcp # SMTPS (Legacy)
sudo ufw allow 143/tcp # IMAP
sudo ufw allow 993/tcp # IMAPS
sudo ufw allow 110/tcp # POP3
sudo ufw allow 995/tcp # POP3S

Enable UFW:

sudo ufw enable

Check UFW status:

sudo ufw status verbose

Your server is now basically secured and ready for mail software installation.

Software Installation — Step-by-Step

In this section, we will install all necessary components: Postfix, Dovecot, Nginx, PHP-FPM, MariaDB, and Roundcube. We will use the current software versions available in Debian 12 repositories (as of 2026, these will be stable branches, e.g., Postfix 3.x, Dovecot 2.x, Nginx 1.x, PHP 8.x, MariaDB 10.x).

1. Postfix Installation

Postfix is our Mail Transfer Agent (MTA). It will be responsible for sending and receiving email.

sudo apt install postfix -y # Install Postfix

During Postfix installation, it will ask several questions. Select "Internet Site" and specify your server's fully qualified domain name (FQDN) (e.g., mail.example.com), which will be used as the hostname for sending mail. Replace example.com with your actual domain.

For System mail name, specify your primary domain (e.g., example.com).

2. Dovecot Installation

Dovecot is our Mail Delivery Agent (MDA) and IMAP/POP3 server, which will allow users to access their mailboxes.

sudo apt install dovecot-imapd dovecot-pop3d dovecot-lmtpd dovecot-mysql -y # Install Dovecot with IMAP, POP3, LMTP, and MySQL support for authentication

dovecot-lmtpd is needed for more efficient mail delivery from Postfix to Dovecot, and dovecot-mysql is for user authentication via a database, which significantly simplifies management.

3. Nginx, PHP-FPM, and MariaDB Installation

For Roundcube to work, we will need a web server (Nginx), a PHP interpreter (PHP-FPM), and a database (MariaDB).

sudo apt install nginx -y # Install Nginx web server
sudo apt install php-fpm php-mysql php-mbstring php-intl php-zip php-gd php-json php-curl php-imap php-xml php-pear -y # Install PHP-FPM and necessary extensions for Roundcube
sudo apt install mariadb-server -y # Install MariaDB server

After installing PHP-FPM, Nginx, and MariaDB, ensure that all services are running and enabled for autostart:

sudo systemctl enable nginx
sudo systemctl start nginx
sudo systemctl enable php8.2-fpm # Or the current version, e.g., php8.3-fpm
sudo systemctl start php8.2-fpm
sudo systemctl enable mariadb
sudo systemctl start mariadb

Run the script for basic MariaDB security:

sudo mysql_secure_installation # Follow the instructions: set root password, remove anonymous users, disallow remote root login, remove test database.

4. Roundcube Installation

Roundcube is a web interface for your mail. We will install it from the official repositories.

sudo apt install roundcube roundcube-core roundcube-plugins roundcube-sqlite3 -y # Install Roundcube and its components. We will use MySQL, but 'roundcube-sqlite3' might be installed as a dependency.

During Roundcube installation, it will ask about database configuration. Select dbconfig-common and mysql, then enter the password for the Roundcube user in MariaDB. If you do not want Roundcube to create the database automatically, select "No" and create it manually (we will cover this option in the configuration section).

If you chose automatic setup, Roundcube will create the roundcubemail database and the roundcube user with the password you specified.

5. Creating a Database and User for Roundcube (if not automatically configured)

If you preferred manual database setup for Roundcube, follow these steps. Otherwise, this step can be skipped.

sudo mysql -u root -p # Enter the root password you set earlier

Inside MariaDB, create a database and user for Roundcube:

CREATE DATABASE roundcubemail CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'roundcube'@'localhost' IDENTIFIED BY 'your_roundcube_password';
GRANT ALL PRIVILEGES ON roundcubemail. TO 'roundcube'@'localhost';
FLUSH PRIVILEGES;
EXIT;

sudo mysql -u roundcube -p roundcubemail < /usr/share/roundcube/SQL/mysql.initial.sql # Enter 'your_roundcube_password'

Configuration

This section is the most important, as here we will link all components together and configure them for safe and efficient operation. We will use the domain пример.com and the subdomain mail.пример.com for our mail server.

1. DNS Record Configuration

Correct DNS records are critically important for mail deliverability. You will need to add or modify the following records in your domain registrar's or DNS provider's control panel. Replace ваш_IP_VPS with your server's actual IP address.

• A-record for the domain: Points to your VPS.
• A-record for the mail subdomain: Points to your VPS.
• MX-record: Specifies which server is responsible for receiving mail for your domain.
• SPF-record: Helps prevent sender spoofing by indicating which servers are authorized to send mail on behalf of your domain.
• DKIM-record: Adds a digital signature to outgoing emails, confirming their authenticity.
• DMARC-record: Specifies how to handle emails that fail SPF and DKIM checks.

Record Type	Host/Name	Value/Data	Priority
A	@ or пример.com	ваш_IP_VPS	-
A	mail or mail.пример.com	ваш_IP_VPS	-
MX	@ or пример.com	mail.пример.com	10
TXT (SPF)	@ or пример.com	v=spf1 mx a ip4:ваш_IP_VPS ~all	-
TXT (DKIM)	dkim._domainkey	Will be generated by Postfix (see below)	-
TXT (DMARC)	_dmarc	v=DMARC1; p=quarantine; rua=mailto:admin@пример.com; ruf=mailto:admin@пример.com; fo=1	-

Wait some time (up to 24 hours) for DNS records to propagate.

2. Postfix Configuration

The main Postfix configuration files are: /etc/postfix/main.cf and /etc/postfix/master.cf.

sudo nano /etc/postfix/main.cf

Ensure the following parameters are configured (add or modify):

# Main parameters
myhostname = mail.пример.com
mydomain = пример.com
myorigin = $mydomain
inet_interfaces = all
inet_protocols = all

# Domains served by Postfix
mydestination = $myhostname, $mydomain, localhost.$mydomain, localhost
virtual_alias_domains = $mydomain
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-aliases.cf
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailboxes.cf
virtual_transport = lmtp:unix:private/dovecot-lmpa

# TLS/SSL Configuration
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.пример.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.пример.com/privkey.pem
smtpd_tls_security_level = may
smtpd_tls_loglevel = 1
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_use_tls = yes
smtpd_tls_received_header = yes
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1

# Restrictions and spam protection
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
smtpd_recipient_restrictions =
    permit_mynetworks,
    permit_sasl_authenticated,
    reject_unauth_destination,
    reject_invalid_hostname,
    reject_non_fqdn_hostname,
    reject_non_fqdn_sender,
    reject_non_fqdn_recipient,
    reject_unknown_sender_domain,
    reject_unknown_recipient_domain,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    permit

# SASL authentication for outgoing mail
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes

# DKIM (will be configured later)
milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:127.0.0.1:8891
non_smtpd_milters = inet:127.0.0.1:8891

Create configuration files for Postfix MySQL queries:

sudo nano /etc/postfix/mysql-virtual-domains.cf

user = roundcube
password = your_roundcube_password
hosts = 127.0.0.1
dbname = roundcubemail
query = SELECT 1 FROM virtual_domains WHERE name='%s'

sudo nano /etc/postfix/mysql-virtual-mailboxes.cf

user = roundcube
password = your_roundcube_password
hosts = 127.0.0.1
dbname = roundcubemail
query = SELECT maildir FROM virtual_users WHERE email='%s'

sudo nano /etc/postfix/mysql-virtual-aliases.cf

user = roundcube
password = your_roundcube_password
hosts = 127.0.0.1
dbname = roundcubemail
query = SELECT destination FROM virtual_aliases WHERE source='%s'

Change file permissions for these files:

sudo chmod 640 /etc/postfix/mysql-virtual-.cf
sudo chown root:postfix /etc/postfix/mysql-virtual-.cf

sudo systemctl restart postfix

sudo nano /etc/dovecot/conf.d/10-mail.conf

mail_location = maildir:~/Maildir
mail_privileged_group = mail

sudo nano /etc/dovecot/conf.d/10-auth.conf

disable_plaintext_auth = yes # Disable insecure authentication
auth_mechanisms = plain login
!include auth-sql.conf.ext

sudo nano /etc/dovecot/conf.d/10-master.conf

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmpa {
    mode = 0660
    user = postfix
    group = postfix
  }
}

sudo nano /etc/dovecot/conf.d/15-lda.conf

postmaster_address = postmaster@пример.com

sudo nano /etc/dovecot/conf.d/auth-sql.conf.ext

passdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}
userdb {
  driver = sql
  args = /etc/dovecot/dovecot-sql.conf.ext
}

sudo nano /etc/dovecot/dovecot-sql.conf.ext

driver = mysql
connect = host=127.0.0.1 dbname=roundcubemail user=roundcube password=your_roundcube_password
default_pass_scheme = SHA512-CRYPT
password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'
user_query = SELECT maildir, 1000 AS uid, 1000 AS gid FROM virtual_users WHERE email='%u'

sudo chmod 640 /etc/dovecot/dovecot-sql.conf.ext
sudo chown root:dovecot /etc/dovecot/dovecot-sql.conf.ext

sudo nano /etc/dovecot/conf.d/10-ssl.conf

ssl = yes
ssl_cert = </etc/letsencrypt/live/mail.пример.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.пример.com/privkey.pem

sudo systemctl restart dovecot

sudo mysql -u root -p

USE roundcubemail;

CREATE TABLE virtual_domains (
  id int(11) NOT NULL AUTO_INCREMENT,
  name varchar(50) NOT NULL,
  PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

CREATE TABLE virtual_users (
  id int(11) NOT NULL AUTO_INCREMENT,
  domain_id int(11) NOT NULL,
  password varchar(255) NOT NULL,
  email varchar(100) NOT NULL,
  maildir varchar(255) NOT NULL,
  PRIMARY KEY (id),
  UNIQUE KEY email (email),
  FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

CREATE TABLE virtual_aliases (
  id int(11) NOT NULL AUTO_INCREMENT,
  domain_id int(11) NOT NULL,
  source varchar(100) NOT NULL,
  destination varchar(100) NOT NULL,
  PRIMARY KEY (id),
  FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

INSERT INTO virtual_domains (name) VALUES ('пример.com');

-- Replace 'your_admin_password' with a strong password
-- and generate it using doveadm pw -s SHA512-CRYPT
INSERT INTO virtual_users (domain_id, password, email, maildir)
VALUES (1, '{SHA512-CRYPT}encrypted_password_for_admin', 'admin@пример.com', 'пример.com/admin/');

-- Alias example: emails to info@пример.com will be redirected to admin@пример.com
INSERT INTO virtual_aliases (domain_id, source, destination)
VALUES (1, 'info@пример.com', 'admin@пример.com');

FLUSH PRIVILEGES;
EXIT;

sudo doveadm pw -s SHA512-CRYPT # Enter the password, then copy the output

sudo apt install certbot python3-certbot-nginx -y # Install Certbot and the Nginx plugin

sudo certbot --nginx -d mail.пример.com # Follow Certbot's instructions. Choose to redirect HTTP to HTTPS.

sudo rm /etc/nginx/sites-enabled/default
sudo nano /etc/nginx/sites-available/roundcube.conf

server {
    listen 80;
    server_name mail.пример.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    server_name mail.пример.com;

    ssl_certificate /etc/letsencrypt/live/mail.пример.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mail.пример.com/privkey.pem;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256";
    ssl_prefer_server_ciphers on;
    ssl_stapling on;
    ssl_stapling_verify on;
    resolver 8.8.8.8 8.8.4.4 valid=300s;
    resolver_timeout 5s;

    root /usr/share/roundcube;
    index index.php index.html;

    location ~ ^/(README|INSTALL|LICENSE|CHANGELOG|UPGRADING)$ {
        deny all;
    }

    location ~ ^/(bin|SQL|config|temp|logs)/ {
        deny all;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_pass unix:/var/run/php/php8.2-fpm.sock; # Ensure this is the correct path for your PHP version
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location ~ /\.ht {
        deny all;
    }
}

sudo ln -s /etc/nginx/sites-available/roundcube.conf /etc/nginx/sites-enabled/
sudo nginx -t

sudo systemctl restart nginx

sudo nano /etc/roundcube/config.inc.php

$config['default_host'] = 'ssl://mail.пример.com';
$config['default_port'] = 993; // IMAPS
$config['smtp_server'] = 'tls://mail.пример.com';
$config['smtp_port'] = 587; // SMTPS submission port
$config['smtp_user'] = '%u';
$config['smtp_pass'] = '%p';
$config['product_name'] = 'My Mail on Пример.com'; // The name that will be displayed in Roundcube
$config['plugins'] = array('archive', 'zipdownload', 'managesieve', 'password'); // Example plugins
$config['temp_dir'] = '/var/lib/roundcube/temp';
$config['log_dir'] = '/var/log/roundcube';
$config['db_dsnw'] = 'mysql://roundcube:your_roundcube_password@localhost/roundcubemail';

sudo mkdir -p /var/lib/roundcube/temp /var/log/roundcube
sudo chown -R www-data:www-data /var/lib/roundcube/temp /var/log/roundcube
sudo chmod 755 /var/lib/roundcube/temp /var/log/roundcube

sudo systemctl restart php8.2-fpm # Or the current version

sudo apt install opendkim opendkim-tools -y # Install OpenDKIM

sudo mkdir -p /etc/mail/dkim
sudo chown -R opendkim:opendkim /etc/mail/dkim

sudo opendkim-genkey -b 2048 -d пример.com -s dkim -v -C /etc/mail/dkim
sudo chown opendkim:opendkim /etc/mail/dkim/dkim.private

sudo nano /etc/opendkim.conf

Canonicalization    relaxed/simple
Mode                sv
SubDomains          no
AutoRestart         Yes
AutoRestartRate     10/1h
Background          Yes
UMask               002
OversignHeaders     From
SignatureAlgorithm  rsa-sha256

# Local socket for Postfix
Socket              inet:8891@localhost

# User and group
UserID              opendkim:opendkim

# Files
PidFile             /var/run/opendkim/opendkim.pid
KeyFile             /etc/mail/dkim/dkim.private
LogWhy              Yes

# Tables
KeyTable            /etc/mail/dkim/key.table
SigningTable        /etc/mail/dkim/signing.table
ExternalIgnoreList  /etc/mail/dkim/trusted.hosts
InternalHosts       /etc/mail/dkim/trusted.hosts

sudo nano /etc/mail/dkim/key.table

dkim._domainkey.пример.com пример.com:dkim:/etc/mail/dkim/dkim.private

sudo nano /etc/mail/dkim/signing.table

@пример.com dkim._domainkey.пример.com

sudo nano /etc/mail/dkim/trusted.hosts

127.0.0.1
localhost
пример.com
mail.пример.com

Change permissions:

sudo chown opendkim:opendkim /etc/mail/dkim/
sudo chmod 640 /etc/mail/dkim/

Edit the file /etc/default/opendkim to specify the socket:

sudo nano /etc/default/opendkim

Uncomment or add:

SOCKET="inet:8891@localhost"

Restart OpenDKIM and Postfix:

sudo systemctl restart opendkim
sudo systemctl restart postfix

Now you need to add the public DKIM key to DNS. Output the file content:

sudo cat /etc/mail/dkim/dkim.txt

You will see something like:

dkim._domainkey IN TXT ( "v=DKIM1; h=sha256; k=rsa; "
       "p=MIIBIjANw..." )  ; ----- DKIM key dkim for пример.com

Copy the p=... value (without quotes) and add it as a TXT record for the host dkim._domainkey in your DNS.

Operational Check

After all configurations, it is necessary to check that everything is working correctly.

• DNS Check: Use online tools (e.g., mxtoolbox.com) to check your domain's MX, SPF, DKIM, and DMARC records.
• Roundcube Access: Open https://mail.пример.com in your browser. You should see the Roundcube login page. Try logging in with the created user admin@пример.com.
• Sending and Receiving Mail:
  1. Send an email from admin@пример.com to an external address (e.g., Gmail). Check that the email arrived and did not go to spam. Check the email headers for SPF, DKIM, DMARC.
  2. Send an email from an external address to admin@пример.com. Make sure it appears in Roundcube.
• Log Check:

  
  tail -f /var/log/mail.log # Postfix and Dovecot logs
  tail -f /var/log/nginx/access.log # Nginx access logs
  tail -f /var/log/nginx/error.log # Nginx error logs
  tail -f /var/log/roundcube/errors.log # Roundcube error logs
              

Backups and Maintenance

A reliable backup system and regular maintenance are critically important for any server, especially for a mail server where valuable data is stored.

What to Back Up

• MariaDB Database: Contains information about domains, users, and their passwords.
• Mailboxes: User Maildir directories (usually /var/vmail/example.com/user/Maildir).
• Configuration Files: Postfix (/etc/postfix/), Dovecot (/etc/dovecot/), Nginx (/etc/nginx/sites-available/roundcube.conf), Roundcube (/etc/roundcube/), OpenDKIM (/etc/mail/dkim/), Certbot (/etc/letsencrypt/).

Simple Auto-Backup Script

We will create a simple script that will archive Maildir and configurations, as well as dump the database. For storage, you can use an external S3-compatible service or another VPS.

sudo nano /usr/local/bin/backup_mail_server.sh

#!/bin/bash

# Parameters
BACKUP_DIR="/var/backups/mail_server"
DATE=$(date +%Y-%m-%d_%H-%M-%S)
DB_NAME="roundcubemail"
DB_USER="roundcube"
DB_PASS="your_roundcube_password" # Use the password from Dovecot/Postfix configuration

# Create backup directory if it doesn't exist
mkdir -p "$BACKUP_DIR"

# 1. Database Backup
echo "Dumping MariaDB database..."
mysqldump -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" > "$BACKUP_DIR/$DB_NAME-$DATE.sql"
if [ $? -eq 0 ]; then
    echo "Database backup successful."
else
    echo "Database backup failed!"
    exit 1
fi

# 2. Mailbox Backup (Maildir)
echo "Archiving Maildir directories..."
tar -czf "$BACKUP_DIR/maildirs-$DATE.tar.gz" /var/vmail # Make sure this is the correct path to Maildir
if [ $? -eq 0 ]; then
    echo "Maildir backup successful."
else
    echo "Maildir backup failed!"
    exit 1
fi

# 3. Configuration Files Backup
echo "Archiving configuration files..."
tar -czf "$BACKUP_DIR/configs-$DATE.tar.gz" \
    /etc/postfix \
    /etc/dovecot \
    /etc/nginx/sites-available/roundcube.conf \
    /etc/roundcube \
    /etc/mail/dkim \
    /etc/letsencrypt
if [ $? -eq 0 ]; then
    echo "Configuration files backup successful."
else
    echo "Configuration files backup failed!"
    exit 1
fi

# 4. Delete old backups (e.g., older than 7 days)
echo "Cleaning up old backups..."
find "$BACKUP_DIR" -type f -name "." -mtime +7 -delete
echo "Backup process complete."

Make the script executable:

sudo chmod +x /usr/local/bin/backup_mail_server.sh

Set up daily execution using cron:

sudo crontab -e

Add the following line to the end of the file for daily execution, for example, at 3:00 AM:

0 3   * /usr/local/bin/backup_mail_server.sh > /var/log/mail_backup.log 2>&1

Where to Store Backups

It is critically important to store backups off the main server. Options:

• S3-compatible object storage: Amazon S3, DigitalOcean Spaces, Backblaze B2, MinIO. These are reliable and scalable solutions.
• Separate VPS: If you have another VPS, you can set up Rsync or SFTP to transfer backups there.
• Local storage on your computer: For very small volumes and personal use, but less reliable.

For automatic sending of backups to S3, you can use utilities like s3cmd or rclone by adding them to the script.

Updates: rolling vs maintenance window

Keeping software up-to-date is key to security and stability.

• Rolling updates: Regular application of security patches and minor updates. For Debian/Ubuntu, this is sudo apt update && sudo apt upgrade -y. It is recommended to perform this weekly.
• Maintenance window: For major updates (e.g., upgrading from PHP 8.2 to 8.3 or from Debian 12 to 13), plan a separate maintenance window. This may require stopping services, checking compatibility, and manual configuration. Always perform a full backup before such updates.

Monitoring logs (/var/log/mail.log, /var/log/syslog) and resource usage (htop, df -h) is also an important part of maintenance.

Troubleshooting + FAQ

Setting up a mail server is a complex task, and issues can arise at any stage. Here are answers to frequently asked questions and typical troubleshooting scenarios.

Cannot send emails, they are stuck in the Postfix queue. What to do?

Check the Postfix queue with the command sudo mailq. If emails are there, check Postfix logs (tail -f /var/log/mail.log). Typical reasons:

• DNS issues: Incorrect MX record, missing SPF/DKIM/DMARC, or their incorrect configuration. Check records via mxtoolbox.com.
• Reverse DNS record (PTR): Ensure that a PTR record corresponding to mail.example.com is configured for your IP address. Many mail servers reject emails without a correct PTR. This is configured with your VPS provider.
• IP address on blacklists (RBL): Check your VPS's IP on RBL lists (e.g., via mxtoolbox.com). If the IP is blacklisted, you need to submit a request for removal.
• TLS/SSL issues: Incorrect certificate paths or outdated protocols.
• Firewall settings: Ensure that outgoing traffic on port 25 (SMTP) is allowed.

Cannot receive emails, they are not reaching my server.

This is almost always related to DNS or the firewall:

• MX record: Ensure that your MX record for the domain example.com points to mail.example.com and has the correct priority.
• A record for mail.example.com: Must point to your VPS's IP.
• Firewall (UFW): Ensure that incoming connections on port 25 (SMTP) are allowed (sudo ufw status verbose).
• Postfix logs: Check /var/log/mail.log for mail reception errors.

Cannot log in to Roundcube or mail clients are not connecting.

Authentication or Dovecot connection issues:

• Incorrect credentials: Ensure you are using the correct email ([email protected]) and password. Check the password in the database.
• Dovecot settings: Check the files /etc/dovecot/conf.d/10-auth.conf, auth-sql.conf.ext, and dovecot-sql.conf.ext. Ensure that the database connection parameters are correct and that the password in dovecot-sql.conf.ext matches the Roundcube user's password in MariaDB.
• TLS/SSL for Dovecot: Ensure that the certificate paths in /etc/dovecot/conf.d/10-ssl.conf are correct.
• Firewall (UFW): Ensure that ports 143 (IMAP), 993 (IMAPS), 110 (POP3), 995 (POP3S) are allowed.
• Dovecot logs: Check /var/log/mail.log for authentication errors or connection issues.
• Roundcube logs: Check /var/log/roundcube/errors.log for errors.

What is the minimum suitable VPS configuration?

For personal use or a small team (up to 10-15 users with moderate activity), a VPS with 1 CPU core, 2 GB RAM, and 50-80 GB SSD will be minimally suitable. This will allow running all necessary services (Postfix, Dovecot, Nginx, PHP-FPM, MariaDB) and provide sufficient headroom for stable operation. However, for higher performance and scalability, 2 CPU cores and 4 GB RAM are recommended.

What to choose — VPS or dedicated for this task?

For most scenarios described in this guide (personal mail server, email for a small startup or team, email for several personal projects), a VPS is the optimal choice. It offers sufficient performance, flexibility, and cost-effectiveness. A dedicated server becomes necessary only under very high loads, for example, if you plan to serve hundreds or thousands of active mailboxes, process huge volumes of traffic, or store terabytes of mail data. In such cases, a dedicated server provides higher performance and resource isolation.

Emails are going to spam. How to fix this?

The main reason emails go to spam is a poor sender reputation. Ensure that you have configured all necessary records:

• SPF: Correctly specifies allowed sender IP addresses.
• DKIM: Signs outgoing emails.
• DMARC: Policy for handling emails that fail SPF/DKIM.
• PTR (Reverse DNS): Your IP address should resolve to mail.example.com.
• Not on RBL: Check that your IP is not on blacklists.
• Email content: Avoid spam words, excessive use of capital letters, suspicious links.

How to add a new user or domain?

To add a new domain, insert a record into the virtual_domains table:

sudo mysql -u root -p
USE roundcubemail;
INSERT INTO virtual_domains (name) VALUES ('new_domain.com');
FLUSH PRIVILEGES;
EXIT;

To add a new user, first get the encrypted password using sudo doveadm pw -s SHA512-CRYPT, then insert a record into the virtual_users table, using the domain_id of the corresponding domain:

sudo mysql -u root -p
USE roundcubemail;
-- Assume 'new_domain.com' has id=2
INSERT INTO virtual_users (domain_id, password, email, maildir)
VALUES (2, '{SHA512-CRYPT}encrypted_password', 'user@new_domain.com', 'new_domain.com/user/');
FLUSH PRIVILEGES;
EXIT;

Don't forget to restart Postfix and Dovecot after significant changes to the database or configuration files.

Conclusions and Next Steps

You have successfully set up and launched your own mail server on a VPS, using Postfix, Dovecot, and Roundcube. Now you have full control over your email, a high degree of privacy, and the ability to flexibly configure it for any needs. This is a serious step towards digital independence and effective communication management.

For further development and optimization of your mail server, consider the following steps:

• Scaling and Optimization: If the number of users and mail volume grows, it may be necessary to optimize the database configuration, increase VPS resources, or consider migrating to a dedicated server. Explore caching options for Roundcube and performance tuning for Postfix/Dovecot.
• Improved Spam and Virus Protection: Integrate additional tools such as SpamAssassin for spam filtering and ClamAV for virus scanning. This will significantly enhance the security and quality of incoming mail.
• Web Administration: For convenient management of users, domains, and aliases, consider installing web panels such as Postfix Admin. This will simplify server administration without the need for direct database interaction.
• Monitoring: Set up a monitoring system (e.g., Prometheus + Grafana or Zabbix) to track server status, resource usage, and mail service performance. This will help identify and resolve potential issues in a timely manner.