To create your own mail server, providing full control over your correspondence and high fault tolerance, you will need a powerful VPS or dedicated server, configured using a combination of MTA (e.g., Postfix) and MDA (Dovecot), as well as carefully crafted DNS records for email authentication.
In a world where data privacy and independence from third-party services are becoming increasingly valuable, deploying a self-hosted email solution is a strategic advantage. This is not just email hosting, but full-fledged email server hosting that allows you to manage every aspect of email operations, from routing to storage and security. Let's explore how to build a reliable self-hosted mail server from scratch using proven technologies and Valebyte resources.
Why choose your own mail server (self-hosted email)?
Choosing self-hosted email is a decision driven by the desire for maximum control, security, and flexibility. Unlike popular cloud email services (Gmail, Outlook 365), your own mail server gives you complete autonomy.
- Full data control: All your correspondence is stored on your server, under your management. This is critical for companies with high demands for confidentiality and compliance with regulations (GDPR, HIPAA).
- Independence from providers: You are not dependent on the policies, failures, or limitations of third-party services. You define the rules of use, storage volume, and functionality yourself.
- Flexible configuration and scaling: You can configure the server to your unique needs, integrate it with other internal systems, use any anti-spam solutions, and easily scale resources as you grow.
- Long-term economic benefit: For a large number of users or specific requirements, the cost of owning a mail server VPS or a dedicated server may be lower than subscriptions to corporate cloud services.
However, it is worth noting that maintaining your own mail server requires certain technical knowledge and time for administration. This includes software configuration, DNS record management, spam fighting, and security assurance.
What VPS is needed for email server hosting?
Choosing the right mail server VPS is the first and one of the most important steps. The resources required for email server hosting depend on the anticipated load: the number of active mailboxes, traffic volume, and anti-spam solutions used.
Minimum requirements for a small server (up to 20-30 users):
- CPU: 2 vCPU. For processing incoming/outgoing emails and anti-spam filters.
- RAM: 4 GB. Postfix, Dovecot, SpamAssassin, and other services require sufficient RAM.
- Disk: 50-80 GB NVMe SSD. A fast disk is critical for database performance and mail storage. NVMe provides significantly higher read/write speeds compared to SATA SSD.
- Bandwidth: 100 Mbps. For stable operation and fast mail delivery.
- OS: Ubuntu Server 22.04 LTS, Debian 12, or CentOS Stream 9.
Recommended requirements for a medium server (50-100 users):
- CPU: 4 vCPU.
- RAM: 8-16 GB.
- Disk: 100-200 GB NVMe SSD.
- Bandwidth: 200-500 Mbps.
For larger installations, as well as for servers with high loads or a large number of mailboxes, it is recommended to consider dedicated servers or VPS with even more RAM and more powerful processors.
Valebyte plans for your mail server
Valebyte offers flexible VPS plans, ideally suited for hosting your own mail server. Below are some recommended configurations:
| Valebyte Plan |
vCPU |
RAM (GB) |
NVMe SSD (GB) |
Bandwidth (Mbps) |
Price (from, $/month) |
Recommendation |
| VPS-Standard-4 |
2 |
4 |
80 |
100 |
14.99 |
Excellent start for 10-30 users |
| VPS-Pro-8 |
4 |
8 |
160 |
200 |
29.99 |
Optimal for 50-100 users |
| VPS-Enterprise-16 |
6 |
16 |
320 |
500 |
59.99 |
For large companies and high load |
| Dedicated Server |
8+ |
32+ |
500+ |
1000 |
from 150 |
Maximum performance and control |
All plans include DDoS protection and 24/7 technical support.
Looking for a reliable server for your projects?
Valebyte offers VPS and dedicated servers with guaranteed resources and fast activation.
View offers →
Key Components of Your Mail Server: Postfix and Dovecot
The foundation of any self-hosted mail server consists of two key components: Mail Transfer Agent (MTA) and Mail Delivery Agent (MDA). In most cases, Postfix server and Dovecot are chosen for these roles.
Postfix: MTA (Mail Transfer Agent)
Postfix server is one of the most popular and reliable open-source MTAs. Its primary task is sending and receiving email. Postfix acts as a "mail courier" that accepts emails from clients, routes them to recipients, and delivers incoming mail to local mailboxes.
The basic Postfix configuration is stored in the /etc/postfix/main.cf file. Here's an example of some key parameters:
# server hostname
myhostname = mail.yourdomain.com
# domain that Postfix will use for local delivery
mydomain = yourdomain.com
# list of domains for which this server will accept mail
mydestination = $myhostname, yourdomain.com, localhost.$mydomain, localhost
# list of networks allowed to relay mail
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
# Add IP addresses of your local networks here if necessary
# SSL/TLS usage
smtpd_tls_cert_file=/etc/ssl/certs/mail.pem
smtpd_tls_key_file=/etc/ssl/private/mail.key
smtpd_tls_security_level=may
It is important to configure Postfix so that it does not become an open relay for spammers, allowing mail to be sent only by authorized users or from specific IP addresses.
Dovecot: MDA/IMAP/POP3 (Mail Delivery Agent)
Dovecot is a high-performance and secure MDA that provides access to mailboxes via IMAP and POP3 protocols. After Postfix delivers an email to the server, Dovecot allows users to access it using email clients (Outlook, Thunderbird, mobile apps).
Dovecot is responsible for:
- Mail Storage: Managing storage formats (Maildir or mbox). Maildir is preferred because each email is stored in a separate file, which prevents corruption of the entire folder in case of an error.
- Authentication: Verifying user credentials (username/password) using local system users, databases, or LDAP.
- IMAP/POP3 Access: Providing secure access to mailboxes.
- SSL/TLS: Encrypting all connections to protect confidentiality.
Example of a portion of the /etc/dovecot/conf.d/10-auth.conf configuration:
disable_plaintext_auth = yes
auth_mechanisms = plain login
#!include auth-system.conf.ext
#!include auth-passwdfile.conf.ext
And /etc/dovecot/conf.d/10-mail.conf:
mail_location = maildir:~/Maildir
These two components, Postfix server and Dovecot, together form a powerful and flexible solution for email server hosting.
Configuring DNS Records for Reliable Mail Delivery (SPF, DKIM, DMARC)
Even the most powerful mail server VPS cannot effectively deliver mail without properly configured DNS records. SPF, DKIM, and DMARC are critically important authentication mechanisms that help confirm the legitimacy of your server and prevent emails from landing in spam.
SPF (Sender Policy Framework)
SPF is a TXT record in DNS that specifies which servers are authorized to send mail on behalf of your domain. This helps prevent sender spoofing.
yourdomain.com. IN TXT "v=spf1 ip4:YOUR_SERVER_IP_ADDRESS a mx ~all"
v=spf1: Specifies the SPF version.
ip4:YOUR_SERVER_IP_ADDRESS: Authorizes mail sending from the specified IP address of your VPS.
a: Authorizes mail sending from the IP address specified in the domain's A record.
mx: Authorizes mail sending from the IP addresses specified in the domain's MX records.
~all: Softfail – emails from other sources will be marked as suspicious but not immediately rejected. Use -all for strict rejection (recommended after testing).
DKIM (DomainKeys Identified Mail)
DKIM adds a digital signature to outgoing emails, allowing the recipient to verify that the email was indeed sent from your domain and was not altered during transit. This is also a TXT record.
The opendkim-genkey utility or similar is typically used to generate DKIM keys. After generation, you will have a private key (stored on the server) and a public key (added to DNS).
# Example TXT record for DKIM
default._domainkey.yourdomain.com. IN TXT "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQD...AQAB"
default is a selector that you can choose arbitrarily.
DMARC (Domain-based Message Authentication, Reporting & Conformance)
DMARC uses SPF and DKIM to verify email authenticity and defines a policy for handling emails that fail authentication. It also allows you to receive reports on attempts to send emails on behalf of your domain.
_dmarc.yourdomain.com. IN TXT "v=DMARC1; p=none; fo=1; ruf=mailto:[email protected]; rua=mailto:[email protected]"
v=DMARC1: DMARC version.
p=none: "Do nothing" policy (used for monitoring). Can be changed to quarantine (send to spam) or reject (decline).
fo=1: Send reports if any of the checks (SPF/DKIM) fail.
ruf/rua: Addresses for sending failure reports and aggregate reports, respectively.
Proper configuration of these records significantly enhances the reputation of your email server hosting and ensures that your emails reach recipients instead of landing in spam.
How to Combat Spam and Ensure Mail Server Security?
Combating spam and ensuring security is an ongoing process for any self-hosted mail server. Underestimating these aspects can lead to your server being used by spammers or ending up on blacklists.
Key measures to combat spam:
- SpamAssassin: A popular and effective open-source spam filter. It analyzes email headers and content, assigns scores, and based on that, determines whether an email is spam. Integrates with Postfix.
- RBL (Real-time Blackhole Lists): Using public blacklists of IP addresses known as spam sources. Postfix can be configured to reject mail from servers on such lists (e.g., Spamhaus, SORBS).
- Greylisting: Temporarily rejecting the first email from an unknown sender. Legitimate servers will retry sending after some time, while spammers will not.
- Rate Limiting: Limiting the number of emails that a single user or IP address can send within a specific period to prevent spamming.
- Content Filtering: Filtering by keywords, attachments, and other parameters.
Security measures:
- SSL/TLS for all connections: Encrypt all connections (SMTP, IMAP, POP3) using SSL/TLS certificates. Valebyte recommends using free Let's Encrypt certificates.
- Fail2Ban: A utility for blocking IP addresses that attempt to brute-force passwords for your services (SSH, Postfix, Dovecot) through multiple failed login attempts.
- Strong Passwords: Require users to use complex, unique passwords and consider two-factor authentication (2FA) if possible.
- Regular Updates: Keep the operating system and all software (Postfix, Dovecot, anti-spam) up to date by installing security patches.
- Firewall: Configure a firewall (e.g., UFW on Linux) to allow access only to necessary ports (25, 587, 465 for SMTP; 143, 993 for IMAP; 110, 995 for POP3; 22 for SSH).
- Log Monitoring: Regularly review Postfix, Dovecot, and system logs for suspicious activity.
Recommendations for Choosing a VPS for Email Server Hosting from Valebyte
When choosing a mail server VPS for your mail server, consider the following recommendations:
- Start with adequate resources: As mentioned above, for most small and medium-sized companies, a VPS with 4-8 GB RAM and 2-4 vCPU will be sufficient. You can always scale up, but underestimating resources at the start will lead to problems.
- Prioritize NVMe SSD: Disk subsystem speed is critical for working with mailboxes and anti-spam filter databases. NVMe SSD significantly outperforms regular SSDs and HDDs.
- Choose a reliable provider: Valebyte provides stable VPS with guaranteed availability (SLA 99.9%) and quality support, which is critically important for email server hosting.
- DDoS protection: Mail servers can be targets of DDoS attacks. All Valebyte plans include basic DDoS protection.
- Static IP address: A mandatory condition for a mail server. Valebyte provides a dedicated static IP address with each VPS.
Remember that successful email server hosting is not just about powerful hardware, but also proper software configuration, and continuous attention to security and monitoring.
Conclusion
Deploying your own mail server on a VPS from Valebyte provides full control over your correspondence, ensuring high security and scalability. By choosing the right plan and following the recommendations for configuring Postfix, Dovecot, and DNS records, you can create a reliable and efficient platform for email server hosting, tailored to your unique needs.
Ready to choose a server?
Compare VPS and dedicated servers from trusted providers on Valebyte.
Get started now →