Server for IoT platform: MQTT, time-series data

For deploying an IoT platform with an MQTT broker and time-series data storage, a VPS or dedicated server with 4-8 vCPU, 8-16 GB RAM, and fast NVMe disks is optimal — such configurations can efficiently handle up to 1000 devices and data streams, ensuring low latency and high reliability.

The development and operation of the Internet of Things (IoT) require a reliable and scalable server infrastructure. From collecting data from thousands of sensors to analyzing and visualizing it, every stage is critical. At Valebyte.com, we understand these needs and offer optimal solutions for an IoT server, capable of withstanding any load.

What is an IoT platform and why is the right IoT server important?

An IoT platform is a complex of software and hardware tools that ensures communication between IoT devices, as well as data collection, processing, storage, and analysis. It acts as a central hub managing the entire data lifecycle. Key components typically include:

• MQTT broker: for lightweight and efficient message transmission.
• Time-series database: for storing timestamped data.
• Rules and processing engines: for real-time data analysis.
• APIs and user interfaces: for interacting with the platform.

Choosing the right IoT server directly impacts the performance, scalability, security, and total cost of ownership of your IoT system. Insufficient resources will lead to delays, data loss, and failures, while excessive resources will result in unjustified expenses.

MQTT Server Hosting: Choosing a broker and its requirements

MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol, ideally suited for IoT due to its efficiency and low resource consumption. The central element here is the MQTT broker, which receives messages from devices (publishers) and forwards them to interested clients (subscribers).

When choosing MQTT server hosting and the broker itself, consider several popular options:

• Mosquitto: Open source, lightweight, excellent for small to medium projects. Easy to install and configure.
• EMQX: High-performance, scalable broker written in Erlang/OTP. Supports millions of concurrent connections and high message throughput. Has an extensive set of features, including database integrations, bridges, and plugins.
• HiveMQ: Commercial broker, focused on enterprise solutions. Characterized by high reliability, scalability, and advanced security features.

Server requirements for an MQTT broker depend on the number of simultaneous connections, frequency, and size of messages. For 1000-5000 active devices, Mosquitto can run on a VPS with 2-4 vCPU and 4-8 GB RAM. For tens and hundreds of thousands of devices, EMQX or HiveMQ will require dedicated servers with more powerful processors (8+ cores), 16-32 GB RAM, and fast network interfaces.

Example of a basic Mosquitto configuration:

# mosquitto.conf
listener 1883
allow_anonymous false
password_file /etc/mosquitto/passwd
persistence true
persistence_location /var/lib/mosquitto/
log_dest file /var/log/mosquitto/mosquitto.log

Time-Series Data Storage: InfluxDB or TimescaleDB?

Data from IoT devices is inherently time-series: each reading has a timestamp. Traditional relational databases (PostgreSQL, MySQL) are not always optimal for this type of data due to their high write speed and specific query patterns. This is where specialized time series server solutions come to the rescue.

• InfluxDB:
  • Type: NoSQL, specialized time-series database.
  • Advantages: High write and query speed, efficient data compression, built-in query language InfluxQL (similar to SQL) and Flux (for more complex processing). Easily scalable.
  • Disadvantages: Lack of full JOIN operations, different data model (non-relational), which requires adaptation.
  • Resources: Requires fast NVMe disks for write operations, sufficient RAM for caching indexes and active data. CPU is used for query processing and compression.
• TimescaleDB:
  • Type: An extension for PostgreSQL, transforming it into a powerful time-series database.
  • Advantages: Fully compatible with PostgreSQL, supports SQL, which is convenient for developers familiar with relational databases. Excellent capabilities for analytics, JOINs, aggregations.
  • Disadvantages: More resource-intensive compared to InfluxDB for very high write volumes, although modern versions have significantly improved.
  • Resources: Like InfluxDB, benefits from fast disks. Requires more RAM than "pure" PostgreSQL for caching data and indexes. CPU is important for complex SQL queries and aggregations.

The choice between InfluxDB and TimescaleDB often comes down to team preferences and project specifics. For pure time series with an emphasis on write speed and simple aggregations, InfluxDB may be preferable. If complex analytics, integration with other relational data, and a familiar SQL interface are required, TimescaleDB is an excellent choice. For large-scale analytical tasks, solutions like ClickHouse or Elasticsearch can also be used, which we discussed in the article Server for Big Data Analytics: ClickHouse, Elasticsearch.

How many devices can an IoT platform server handle? Resource scaling

The performance of an IoT platform server depends not only on the number of devices but also on many other factors:

• Message frequency: How often does each device send data?
• Payload size: The volume of data in each message.
• Number of subscribers: How many clients (applications, dashboards) receive data simultaneously?
• Processing complexity: Are complex calculations performed on the server, or just simple storage?
• Database type: InfluxDB and TimescaleDB have different load profiles.

Approximate estimates:

• Up to 100 devices (1 message/min per device, 100-500 bytes payload): VPS with 2 vCPU, 4 GB RAM, 50 GB NVMe.
• 100-1000 devices (1 message/10 sec per device, 500-1000 bytes payload): VPS with 4-8 vCPU, 8-16 GB RAM, 100-200 GB NVMe.
• 1000-10000 devices (1 message/sec per device, 1 KB payload): Dedicated server with 8-16 CPU cores (e.g., Intel Xeon E3/E5 or AMD Ryzen), 32-64 GB RAM, 500 GB - 1 TB NVMe RAID 1.
• More than 10000 devices: Multiple dedicated servers, cluster solutions for MQTT (EMQX, HiveMQ) and databases (InfluxDB Cluster, TimescaleDB HA).

Scaling can be vertical (increasing resources of a single server) or horizontal (adding new servers). For IoT platforms, especially with increasing load, horizontal scaling is preferable as it provides greater fault tolerance. You can read more about scaling strategies in our article How to scale a server with increasing load.

Edge vs. Cloud: Where to host an IoT server?

The question of where to host an IoT server is a strategic decision that depends on latency, security, data volume, and cost requirements.

• Edge Computing (at the network edge):
  • Advantages: Low latency (data processing occurs closer to the source), reduced data volume transmitted to the cloud, increased security (data does not leave the local network), operation in conditions of limited connectivity.
  • Disadvantages: Limited computing resources on edge devices, complexity of managing and updating multiple distributed nodes, high cost of deploying and maintaining each node.
  • Examples: Local gateways, microservers in manufacturing, smart cameras with AI analysis.
• Cloud Computing (in the cloud/centralized):
  • Advantages: High scalability and flexibility, centralized management and monitoring, access to powerful computing resources and specialized services (ML, Big Data), cost savings on hardware.
  • Disadvantages: Dependence on a stable internet connection, potentially high latency for critical applications, data privacy and security concerns during transmission and storage.
  • Examples: Most IoT platforms requiring processing of large data volumes and complex analytics.

Often, the optimal solution is a hybrid approach: some data is processed at the edge for immediate reaction, while aggregated or critically important data is sent to a central cloud for long-term storage, deep analytics, and global decision-making.

Choosing IoT hosting: VPS or Dedicated Server?

The choice between a VPS and a dedicated server for IoT hosting depends on the scale of your project, performance requirements, security, and budget.

Comparison of hosting types for IoT

For startups and small projects, a VPS is an excellent solution due to its flexibility and affordability. As the number of devices and data volume grow, you can easily upgrade to a more powerful VPS or a dedicated server. We compared these options in detail in the article VPS or dedicated server: what to choose for business.

Server configuration recommendations for IoT

1. Processor (CPU): For MQTT and databases, not only frequency but also the number of cores is important. Choose processors with 4+ cores. Modern Intel Xeon E3/E5 or AMD Ryzen/EPYC are well-suited for InfluxDB and TimescaleDB.
2. Random Access Memory (RAM): IoT platforms actively use RAM for data caching, especially time-series databases. Start with 8 GB RAM for small projects and scale up to 32-64 GB and higher for large systems.
3. Disk Subsystem: Critically important. Exclusively NVMe SSD. HDDs are absolutely unacceptable due to low IOPS. For large data volumes, consider NVMe RAID 1 or RAID 10 for a balance of performance and fault tolerance.
4. Network Connection: Minimum 1 Gbps, preferably with guaranteed bandwidth. For high-load systems with many devices and active data transfer, consider 10 Gbps ports.
5. Operating System: Linux (Ubuntu Server, Debian, CentOS) — the de facto standard for IoT infrastructure due to its flexibility, performance, and extensive community.

Typical configurations and prices on Valebyte.com

At Valebyte.com, we offer flexible VPS and dedicated server plans that are ideally suited for deploying IoT platforms. Below are examples of configurations tailored for various scales of IoT projects:

• Valebyte VPS-IoT-Micro (up to 500 devices):
  • CPU: 2 vCore (Intel Xeon E3/E5)
  • RAM: 4 GB DDR4 ECC
  • Disk: 80 GB NVMe SSD
  • Port: 1 Gbps
  • Price: from $25/month
• Valebyte VPS-IoT-Standard (up to 5000 devices):
  • CPU: 4 vCore (Intel Xeon E3/E5)
  • RAM: 8-16 GB DDR4 ECC
  • Disk: 200-400 GB NVMe SSD
  • Port: 1 Gbps
  • Price: from $50/month
• Valebyte Dedicated-IoT-Pro (from 10000 devices):
  • CPU: Intel Xeon E-23xx (6-8 cores) or AMD Ryzen 9 (12-16 cores)
  • RAM: 32-64 GB DDR4 ECC
  • Disk: 1-2 TB NVMe SSD (RAID 1)
  • Port: 10 Gbps
  • Price: from $150/month
• Valebyte Dedicated-IoT-Enterprise (for critical and large-scale platforms):
  • CPU: Dual Intel Xeon E5/E7 or AMD EPYC (24+ cores)
  • RAM: 128+ GB DDR4 ECC
  • Disk: 4+ TB NVMe SSD (RAID 10)
  • Port: 10 Gbps with guaranteed bandwidth
  • Price: from $300/month

We offer flexible customization options, including choice of processors, RAM volumes, and disk subsystems, so that your IoT server precisely meets the unique requirements of your project.

Conclusion

Choosing the right server for an IoT platform with MQTT and time-series data is an investment in the stability and scalability of your project. The optimal configuration combines sufficient computing resources, a fast NVMe SSD disk subsystem, and a reliable network connection. Valebyte.com offers a wide range of VPS and dedicated servers capable of providing high performance and fault tolerance for any IoT task, from small pilot projects to large-scale industrial solutions.