How to scale a server as the load grows

There are two main ways to scale a server as load increases: vertical scaling, by increasing the resources (CPU, RAM, disk) of a single server, or horizontal scaling, by adding new servers to a cluster and distributing the load among them using a load balancer.

Audience growth, increasing data volumes, or launching new features inevitably lead to higher demands on server infrastructure. Ignoring these signals results in slowdowns, errors, and loss of users. At Valebyte.com, we understand how critical it is to ensure the uninterrupted operation of your applications, so let's explore when and how to effectively apply server scaling strategies.

Why is server scaling necessary? Signs that it's time to scale

Before you start scaling, it's important to recognize the need for it. Monitoring is a key tool for identifying bottlenecks. Here are the main signs indicating that your server requires server scaling:

• High CPU utilization: If the processor regularly runs at 80-100% during peak hours, this is a signal of insufficient computing power. The top or htop command will show current utilization.
• Insufficient RAM: Active use of a swap file slows down the system as data is moved to disk. You can check this with the command free -h.
• Slow disk subsystem performance: Long read/write operations, especially when working with databases or a large number of files, indicate slow disks (HDD instead of NVMe SSD).
• High network latency: If the network interface is constantly busy (e.g., 90% of 1 Gbps), this can be a reason for slow content delivery.
• Long application response time: Users complain about slow page loading or operation execution. This can be tracked using APM systems (Application Performance Monitoring).
• Frequent 5xx errors: Errors such as 500 (Internal Server Error), 502 (Bad Gateway), 503 (Service Unavailable) often indicate server or component overload.

Regular monitoring of these metrics allows for proactive response to load growth and planning for server scaling before problems become critical.

# Example htop output to assess CPU and RAM usage
  PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  COMMAND
 1234 www-data   20   0 1.2G  150M 50M S 85.0  1.5  0:25.12 php-fpm
 5678 mysql      20   0 2.5G  500M 100M S 15.0  5.0  0:10.45 mysqld
    1 root       20   0 120M  5M   3M S  0.0  0.0  0:01.00 systemd

Vertical Scaling (Scaling Up): When is it effective?

Vertical scaling, or scaling up, means increasing the resources of a single existing server. You simply add more RAM, a faster processor, a larger and/or faster disk (e.g., NVMe SSD), or increase network bandwidth. This is the simplest way to scale a server, often requiring no significant changes to the application architecture.

Advantages and Disadvantages of Vertical Scaling

• Advantages:
  • Simplicity of implementation: Does not require changes to application code or complex network infrastructure setup.
  • Less complexity: Managing one powerful server is simpler than managing a cluster of several.
  • Efficiency for monolithic applications: Ideal for applications not originally designed for distributed operation.
• Disadvantages:
  • Limited ceiling: There are physical limits for a single server (maximum number of CPU cores, RAM volume).
  • Single Point of Failure (SPOF): Failure of one server leads to the unavailability of the entire application.
  • Downtime during upgrades: Usually requires a server reboot, which means temporary service unavailability.
  • Cost: Beyond a certain threshold, each subsequent performance increase becomes exponentially more expensive. For example, a server with 128 GB RAM and 32 vCPU can cost significantly more than two servers with 64 GB RAM and 16 vCPU each.

Vertical scaling is excellent for startups, small to medium-sized projects, and applications that cannot be easily broken down into microservices. If you are looking for maximum performance for a single machine, consider powerful dedicated servers with AMD EPYC and Intel Xeon, which Valebyte.com offers for enterprise tasks.

Horizontal Scaling (Scaling Out): A Solution for High Loads

Horizontal scaling, or scaling out, involves adding new servers to your infrastructure to distribute the load. Instead of making one server more powerful, you add several less powerful servers that work in parallel. This allows you to handle a significantly larger volume of requests and provides high fault tolerance.

Advantages and Disadvantages of Horizontal Scaling

• Advantages:
  • Virtually unlimited scalability: You can add as many servers as needed to handle any load.
  • High availability and fault tolerance: The failure of one server does not lead to a complete system crash, as the load is automatically redistributed to the remaining servers.
  • No downtime during upgrades: New servers can be added or removed without stopping service operation.
  • Cost-effectiveness: It is often cheaper to use several medium-sized servers than one very powerful one, especially at large scales.
• Disadvantages:
  • Architectural complexity: Requires the use of a load balancer, distributed databases, queuing systems, and other components.
  • Application changes: The application must be designed as "stateless" (without preserving state on the server) so that any request can be processed by any of the servers. Sessions and user data must be stored in external storage (Redis, database).
  • Data management: Data synchronization between multiple servers (especially for databases) becomes more complex.
  • Initial setup cost: Requires more effort and infrastructure investment at the start.

Horizontal scaling is the standard for large web applications, SaaS platforms, e-commerce projects, and high-traffic services. For example, for a server for a SaaS application, such an architecture would be optimal.

Load Balancer: Key to Effective Horizontal Scaling

A load balancer is a critically important component in a horizontal scaling architecture. Its primary task is to evenly distribute incoming network traffic among multiple servers so that none of them become overloaded.

Load balancers perform several key functions:

• Traffic distribution: Use various algorithms (Round Robin, Least Connections, IP Hash) to direct requests to the least loaded or most suitable servers.
• Health Checks: Continuously monitor the availability and operational status of servers in the pool. If a server fails, the load balancer stops directing traffic to it.
• SSL Termination: Can handle SSL encryption, offloading this burden from backend servers.
• High availability: The load balancer itself often operates in Active-Passive or Active-Active mode to avoid a single point of failure.

Popular load balancing solutions include Nginx, HAProxy, AWS ELB/ALB, Google Cloud Load Balancing, and others. Nginx is often used as a high-performance and flexible software load balancer.

# Example Nginx load balancer configuration
http {
    upstream backend_servers {
        # Round Robin algorithm (default)
        server backend1.example.com;
        server backend2.example.com;
        server backend3.example.com;
    }

    server {
        listen 80;
        server_name yourdomain.com;

        location / {
            proxy_pass http://backend_servers;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
    }
}

Additional Optimization and Scaling Strategies

Beyond choosing between vertical and horizontal scaling, there are other important approaches to optimizing performance and reducing server load.

CDN (Content Delivery Network)

A CDN is a distributed network of servers that cache static content (images, videos, CSS, JavaScript) and deliver it to users from the closest geographical point. This significantly reduces the load on your main server, improves loading speed for users worldwide, and increases fault tolerance. Using a CDN is especially critical for global projects and websites with a large volume of media content.

Data Caching

Caching is the process of temporarily storing frequently used data to speed up its subsequent retrieval. There are several levels of caching:

• Browser caching: The web server tells the browser how long to store static content.
• Application-level caching: Storing the results of expensive computations or database queries in RAM or specialized stores (Redis, Memcached).
• Database caching: Configuring the DBMS to use its own query and data cache.
• Reverse proxy caching (Varnish, Nginx): Caching full HTML pages or API responses at the web server level.

Properly configured caching can significantly reduce the load on the database and server computing resources.

Code and Database Optimization

Often, the cause of low performance is not a lack of resources, but inefficient code or a poorly configured database. Code refactoring, optimizing database queries, creating proper indexes, and denormalizing data (where appropriate) can provide a significant performance boost without additional hardware investment.

# Example of creating an index to speed up MySQL queries
ALTER TABLE users ADD INDEX idx_email (email);

Comparison of Vertical vs. Horizontal Scaling

To help you make a decision, let's compare the key aspects of the two main server scaling strategies:

How to choose a scaling strategy for your server?

Choosing the optimal server scaling strategy depends on many factors, including current load, application architecture, budget, and long-term plans. Here are some recommendations:

1. Start with monitoring: Before scaling, precisely identify what the "bottleneck" is. This could be CPU, RAM, disk I/O, or network bandwidth. Invest in good monitoring tools.
2. Optimize the application: Often, optimizing code, database queries, and proper caching can postpone the need for scaling or reduce its scope. This is cheaper than buying new hardware.
3. Consider vertical scaling as a first step: For many projects, especially in the early stages, increasing the resources of an existing VPS or dedicated server is the fastest and simplest way to solve performance problems. For example, upgrading from 4 vCPU/8GB RAM to 8 vCPU/16GB RAM might cost an additional $50-100/month and significantly improve the situation.
4. Plan for horizontal scaling in advance: If you expect exponential growth or your application is inherently designed as distributed (e.g., microservices architecture), start thinking about horizontal scaling early. This will help avoid costly reworks in the future.
5. Use CDN and caching: Regardless of the chosen strategy, implementing a CDN for static content and various levels of caching will always be beneficial, reducing the load on the backend.
6. Choose the right hosting type: For rapid vertical scaling, VPS hosting offers flexibility, allowing easy plan changes. Horizontal scaling often requires multiple VPS or dedicated servers. You can read more about choosing in our article VPS or dedicated server: what to choose for business.
7. Load test: Before implementing new components or significantly changing the architecture, conduct load testing to ensure that the changes truly solve the problem and do not create new ones.

Conclusion

Effective server scaling is not a one-time action but a continuous process requiring monitoring, analysis, and strategic planning. The choice between vertical and horizontal scaling, as well as the application of additional strategies such as CDN and caching, should be driven by the unique requirements of your project and its projected growth. At Valebyte.com, we offer flexible VPS and dedicated server solutions to help you effectively scale your infrastructure at any stage of development.