Why use a dedicated server for CI/CD?
Continuous integration and continuous delivery workloads are resource-intensive because a single pipeline may compile source code, build container images, run tests, scan dependencies, package artifacts, and deploy services. Several pipelines running together can consume every CPU thread, fill temporary storage, and create high disk I/O.
A dedicated server gives Jenkins or GitLab Runner exclusive access to physical CPU cores, RAM, NVMe storage, and network capacity. This improves build-time consistency and prevents an unrelated tenant from affecting compilation or test performance. Bare metal is especially useful for Docker-heavy pipelines, Android builds, large monorepos, browser testing, machine-learning packages, and projects that require privileged virtualization or custom kernel modules.
VPS versus dedicated server for Jenkins and GitLab Runner
When a VPS is enough
A VPS works well for a small development team with short pipelines and limited concurrency. A 4 vCPU, 8 GB RAM, and 160 GB NVMe VPS can handle roughly 1–4 active jobs when builds are modest and the runner does not compile several large projects simultaneously.
- One to four concurrent jobs.
- Small Node.js, Python, PHP, Go, or Java projects.
- Light unit tests and infrequent container builds.
- Artifact storage below 100 GB.
- Builds that do not need nested virtualization, custom kernels, or sustained all-core CPU usage.
Use separate runners or autoscaling workers if production deployment credentials, untrusted pull requests, and trusted release jobs must not share the same environment.
When dedicated hardware is the better choice
Move to bare metal when builds regularly saturate the VPS, queue time becomes longer than execution time, or performance varies between runs. A dedicated server is also preferable when the runner executes untrusted code, builds large container images, runs integration-test databases, or processes many jobs with high disk activity.
- More than 4–6 concurrent jobs on a regular basis.
- More than 20 minutes of sustained compilation per job.
- Large Docker or OCI image builds using several gigabytes of temporary layers.
- Android, Chromium, Rust, C++, Unreal Engine, or other CPU-heavy builds.
- Persistent integration-test databases, Kubernetes clusters, or virtual machines on the runner.
- Strict isolation requirements for source code, secrets, or customer builds.
Recommended dedicated server specifications
CPU
Choose recent server-grade AMD EPYC or Intel Xeon hardware with at least 8 physical cores for a general-purpose CI/CD host. Physical cores matter more than a high virtual-thread count because compilation and test suites often run many CPU-bound processes at once. A 16-core server is more appropriate for 20–40 concurrent jobs or for teams that need short queue times.
Prioritize sustained all-core performance, sufficient cooling, and predictable CPU allocation over peak single-thread benchmark scores. Jenkins executors and GitLab Runner concurrency should be based on physical capacity, not simply the number of logical threads.
Memory
Start with 32 GB RAM for an 8-core build server. Allocate approximately 2–4 GB per ordinary job, then add capacity for the operating system, Docker page cache, test databases, and language toolchains. Use 64 GB for 16 cores, large Java or Android builds, browser test fleets, or multiple containers per job.
Memory pressure causes swapping, which can make builds several times slower and can terminate containers through the Linux OOM killer. Monitor memory high-water marks before increasing runner concurrency.
Storage
Use enterprise NVMe storage rather than a spinning disk or low-end SATA SSD. CI/CD creates many small files, extracts dependencies, writes compiler intermediates, and creates container layers. A 960 GB NVMe drive is a practical starting point for source checkouts, Docker images, workspaces, and cached dependencies.
Use 1.92 TB or larger when builds retain artifacts locally, repositories contain large binaries, or Docker images are rebuilt frequently. Keep at least 20–30% of the filesystem free. For important build infrastructure, use mirrored storage such as RAID 1 and store final artifacts in a separate registry or object-storage system.
Network and bandwidth
A 1 Gbps port is adequate for most teams, provided the server includes at least 10 TB monthly transfer for dependency downloads, container images, repository traffic, and artifact uploads. Select 10 Gbps networking for large binary artifacts, high-volume container registries, or more than 25 concurrent jobs moving data at the same time.
Bandwidth is not the same as latency. A runner located near the Git repository, package registry, container registry, and deployment targets can reduce pipeline duration even when its peak port speed is unchanged.
CI/CD scale-to-spec table
For up to 4 concurrent jobs, a 4 vCPU VPS is enough; past 10 jobs, use an 8-core dedicated server, and for 20–40 jobs choose 16 physical cores with 64 GB RAM.
| Concurrent CI/CD jobs | vCPU or CPU cores | RAM | Disk | Monthly bandwidth |
|---|---|---|---|---|
| 1–4 jobs | 4 vCPU | 8 GB | 160 GB NVMe | 5 TB |
| 5–12 jobs | 8 physical cores | 32 GB | 960 GB enterprise NVMe | 10 TB |
| 13–24 jobs | 12 physical cores | 48 GB | 1,920 GB enterprise NVMe | 20 TB |
| 25–40 jobs | 16 physical cores | 64 GB | 3,840 GB NVMe RAID 1 | 30 TB |
Step-by-step setup
1. Install a supported operating system
Use a current Ubuntu LTS or Debian release with a minimal package set. Create a separate administrative user, disable direct root login, and apply updates before installing Jenkins or GitLab Runner.
sudo apt update && sudo apt full-upgrade -y
sudo adduser cicd
sudo usermod -aG sudo cicd2. Configure SSH and the firewall
Install an SSH key for the administrative account, then disable password authentication after confirming key access. Permit only required services. Jenkins commonly uses TCP 8080 behind a reverse proxy, while GitLab Runner usually needs outbound access and no public inbound port.
sudo apt install -y ufw fail2ban
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow from YOUR_ADMIN_IP to any port 22 proto tcp
sudo ufw enable3. Install Docker for isolated jobs
Docker makes build environments reproducible, but adding a user to the Docker group grants root-equivalent control. Use dedicated runner hosts, trusted jobs, and short-lived agents when processing untrusted code. Install Docker from the vendor-maintained repository and verify its service.
sudo apt install -y docker.io
sudo systemctl enable --now docker
sudo docker run --rm hello-world4. Install Jenkins or GitLab Runner
Install one primary orchestration service on the host, or dedicate separate nodes for the controller and workers. Jenkins controllers should schedule work rather than execute untrusted builds directly. GitLab Runner should use the Docker executor or another isolated executor for repeatable jobs.
sudo apt install -y openjdk-17-jre
sudo systemctl enable --now jenkins
sudo systemctl status jenkinsFor GitLab Runner, register the machine with a project, group, or instance token and select the Docker executor. Protect registration tokens and use protected runners for release pipelines.
sudo apt install -y gitlab-runner
sudo gitlab-runner register
sudo systemctl enable --now gitlab-runner5. Set concurrency deliberately
Do not set concurrency equal to every logical CPU thread. Start with 6–8 jobs on an 8-core, 32 GB machine and measure CPU wait, RAM use, I/O wait, and queue time. For a 16-core, 64 GB host, start at 12–16 jobs and increase only if builds remain responsive.
sudoedit /etc/gitlab-runner/config.toml
sudo systemctl restart gitlab-runnerIn Jenkins, configure the executor count under the node settings and apply labels such as docker, android, or release so expensive jobs land on suitable workers.
6. Add monitoring and log rotation
Install node-exporter, Netdata, Prometheus, or another monitoring agent. Track CPU utilization, load average, memory available, swap activity, disk latency, filesystem capacity, network transfer, failed jobs, and queue time. Configure journald and application log rotation so diagnostic output cannot fill the root filesystem.
df -h
free -h
iostat -xz 1 5
uptime7. Test recovery before production use
Back up Jenkins configuration, credentials metadata, job definitions, and runner configuration without exposing secret values. Test restoring the controller, reconnecting workers, rebuilding a project from a clean workspace, and recovering after a full disk or server replacement.
Performance optimization tips
Cache dependencies safely
Cache Maven, Gradle, npm, pip, Go, Composer, and Cargo dependencies where appropriate. Use lockfiles and cache keys based on the operating system, architecture, and dependency checksum. A stale or shared cache can produce incorrect builds, so provide a manual cache purge path.
Use clean, disposable workspaces
Reuse of workspaces saves download time but can leave generated files that affect later jobs. Prefer ephemeral containers or clean checkout modes for release and security-sensitive pipelines. Remove abandoned workspaces and old Docker layers with a scheduled policy rather than deleting active build data.
docker system df
sudo docker image prune --filter "until=168h"Separate build classes
Use labels and runner tags to separate CPU-heavy compilation, browser tests, deployment jobs, and low-resource linting. This prevents a large C++ build from starving a release job. If different teams have conflicting security requirements, use separate dedicated workers instead of relying only on labels.
Reduce unnecessary artifact traffic
Compress large artifacts, upload only files required by downstream stages, and set retention periods. Store container images in a registry and long-term build outputs in object storage rather than keeping every artifact on the runner's local NVMe disk.
Optimize container builds
Use multi-stage Dockerfiles, pin base image versions, order stable dependency layers before frequently changing source layers, and enable BuildKit where compatible. Keep build contexts small with a precise .dockerignore file. Local caching is fast, but shared remote caches are easier to reproduce across multiple workers.
Common pitfalls to avoid
- Running everything on the controller: Jenkins controllers should not execute arbitrary pull-request code. Use agents with restricted permissions.
- Overcommitting concurrency: Forty jobs on 8 physical cores can increase queue time and total completion time. Measure throughput instead of chasing a high executor count.
- Using the root Docker socket for untrusted code: Access to the socket can provide host-level control. Use isolated workers, rootless alternatives where practical, or disposable machines.
- Ignoring disk capacity: Docker layers, package caches, test databases, and workspaces can fill a 160 GB disk quickly. Alert at 70% usage and take action before 85%.
- Keeping secrets in job logs: Mask credentials, restrict pull-request pipelines, rotate exposed tokens, and prevent verbose commands from printing environment variables.
- Relying on one server without backups: A dedicated host is not a backup. Keep configuration and artifacts in recoverable external storage and test restoration.
- Skipping patch management: Schedule operating-system, Jenkins, runner, Docker, and dependency updates. Test upgrades on a staging worker before changing the only production runner.
- Mixing trusted and untrusted builds: Fork-based pull requests can execute modified pipeline definitions. Use separate labels, permissions, credentials, and preferably separate worker hosts.
Choosing the right architecture
A small team can run Jenkins or GitLab Runner on a 4 vCPU VPS with 8 GB RAM when jobs are infrequent and builds are lightweight. An 8-core dedicated server is the practical middle ground for daily container builds, 5–12 concurrent jobs, and predictable performance. Teams with mobile builds, large monorepos, browser farms, or more than 20 concurrent jobs should use a 16-core or larger dedicated host, potentially split into specialized workers.
For the strongest isolation, keep the orchestration controller separate from build workers and provision disposable agents for untrusted code. This architecture costs more than a single server but limits credential exposure and makes capacity easier to scale. Start with measured pipeline data—average duration, peak concurrency, CPU time, RAM high-water mark, artifact size, and queue time—then adjust hardware based on evidence.