Ollama Server Requirements: RAM, VRAM, and CPU by Model Size

For Ollama, the hardware requirements are 16 GB of RAM for a 7B model with CPU inference or 8–12 GB of VRAM with GPU inference, while a 70B model needs at least 64 GB of RAM or 48 GB of VRAM, respectively.

What Ollama server requirements look like for different models

What 3B, 7B, 13B, and 70B model sizes mean

Ollama runs large language models locally through llama.cpp and compatible backend components. Inference can run on the CPU, GPU, or in a hybrid mode, where some model layers are placed in video memory and the rest in system RAM. As a result, ollama server requirements depend not only on the model name but also on the specific tag: for example, 7B Q4_K_M and 7B in FP16 require fundamentally different amounts of memory.

The numbers 3B, 7B, 13B, and 70B indicate the approximate number of billions of parameters. Q4, Q5, and Q8 quantization reduces model weight size and memory requirements, but may slightly reduce accuracy. For server deployments, Q4_K_M or Q5_K_M are the most common choices because they offer a sensible balance of response quality, speed, and hardware cost.

Why actual RAM usage is higher than the model file size

The model file is not the only consumer of memory. Ollama also uses RAM or VRAM for the KV cache, context, working buffers, tokenizer, and operating system. With an 8K–16K context, memory usage increases noticeably faster, especially for 13B and 70B models. With multiple parallel requests, the KV cache grows almost proportionally to the number of concurrent users.

Practical server requirements for self-hosted AI and LLM workloads help estimate not only model weight size but also the headroom needed for context, API services, monitoring, and disk cache.

Ollama hardware requirements: RAM and VRAM by model size

CPU and GPU inference requirements at a glance

The figures below are guidelines for Q4_K_M quantized models with a context of up to 8K and one active request. Speed is shown in tokens per second and depends on CPU architecture, GPU generation, memory bandwidth, context length, and Ollama version.

Model size	Minimum RAM for CPU	Recommended VRAM for GPU	Recommended CPU	Disk for model weights	Approximate speed
3B	8 GB	4 GB	4 vCPU, 3,0+ GHz	8–12 GB NVMe	CPU: 15–35 tokens/s; GPU: 40–100 tokens/s
7B	16 GB	8–12 GB	6–8 vCPU, 3,0+ GHz	10–16 GB NVMe	CPU: 8–20 tokens/s; GPU: 35–90 tokens/s
13B	24–32 GB	16 GB	8–12 vCPU, 3,2+ GHz	18–28 GB NVMe	CPU: 4–10 tokens/s; GPU: 20–55 tokens/s
70B	64–96 GB	48–80 GB	16–32 vCPU, 3,0+ GHz	45–60 GB NVMe	CPU: 0,8–3 tokens/s; GPU: 8–25 tokens/s

The table shows memory required to run the model, not the total VPS size. On a server with 16 GB of RAM, a 7B model leaves little headroom if Nginx, an API, a database, and monitoring are running at the same time. For a stable production deployment, it is better to plan for 24 GB of RAM or use a GPU with 12 GB of VRAM and 16 GB of system memory.

FP16 models require approximately twice as much memory as Q8 and four times as much as Q4, although the exact amount depends on the architecture. For example, a 7B FP16 model may use around 14–16 GB just for its weights, while a 7B Q4 model typically fits in 4–6 GB.

Ollama CPU vs GPU inference: which should you choose?

When CPU inference makes sense

Ollama CPU vs GPU inference is a trade-off between lower cost and higher speed. A CPU is suitable for development, prompt testing, background classification, embedding generation, and occasional administrative requests. A 3B model on 4–8 vCPU is usually fast enough for a personal assistant, while a 7B model on 8–16 vCPU works for one user with a moderate context.

CPU inference also makes sense when the model must run around the clock but requests are infrequent. A server with 32 GB of RAM and fast NVMe can serve a 7B or 13B model without a dedicated GPU. The downside is clear: the first token appears more slowly, and generation with 13B—and especially 70B—becomes noticeably less comfortable.

When you need a GPU

A GPU is needed for an interactive API, a chatbot with multiple users, high-frequency RAG retrieval, coding in an IDE, and generating long responses. For 7B, 8–12 GB of VRAM is appropriate; 13B needs approximately 16 GB; and 70B requires one or more GPUs with a combined 48–80 GB of VRAM.

The GPU does not always need to hold the entire model. Ollama can offload some layers to video memory and keep the rest in RAM, but hybrid execution is usually slower because data must move over PCIe. If consistent latency matters, choose enough VRAM to leave 15–25% headroom over the model and KV cache requirements.

Choosing an accelerator for 7B and 70B models depends heavily on memory type, bus width, and GPU count; this separate guide to GPUs for LLM inference compares these scenarios in detail.

How much RAM does Ollama need for context and multiple users?

KV cache and context length

You cannot answer “how much RAM does Ollama need?” based only on the model file size. During generation, the model stores the token history in the KV cache. The longer the context and the more parallel conversations you have, the more memory is required. Increasing the context from 4K to 16K tokens can multiply memory usage, while four simultaneous requests require approximately four times more KV cache than one.

For one user and an 8K context, the following headroom is usually sufficient:

• 3B: 8 GB RAM or 4 GB VRAM;
• 7B: 16–24 GB RAM or 8–12 GB VRAM;
• 13B: 32 GB RAM or 16–24 GB VRAM;
• 70B: 96 GB RAM for CPU inference or 64 GB of combined VRAM for comfortable operation.

If you need a 32K context, add more headroom to these figures. For parallel requests, size the server according to the actual workload profile. For a production server, it is sensible to keep 20–30% of RAM free so that Linux, the Ollama API, the reverse proxy, and background processes do not compete with the model for memory.

Parallel requests and queuing

Increasing the number of users does not turn one GPU into several independent accelerators. Requests can be processed in parallel, but each one receives a share of the compute resources and KV cache. If a GPU produces 30 tokens per second for one request, the average speed per request may drop substantially with four active conversations.

For a small API, it is more effective to limit concurrency and add a queue than to allow RAM to become exhausted by simultaneous requests. Set a sensible context limit, timeouts, and maximum number of active requests in the configuration.

OLLAMA_HOST=0.0.0.0:11434
OLLAMA_NUM_PARALLEL=2
OLLAMA_MAX_LOADED_MODELS=1
OLLAMA_KEEP_ALIVE=10m

Environment variables depend on the Ollama version and how the service is started. Do not expose port 11434 directly to the internet: place a reverse proxy in front of it and add authentication and network access controls.

Ollama server specs by model size: disk, CPU, and networking

How much disk space do Ollama model weights need?

Ollama works best with a local NVMe drive. It speeds up model downloads, extraction, and loading into RAM or VRAM. The minimum disk capacity should account not only for the weights of one model but also for multiple tags, temporary files, logs, and update headroom.

• 3B: minimum 20 GB, 40 GB NVMe recommended;
• 7B: minimum 30 GB, 60 GB NVMe recommended;
• 13B: minimum 50 GB, 80–100 GB NVMe recommended;
• 70B: minimum 100 GB, 150–200 GB NVMe recommended.

If you keep Q4, Q5, and Q8 versions of the same model, storage usage grows quickly. A 60 GB disk may be enough for one 7B model, but it becomes tight once you add several models, Docker images, and system logs.

CPU, PCIe, and network port requirements

For CPU inference, physical cores, clock speed, and memory bandwidth matter. Eight fast vCPUs often outperform 16 slow virtual threads. For a GPU server, you also need sufficient PCIe lanes, a compatible driver, adequate power, and the ability to dedicate the GPU without heavy contention from other workloads.

Within a single server, a 1 Gbps network port is usually sufficient: requests and responses are transmitted, not the model weights on every request. For a remote API with many users or streaming output, consider 1–10 Gbps, although inference is more often the bottleneck than networking.

Ollama server requirements by workload scale

Workload scale → VPS and dedicated server specifications

For 10 concurrent users running a 7B model, 8 vCPU, 32 GB of RAM, a 100 GB NVMe disk, and a 1 Gbps port are sufficient.

Workload scale	vCPU	RAM	Disk	Network port	Price
1 user, 3B–7B	4	16 GB	40 GB NVMe	1 Gbps	approximately $15–25/month, March 2026
5 concurrent users, 7B	8	32 GB	80 GB NVMe	1 Gbps	approximately $35–60/month, March 2026
10 concurrent users, 7B–13B	12	64 GB	120 GB NVMe	1 Gbps	approximately $70–120/month, March 2026
25 concurrent users, 13B with GPU	16	64 GB	200 GB NVMe	1–10 Gbps	approximately $180–350/month, March 2026
50 concurrent users, 70B	32	128 GB	300 GB NVMe	10 Gbps	approximately $500–1000/month, March 2026

The prices in the table are general market estimates for March 2026, not fixed rates from an external provider: the final cost depends on region, GPU type, resource allocation guarantees, traffic, and rental term. For several users running a 3B–7B model, a VPS is usually more economical than a dedicated server, while for 13B–70B models, a GPU node often delivers more predictable latency.

When choosing a Valebyte configuration, determine in advance whether you need a persistent GPU, how many models will be loaded simultaneously, and what maximum context length is required. For CPU workloads, it is useful to compare virtual resources with a physical server: the difference in CPU frequency stability is described in detail in this guide to bare metal vs VPS for CPU-based ML inference.

How to install Ollama and check available hardware

Install Ollama and download a model

On Linux, installation is usually completed with a single command, after which you download a model through the Ollama CLI. For production environments, use a dedicated system user, a systemd service, and a firewall.

curl -fsSL https://ollama.com/install.sh | sh

ollama pull llama3.1:8b
ollama run llama3.1:8b

A specific tag may have a different size than expected: check the model page and available disk space before downloading. To test the API:

curl http://127.0.0.1:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.1:8b","prompt":"Проверь скорость ответа","stream":false}'

Check CPU, RAM, and GPU resources

lscpu
free -h
lsblk
nvidia-smi
ollama ps

The ollama ps command shows the loaded model and where it is placed in memory. If the placement field shows a split between CPU and GPU, there is not enough VRAM to keep the entire model on the GPU. On Linux, also check free -h during an actual request rather than only while the server is idle: peak RAM usage occurs when the weights are loaded and the context is created.

How to choose a Valebyte configuration for Ollama

VPS for testing and small teams

A VPS with 4 vCPU and 16 GB of RAM is suitable for 3B and most 7B models during development. A configuration with 8 vCPU and 32 GB of RAM is more comfortable for one or two users, an RAG prototype, and an API with limited concurrency. NVMe is essential if models change frequently or the server runs automated pipelines.

Before ordering, check four parameters: the guaranteed CPU share, disk type, outbound traffic limit, and whether you can install the required drivers. A GPU is not needed for CPU-only workloads, but for 13B and 70B models, the cost of RAM and response time may make this approach uneconomical.

GPU server for production workloads

Choose a GPU server based on VRAM, not just compute performance. 8–12 GB of VRAM is enough for 7B, 16–24 GB is preferable for 13B, and 48 GB or more is needed for 70B. Long contexts and multiple users require additional headroom; otherwise, Ollama will start offloading layers to RAM.

• for a chatbot with up to 5 users: 7B, 8–12 GB VRAM, 32 GB RAM;
• for an internal assistant serving 10–25 users: 13B, 16–24 GB VRAM, 64 GB RAM;
• for a high-quality 70B model: 48–80 GB VRAM, 128 GB RAM, and 24–32 vCPU;
• for high availability: two nodes, model backups, and an API health check.

If Ollama runs alongside a vector database, reranker, and document services, account for their RAM requirements separately. Self-hosted OCR, search, and file-processing applications may require more memory than the LLM size suggests; the same hardware-selection approach is covered in this guide to self-hosted applications and server workload requirements.

Frequently Asked Questions

How much RAM does Ollama need for a 7B model?

For 7B Q4_K_M, a server with 16 GB of RAM is the minimum practical option, but 24–32 GB is a better choice for real-world use. This leaves memory for the operating system, KV cache, an 8K context, and API services. With four parallel requests or a 16K context, 32 GB may not be enough without queue limits.

Can Ollama run without a GPU?

Yes, Ollama runs on the CPU. 3B and 7B models are suitable for CPU inference on 4–8 fast vCPU, while 13B requires approximately 8–12 vCPU and 32 GB of RAM. A 70B model can run on the CPU with 64–96 GB of RAM, but speeds are often only 0,8–3 tokens per second, so a GPU is preferable for interactive use.

How much VRAM does a 13B model need?

A 13B Q4 model typically needs 10–12 GB of memory for its weights, but a GPU with 16 GB of VRAM is recommended to cover the KV cache and leave headroom. With a 16K context or multiple users, 24 GB of VRAM is preferable. Alternatively, you can combine the GPU with 32–64 GB of system RAM, bearing in mind that partial CPU offloading reduces speed.

How much disk space does an Ollama model use?

A quantized 7B model typically uses around 4–8 GB, a 13B model uses 8–15 GB, and a 70B model uses approximately 40–50 GB. For updates, multiple tags, and temporary files, plan for at least 40 GB for 3B–7B, 100 GB for 13B, and 150–200 GB of NVMe storage for 70B.

Conclusion

A VPS with 4–8 vCPU and 16–32 GB of RAM is suitable for 3B–7B models, while interactive use by multiple users is better served by a GPU with 8–12 GB of VRAM. 13B and 70B models require 16–24 GB and 48–80 GB of VRAM, respectively, or 32–96 GB of RAM for CPU inference, so choose a Valebyte configuration based on the model tag, context length, and number of parallel requests.