Configuration

Max lockable memory and Opened files limit

cachegrand uses io_uring, a standard kernel component, to perform disk and network I/O operations; this component shares some memory directly with the kernel which can't get swapped and has to be locked in place once allocated.

The max_clients parameter (explained in the next section) is dependant on the amount of lockable memory, not having enough might prevent cachegrand from starting or, as it's used by other software e.g. browsers, desktop managers, etc., it might cause other kind of failures

It is also important to have a value high enough for the open files as in Linux the network connections are counted towards this limit as well, to manage 10000 connections the open files limit must be greater than 10000, it is usually safe to set it to unlimited or to a very high value.

$ ulimit -a
...
max locked memory       (kbytes, -l) 64
...
open files                      (-n) 1024
...

In this case, both the max locked memory and the open files limits are too low and need to be increased.

These two limits can be safely set very high or to unlimited, cachegrand will try to set very high values at the start but depending on the distribution or on the security settings the operation might not be allowed.

Hugepages

The SLAB Allocator in cachegrand currently relies on 2MB hugepages, it's necessary to enable them otherwise it will fail-back to a dummy malloc-based allocator that will perform extremely badly.

A simple way to enable them is to set /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages to the desired number of hugepages to allocate, bear in mind that each page is 2 MB.

Here an example on how initialize 512 hugepages (1 GB of memory).

echo 512 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

The hugepages can also be configured to at the system boot via sysctl, the specifics depend on the distribution in use.

Config file

cachegrand uses YAML for its configuration file, an example can be found in etc/cachegrand.yaml.skel.