Table of Contents

last update: 1980-01-01


Max lockable memory and Opened files limit

cachegrand uses io_uring, a standard kernel component, to perform disk and network I/O operations; this component shares some memory directly with the kernel which can't get swapped and has to be locked in place once allocated.

The max_clients parameter (explained in the next section) is dependant on the amount of lockable memory, not having enough might prevent cachegrand from starting or, as it's used by other software e.g. browsers, desktop managers, etc., it might cause other kind of failures

It is also important to have a value high enough for the open files as in Linux the network connections are counted towards this limit as well, to manage 10000 connections the open files limit must be greater than 10000, it is usually safe to set it to unlimited or to a very high value.

$ ulimit -a
max locked memory       (kbytes, -l) 64
open files                      (-n) 1024

In this case, both the max locked memory and the open files limits are too low and need to be increased.

These two limits can be safely set very high or to unlimited, cachegrand will try to set very high values at the start but depending on the distribution or on the security settings the operation might not be allowed.


The SLAB Allocator in cachegrand currently relies on 2MB hugepages, it's necessary to enable them otherwise it will fail-back to a dummy malloc-based allocator that will perform extremely badly.

A simple way to enable them is to set /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages to the desired number of hugepages to allocate, bear in mind that each page is 2 MB.

Here an example on how initialize 512 hugepages (1 GB of memory).

echo 512 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

The hugepages can also be configured to at the system boot via sysctl, the specifics depend on the distribution in use.

Config file

cachegrand uses YAML for its configuration file, an example can be found in etc/cachegrand.yaml.skel.

Parameter nameValue typeDefault valueDocumentation
cpuslist (numeric, numeric ranges or all keyword)List of cpus to bind on, values can be provided as numbers, numeric ranges or the special keyword all to bind on all the available cpus
workers_per_cpusnumeric1Number of workers per cpu, suggested 1
run_in_foregroundboolfalseTrue/false flag to run cachegrand in the foregrand (currently unsupported)
pidfile_pathstring/var/run/cachegrand/cachegrand.pidPath to the pid file
use_huge_pagesboolfalseEnable or disable the slab allocator, requires hugepages available
network.backendenum (io_uring)io_uringSet the backend for the network, allowed values io_uring
network.max_clientsnumeric250Max amount of clients that can connect
network.listen_backlognumeric100Max listen backlog
module.typeenum (redis)redisSet the type of protocol, allowed values redis
module.redis.max_key_lengthnumeric8192Maximum allowed key length, it can't be greater than 65536 bytes
module.redis.max_command_lengthnumeric536870912Maximum allowed command length
module.redis.max_command_argumentsnumeric10000Maximum allowed command arguments
module.redis.strict_parsingboolfalseIf set to true, reports error when invalid combination of arguments are used (e.g., SORT with both ASC and DESC set at the same time) timeout in milliseconds, -1 to disable it or greater than 0 to enable it timeout in milliseconds, -1 to disable it or greater than 0 to enable it unsupported unsupported unsupported block of parameters to enable encryption for the protocol to the certificate in x509 format to the private key (any, tls1.0, tls1.1, tls1.2, tls1.3)anyMax TLS version allowed, allowed options any:, tls1.0, tls1.1, tls1.2, tls1.3 (any, tls1.0, tls1.1, tls1.2, tls1.3)anyMax TLS version allowed, allowed options any:, tls1.0, tls1.1, tls1.2, tls1.3 suites allowed, run //path/to/cachegrand-server --list-tls-cipher-suites to get the full list of bindings to listen on (host / port tuples) Address to bind on, can be IPv4 or IPv6 if enabled in the system to listen on, ports <= 1024 require root or disable TLS for a specific binding
database.max_keysnumeric1000000cachegrand currently doesn't autoresize the hashtable, the amount of allowed keys is fixed
database.backendenum (memory/file)memorySet the type of backend, allowed values memory and file
database.filelistThe current implementation of the file backend is a PoC and it's limited in performances and functionalities
database.file.pathstring/var/lib/cachegrandPath to a folder to be used for the shards
database.file.shard_size_mbnumeric100Maximum size of a shard in MB
database.file.max_opened_shardsnumeric1000Maximum number of shards opened (unsupported)
sentry.enableboolfalseIf enabled and if the dsn is provided, in case of a crash a minidump is automatically generated and uploaded to - data stored in cachegrand get be uploaded if part of the stacktrace!
sentry.dsnstringDSN to use with the service
logslistList of log sinks
logs.typeenum (console/file)console and file
logs.levellist set (all, debug, verbose, info, warning, recoverable, error, no-debug, no-verbose, no-info, no-warning, no-recoverable, no-error)all, no-verbose, no-debugLog level
logs.file.pathstring/var/log/cachegrand/cachegrand.logPath to the log file