Key-value store over gRPC

A C++ key-value server that answers GET, PUT and DEL from many clients at once over gRPC, with an in-memory cache in front of on-disk storage. Built for a course at IIT Bombay (2022).

How it works

• Async gRPC server. Requests arrive on gRPC completion queues and are handled by a fixed pool of worker threads (THREAD_POOL_SIZE).
• Cache in front of disk. Reads are served from an in-memory cache when possible. The eviction policy is set in the config file: LRU (least recently used) or LFU (least frequently used), so the two can be compared on the same workload.
• Storage split across 64 files. Each key is hashed (djb2) to one of 64 files, and each file has its own read-write lock. Requests for keys in different files never wait on each other.

Configuration

server_config.txt:

LISTENING_PORT=50052
CACHE_REPLACEMENT_TYPE=LRU     # or LFU
CACHE_SIZE=512
THREAD_POOL_SIZE=20

Build and run

Needs a C++ compiler, CMake, gRPC and Protocol Buffers. The project was built inside gRPC's C++ examples tree: CMakeLists.txt expects the service definition at ../../protos/keyvalue.proto, which is not in this repository.

mkdir -p cmake/build && cd cmake/build
cmake ../.. && make          # builds two binaries: server and client

./server                     # reads server_config.txt from the current directory
./client                     # interactive menu: 1 GET, 2 PUT, 3 DEL (connects to localhost:50052)

Files

File	What it holds
server.cc	gRPC server, completion queues, worker threads
client.cc	command-line client
cache.cpp	LRU and LFU caches
persistentstorage.h	the 64 hashed files and their locks
server_config.txt	port, cache policy and size, thread count