-
Notifications
You must be signed in to change notification settings - Fork 37
nbase arc 1.3 release
Major features of nbase-arc 1.3 release are as follows.
- Background key delete
- Incremental triple-S object purge
- More reliable workflow
- In memory replication support
Before nbase-arc 1.3 (and original Redis), a key is always deleted in main thread in common with other commands. When a big value (hash, set etc.) which has many elements in it is deleted by DEL command or TTL(time to live) expiration, it takes a considerable amount of time (even second), and it is one of the reasons of the latency problem.
nbase-arc 1.3 implements background key delete feature where if the number of elements in a object exceeds configurable threshold (object-bio-delete-min-elems), object destruction is performed in background thread.
The triple-S (s3) object is a hierarchical map structure in which user can manipulate list or set of values where each value has its own TTL. For TTL management of the s3 object values, nbase-arc Redis server has fixed size array which is called s3 GC lines where each line is a linked list header of s3 objects.
Before nbase-arc 1.3, GC lines are purged at fixed rate (e.g. every 10 seconds) line by line in the form of internal job. If there is a big s3 object (which has lots of values) or if there are many s3 objects in a line, this fixed rate purge job causes latency problem because every purge cleans at least one GC line.
In nbase-arc 1.3, fixed rate GC line purge has time bound and if there are remaining s3 objects after the purge, these objects are moved to internal eager purge list. S3 objects in the eager list are purged more frequently.
In nbase-arc 1.3 fail-over and some other workflows performed by the configuration master are completely rewritten based on color based state model. In this model, we divide the state of the a PGS(redis and replictor) into four colors (red, yellow, green, blue) and defined a precise condition and actions based on the state perceived by the configuration master.
Workflows work even if there is a failure (e.g. timeout) in the middle of a workflow execution.
In nbase-arc 1.3, any cluster can be managed in in-memory replication mode by the management tool.
When the option is set, every PGS have a fixed size shared memory area and most recent replication logs are stored in it. The logs in the shared memory area are transferred and synced to disk logs by background thread of the replicator.
When a Redis server or Replicator attaches (via mmap call) the shared memory area, It tries 'mlock' system call to prevent the shared memory area from being paged out by the system.
This option separates read, write log operations from create and sync log operations which are sometimes cause delay from 10 to 100 milliseconds.