- RocksDB has been bumped to
v5.18.3, which fixes a dead-lock occuring in RocksDB at scale when a compaction task is ran under heavy disk writes (ie. disk flushes). This dead-lock was causing Sonic to stop responding to any command issued for the frozen collection. This dead-lock was due to a bug in RocksDB internals (not originating from Sonic itself) [@baptistejamin, 19c4a10].
- Reworked the
FLUSHBcommand internals, which now use the atomicdelete_range()operation provided by RocksDBv5.18[@valeriansaliou, 660f8b7].
- Added the
LANG(<locale>)command modifier forQUERYandPUSH, that lets a Sonic Channel client force a text locale (instead of letting the lexer system guess the text language) [@valeriansaliou, #75]. - The FST word lookup system, used by the
SUGGESTcommand, now support all scripts via a restricted Unicode range forward scan [@valeriansaliou, #64].
- A store acquire lock has been added to prevent 2 concurrent threads from opening the same collection at the same time [@valeriansaliou, 2628077].
- A superfluous mutex was removed from KV and FST store managers, in an attempt to solve a rare dead-lock occurring on high-traffic Sonic setups in the KV store [@valeriansaliou, 60566d2].
- Reverted changes made in
v1.1.5regarding the open filesrlimit, as this can be set from outside Sonic [@valeriansaliou, f6400c6]. - Added Chinese Traditional stopwords [@dsewnr, #87].
- Improved the way database locking is handled when calling a pool janitor; this prevents potential dead-locks under high load [@valeriansaliou, fa78372].
- Added the
server.limit_open_filesconfiguration variable to allow configuringrlimit[@valeriansaliou].
- Added Kannada stopwords [@dileepbapat].
- The Docker image is now much lighter [@codeflows].
- Automatically adjust
rlimitfor the process to the hard limit allowed by the system (allows opening more FSTs in parallel) [@valeriansaliou].
- Limit the size of words that can hit against the FST graph, as the FST gets slower for long words [@valeriansaliou, #81].
- Rework Sonic Channel buffer management using a VecDeque (Sonic should now work better in harsh network environments) [@valeriansaliou, 1c2b9c8].
- FST graph consolidation locking strategy has been improved even further, based on issues with the previous rework we have noticed at scale in production (now, consolidation locking is done at a lower-priority relative to actual queries and pushes to the index) [@valeriansaliou, #68].
- FST graph consolidation locking strategy has been reworked as to allow queries to be executed lock-free when the FST consolidate task takes a lot of time (previously, queries were being deferred due to an ongoing FST consolidate task) [@valeriansaliou, #68].
- Removed special license clause introduced in
v1.0.2, Sonic is fullMPL 2.0now. [@valeriansaliou]
- Change how buckets are stored in a KV-based collection (nest them in the same RocksDB database; this is much more efficient on setups with a large number of buckets -
v1.1.0is incompatible with thev1.0.0KV database format) [@valeriansaliou].
- Bump
jemallocatorto version0.3[@valeriansaliou].
- Re-license from
MPL 2.0toSOSSL 1.0(Sonic has a special license clause) [@valeriansaliou].
- Added automated benchmarks (can be ran via
cargo bench --features benchmark) [@valeriansaliou]. - Reduced the time to query the search index by 50% via optimizations (in multiple methods, eg. the lexer) [@valeriansaliou].
- Initial Sonic release [@valeriansaliou].