Design Draft

Internal Codec Format

General key format is <namespace> <key> <type> <element...>, while general value format is <type> <element...>. If the storage backend support namespace, the namespace part would not be encoded in the key.

                     KeyObject                                                   ValueObject
String      [<namespace>]  <key> KEY_META                                   KEY_STRING <MetaObject>
Hash        [<namespace>]  <key> KEY_META                                   KEY_HASH <MetaObject>
            [<namespace>]  <key> KEY_HASH_FIELD <field>                     KEY_HASH_FIELD <field-value>
Set         [<namespace>]  <key> KEY_META                                   KEY_SET <MetaObject>
            [<namespace>]  <key> KEY_SET_MEMBER <member>                    KEY_SET_MEMBER
List        [<namespace>]  <key> KEY_META                                   KEY_LIST <MetaObject>
            [<namespace>]  <key> KEY_LIST_ELEMENT <index>                   KEY_LIST_ELEMENT <element-value>
Sorted Set  [<namespace>]  <key> KEY_META                                   KEY_ZSET <MetaObject>
            [<namespace>]  <key> KEY_ZSET_SCORE <member>                    KEY_ZSET_SCORE <score>
            [<namespace>]  <key> KEY_ZSET_SORT <score> <member>             KEY_ZSET_SORT 
Stream      [<namespace>]  <key> KEY_META                                   KEY_STREAM <MetaObject>
            [<namespace>]  <key> KEY_STREAM_ELEMENT <stream_id>             KEY_STREAM_ELEMENT <element-value>
            [<namespace>]  <key> KEY_STREAM_PEL <group> <empty>(first)      KEY_STREAM_PEL <stream_id> 
            [<namespace>]  <key> KEY_STREAM_PEL <group> <stream_id>         KEY_STREAM_PEL <consumer> <deliver_count> <deliver_time>

Comparator

For general key format<namespace> <key> <type> <element...>, the comparator would compare namespace first, then the key, then type, then the key elemetns.

Expire

• TTL value is always stored in MetaObject for any meta key.
• Custom Compact Filter Support(rocksdb/forestdb)
  • If storage engine support compact filter, we could check the ttl time while compacting.
• No Custom Compact Filter
  • New key type KEY_TTL_SORT introduced for the background expire checking task.
  • The KEY_TTL_SORT key format is [<namespace>] "" KEY_TTL_SORT <ttl> <key's namespace> <key>, the key part before KEY_TTL_SORT is empty.
  • A background task scan KEY_TTL_SORT periodlly to delete expired keys.