Need info on how to handle 1 billion keys #6441
-
My use case if to reference person's first name, last name & SSN to uniquely identify .Insert that record in key value on if thats unique.If my records grow to 1 billion I want to search unique person and insert only if not existing |
Beta Was this translation helpful? Give feedback.
Answered by
derekcollison
Feb 3, 2025
Replies: 1 comment
-
Currently subject based indexes for lookup are held in memory. So I would suggest using a hashing approach to use multiple streams / KVs and size accordingly with memory. We are looking at improving the memory footprint for very large key/subject spaces. |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
derekcollison
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Currently subject based indexes for lookup are held in memory.
So I would suggest using a hashing approach to use multiple streams / KVs and size accordingly with memory.
We are looking at improving the memory footprint for very large key/subject spaces.