You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When we match a page in the btree search we then delegate to the sub-index to search that page. This works well for high cardinality data and equality / inequality searches. However, it can be inefficient when the data is low cardinality since we have to search a very large page (or many small pages)
Luckily, when we search the btree, we can know if a page is entirely included in the result. In other words, if a page has min: 5, max: 10 and the query is value < 20 then we don't need to search the page because we know we want all of the rows in the page. If we had stored a compressed bitmap of all the row ids in the page then all we would need to do would be to load that.
The text was updated successfully, but these errors were encountered:
This issue has high synergy with #1551 . For example, consider a low cardinality column with only 5 choices and 100 million rows. The query is then something like color == 'blue'. Satisfying this query today is pretty expensive. We need to search all of the tiny pages that match that query.
On the other hand, if we only had five pages, and each page had their own compressed bitmap, then we could satisfy the query very quickly with a single IOP to read the compressed bitmap for the correct page.
This is basically giving us the benefits of both btree indices AND bitmap indices in one combined structure.
When we match a page in the btree search we then delegate to the sub-index to search that page. This works well for high cardinality data and equality / inequality searches. However, it can be inefficient when the data is low cardinality since we have to search a very large page (or many small pages)
Luckily, when we search the btree, we can know if a page is entirely included in the result. In other words, if a page has
min: 5, max: 10
and the query isvalue < 20
then we don't need to search the page because we know we want all of the rows in the page. If we had stored a compressed bitmap of all the row ids in the page then all we would need to do would be to load that.The text was updated successfully, but these errors were encountered: