Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

re_datastore: optimize backwards walk in read path #453

Closed
Tracked by #443 ...
teh-cmc opened this issue Dec 2, 2022 · 2 comments
Closed
Tracked by #443 ...

re_datastore: optimize backwards walk in read path #453

teh-cmc opened this issue Dec 2, 2022 · 2 comments
Labels
🏹 arrow Apache Arrow ⛃ re_datastore affects the datastore itself

Comments

@teh-cmc
Copy link
Member

teh-cmc commented Dec 2, 2022

This currently does a naive linear walk for as long as the component isn't set:

            .filter_map(|(name, index)| {
                let mut row_idx = time_row_idx;
                while !index.is_valid(row_idx as _) {
                    row_idx -= 1;
                    if row_idx < 0 {
                        return None;
                    }
                }

                assert!(index.is_valid(row_idx as usize));
                (*name, index.values()[row_idx as usize]).into()
            })

Should be able to make that O(log(n))-ish using a B-tree or something.

@teh-cmc teh-cmc added 🏹 arrow Apache Arrow ⛃ re_datastore affects the datastore itself labels Dec 2, 2022
@teh-cmc
Copy link
Member Author

teh-cmc commented Dec 12, 2022

And now it does it at the bucket level too! :)

@teh-cmc
Copy link
Member Author

teh-cmc commented Jul 9, 2024

Irrelevant now.

@teh-cmc teh-cmc closed this as not planned Won't fix, can't repro, duplicate, stale Jul 9, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
🏹 arrow Apache Arrow ⛃ re_datastore affects the datastore itself
Projects
None yet
Development

No branches or pull requests

1 participant