Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add Xor Filters #29

Closed
Callidon opened this issue May 9, 2021 · 3 comments
Closed

Add Xor Filters #29

Callidon opened this issue May 9, 2021 · 3 comments

Comments

@Callidon
Copy link
Owner

Callidon commented May 9, 2021

As pointed in #8 we can add Xor Filters to the library

See https://lemire.me/blog/2019/12/19/xor-filters-faster-and-smaller-than-bloom-filters/ for more details

@folkvir
Copy link
Collaborator

folkvir commented May 17, 2021

For copyright, there exists one C-header only impl: https://github.com/FastFilter/xor_singleheader
A pure-python impl as well as a binding could be possible.

@folkvir
Copy link
Collaborator

folkvir commented Dec 17, 2021

Just for references, the official accepted paper :

folkvir added a commit that referenced this issue Dec 30, 2021
folkvir added a commit that referenced this issue Dec 30, 2021
@folkvir folkvir mentioned this issue Jan 3, 2022
5 tasks
@folkvir
Copy link
Collaborator

folkvir commented Feb 13, 2022

integrated in develop, will be pushed in v2.0.0

@folkvir folkvir closed this as completed Feb 13, 2022
@folkvir folkvir mentioned this issue Feb 13, 2022
Callidon pushed a commit that referenced this issue Feb 18, 2022
Author: Arnausd Grall (@folkvir)

* update deps and add gts

* h64 only + fix #43 + gts

* mocha: parallel execution

* utils: rename *Indices to *Indexes, add better documentation

* tests: show difference when using getDistinctIndexes w/wo hashing at each iteration

* utils: rename *Indices to *Indexes

* update ref pdf of CMS

* double hashing: use the enhanced technique

* getDistinctIndexes: implement #43 suggestions + allow to switch to XXH 32/64 if needed

* update changelog

* add .DS_Store in the list of ignored files

* use typescript 4.X.X

* fix typo of #46

* fix error when setting 32/64 bits xxh functions, and allow for overriding the serialize function (#48)

* add code coverage

* lint

* update eslint ignore for faster parsing

* add branch update_outdated to the tested branches

* fix wrong github workflow branch name

* workflow: use 16.x and remove 10.x

* update README and remove useless deps

* tests: fix error with describe/it rules overriding data

* implemnting xor filter for the next release

* update all files to the new project standard, set v2.0.0

* xor filters are working

* testing the xor filter

* xor filter #29

* #29

* documentation

* code ql / tests on develop

* keywords

* fix workflow

* move all hash related functions to the BaseFilter class

* modify tests according to: 12417e7

* parallel mocha tests

* use lts/* with setup-node@v2

* update README.md

* add badge to the readme

* export BaseFilter in the entry

* badge refers to the last action build

* update compatibility readme

* update @types/node version to be the latest minor release

* prettier auto line endings

* update @types/node version to be the latest minor release

* #49 add compatibility with 1.3.4

* #49 compatibility for BloomFilters import only is working

* Use AutoExportable feature for simplfying the export process, bitsets size set to a multiple of bitwords

* create Hashing classes

* remove nyc, use eslint + prettier instead of gts, enforce type checking, make all propeties public, no readonly

* update yarn.lock

* ci: remove node 15

* fix missing dependency and fix eslint errors

* update readme

* fix conflicts with master, eslint is now a rule

* add typedoc-plugin-missing-exports and fix non-overrided doubleHashing function
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

No branches or pull requests

2 participants