s2: Do 2 overlapping match checks #839

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Merged

klauspost merged 1 commit into master from matchlen-do-16

Jul 27, 2023

Owner

klauspost commented Jul 27, 2023 •

edited

Loading

Mainly faster on long matches, but has little to no regression on short matches.

λ s2c -cpu=1 --bench=5 \compress\github-june-2days-2019.json

BEFORE: Compressing... 6273951764 -> 944876986 [15.06%]; 9.43s, 634.5MB/s
AFTER:  Compressing... 6273951764 -> 944876986 [15.06%]; 9.269s, 645.5MB/s

λ s2c -cpu=1 --bench=5 \compress\nyc-taxi-data-10M.csv
BEFORE: Compressing... 3325605752 -> 884713054 [26.60%]; 8.296s, 382.3MB/s
AFTER:  Compressing... 3325605752 -> 884713054 [26.60%]; 8.173s, 388.1MB/s

λ s2c -cpu=1 --bench=5 \compress\enwik9
BEFORE: Compressing... 1000000000 -> 416582393 [41.66%]; 3.706s, 257.3MB/s
AFTER:  Compressing... 1000000000 -> 416582393 [41.66%]; 3.636s, 262.3MB/s


          s2: Do 2 overlapping match checks

b8a7752

Mainly faster on long matches, but has little to no regression on short matches.

klauspost merged commit c1dcc38 into master

klauspost deleted the matchlen-do-16 branch

July 27, 2023 13:26

kodiakhq bot referenced this pull request in cloudquery/filetypes


          fix(deps): Update module github.com/klauspost/compress to v1.17.0 (#319)

e02ab7a

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/klauspost/compress](https://github.com/klauspost/compress) | indirect | minor | `v1.16.7` -> `v1.17.0` |

---

### Release Notes

<details>
<summary>klauspost/compress (github.com/klauspost/compress)</summary>

### [`v1.17.0`](https://github.com/klauspost/compress/releases/tag/v1.17.0)

[Compare Source](https://github.com/klauspost/compress/compare/v1.16.7...v1.17.0)

#### What's Changed

-   Add dictionary builder by [@&#8203;klauspost](https://github.com/klauspost) in [https://github.com/klauspost/compress/pull/853](https://github.com/klauspost/compress/pull/853)
-   Add xerial snappy read/writer by [@&#8203;klauspost](https://github.com/klauspost) in [https://github.com/klauspost/compress/pull/838](https://github.com/klauspost/compress/pull/838)
-   flate: Add limited window compression by [@&#8203;klauspost](https://github.com/klauspost) in [https://github.com/klauspost/compress/pull/843](https://github.com/klauspost/compress/pull/843)
-   s2: Do 2 overlapping match checks by [@&#8203;klauspost](https://github.com/klauspost) in [https://github.com/klauspost/compress/pull/839](https://github.com/klauspost/compress/pull/839)
-   flate: Add amd64 assembly matchlen by [@&#8203;klauspost](https://github.com/klauspost) in [https://github.com/klauspost/compress/pull/837](https://github.com/klauspost/compress/pull/837)
-   gzip: Copy bufio.Reader on Reset by [@&#8203;thatguystone](https://github.com/thatguystone) in [https://github.com/klauspost/compress/pull/860](https://github.com/klauspost/compress/pull/860)
-   zstd: Remove offset from bitReader by [@&#8203;greatroar](https://github.com/greatroar) in [https://github.com/klauspost/compress/pull/854](https://github.com/klauspost/compress/pull/854)
-   fse, huff0, zstd: Remove always-nil error returns by [@&#8203;greatroar](https://github.com/greatroar) in [https://github.com/klauspost/compress/pull/857](https://github.com/klauspost/compress/pull/857)
-   tests: unnecessary use of fmt.Sprintf by [@&#8203;testwill](https://github.com/testwill) in [https://github.com/klauspost/compress/pull/836](https://github.com/klauspost/compress/pull/836)
-   tests: Fix OSS fuzzer t.Run by [@&#8203;klauspost](https://github.com/klauspost) in [https://github.com/klauspost/compress/pull/852](https://github.com/klauspost/compress/pull/852)
-   tests: Use Go 1.21.x by [@&#8203;klauspost](https://github.com/klauspost) in [https://github.com/klauspost/compress/pull/851](https://github.com/klauspost/compress/pull/851)

#### New Contributors

-   [@&#8203;testwill](https://github.com/testwill) made their first contribution in [https://github.com/klauspost/compress/pull/836](https://github.com/klauspost/compress/pull/836)
-   [@&#8203;thatguystone](https://github.com/thatguystone) made their first contribution in [https://github.com/klauspost/compress/pull/860](https://github.com/klauspost/compress/pull/860)

**Full Changelog**: klauspost/compress@v1.16.7...v1.17.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on the first day of the month" (UTC), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi4xMDkuNCIsInVwZGF0ZWRJblZlciI6IjM2LjEwOS40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

# for free to join this conversation on GitHub. Already have an account? # to comment

Labels

None yet