Skip to content
This repository has been archived by the owner on Nov 21, 2024. It is now read-only.

Parsing NGINX log files is needlessly slow. #64

Open
Alextopher opened this issue Mar 6, 2024 · 0 comments
Open

Parsing NGINX log files is needlessly slow. #64

Alextopher opened this issue Mar 6, 2024 · 0 comments

Comments

@Alextopher
Copy link
Member

Alextopher commented Mar 6, 2024

Regular expressions considered harmful.

https://github.com/COSI-Lab/Mirror/blob/046b562660d58b747073618f947ff0b8abb67ca0/nginx.go#L119-L120

Splitting on the " characters is significantly more efficient, somewhere between 10x-100x so. This kind of performance improvement wouldn't be meaningless either. Somewhere between 500 MB and 1500 MB of data is passing through this parser daily.

This would make a good first issue 😉. You can easily compare the output of the previous version to a new version on gigabytes of data.

# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant