Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Refine geodata decoder #322

Merged
merged 3 commits into from
May 6, 2021
Merged

Refine geodata decoder #322

merged 3 commits into from
May 6, 2021

Conversation

Loyalsoldier
Copy link
Collaborator

@Loyalsoldier Loyalsoldier commented May 6, 2021

The new geodata decoder is extremely memory efficient and the benchmark result is excellent.

The benchmark result:

goos: unknown
goarch: amd64
pkg: github.com/p4gefau1t/trojan-go/common/geodata
cpu: Intel(R) Core(TM) i5
BenchmarkLoadGeoIP-8     	1000000000	        0.005175 ns/op	       936.5 KiB(GeoIP-Alloc)	         1.326 MiB(GeoIP-TotalAlloc)	       0 B/op	       0 allocs/op
BenchmarkLoadGeoSite-8   	1000000000	         0.01573 ns/op	     1.961 MiB(GeoSite-Alloc)	       2.762 MiB(GeoSite-TotalAlloc)	       0 B/op	       0 allocs/op
PASS
ok  	github.com/p4gefau1t/trojan-go/common/geodata	0.597s

The followings are the real statistics(memory usage) come from router component with log-level set to debug(0). It loads geoip:cn, geoip:private, geosite:cn, geosite:private and geosite:geolocation-!cn in both geodata decoders.

Memory usage of the new geodata decoder(read geodata files byte by byte):

[DEBUG] 2021/05/06 00:00:00 github.com/p4gefau1t/trojan-go/tunnel/router.NewClient:client.go:453 GeoIP rules -> Alloc: 468.41 KiB; TotalAlloc: 1.45 MiB
[DEBUG] 2021/05/06 00:00:00 github.com/p4gefau1t/trojan-go/tunnel/router.NewClient:client.go:454 GeoSite rules -> Alloc: 2.12 MiB; TotalAlloc: 2.94 MiB
[DEBUG] 2021/05/06 00:00:00 github.com/p4gefau1t/trojan-go/tunnel/router.NewClient:client.go:455 Plaintext rules -> Alloc: 648 B; TotalAlloc: 648 B
[DEBUG] 2021/05/06 00:00:00 github.com/p4gefau1t/trojan-go/tunnel/router.NewClient:client.go:456 Total(router) -> Alloc: 2.58 MiB; TotalAlloc: 4.39 MiB

Memory usage of the original(old) geodata decoder(read geodata files at once entirely):

[DEBUG] 2021/05/06 00:00:00 github.com/p4gefau1t/trojan-go/tunnel/router.NewClient:client.go:514 GeoIP rules -> Alloc: 47.66 MiB; TotalAlloc: 55.34 MiB
[DEBUG] 2021/05/06 00:00:00 github.com/p4gefau1t/trojan-go/tunnel/router.NewClient:client.go:515 GeoSite rules -> Alloc: 8.32 MiB; TotalAlloc: 8.32 MiB
[DEBUG] 2021/05/06 00:00:00 github.com/p4gefau1t/trojan-go/tunnel/router.NewClient:client.go:516 Plaintext rules -> Alloc: 648 B; TotalAlloc: 648 B
[DEBUG] 2021/05/06 00:00:00 github.com/p4gefau1t/trojan-go/tunnel/router.NewClient:client.go:517 Total(router) -> Alloc: 55.98 MiB; TotalAlloc: 63.67 MiB

@Loyalsoldier Loyalsoldier changed the title Refine geodate decoder Refine geodata decoder May 6, 2021
@Loyalsoldier Loyalsoldier merged commit a9ce6d4 into p4gefau1t:master May 6, 2021
@Loyalsoldier Loyalsoldier deleted the refine-geodate-decoder branch May 6, 2021 02:49
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant