Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Rust graph #166

Merged
merged 10 commits into from
Feb 7, 2025
Merged

Rust graph #166

merged 10 commits into from
Feb 7, 2025

Conversation

seddonym
Copy link
Owner

@seddonym seddonym commented Nov 1, 2024

Implements the graph in Rust. 🥳

It took a while to get something that performs better - thanks @Peter554 for figuring out the final implementation.

Copy link

codspeed-hq bot commented Dec 9, 2024

CodSpeed Performance Report

Merging #166 will degrade performances by 96.65%

Comparing rust-graph (ec7e7f9) with master (06401f4)

Summary

⚡ 11 improvements
❌ 1 (👁 1) regressions
✅ 2 untouched benchmarks
🆕 3 new benchmarks
⁉️ 2 (👁 2) dropped benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
test_deep_layers_large_graph_kept 1,473.9 ms 19.8 ms ×75
test_deep_layers_large_graph_violated 1,522.4 ms 11.9 ms ×130
test_top_level_large_graph_kept 766.3 ms 50.2 ms ×15
test_top_level_large_graph_violated 783.6 ms 234.8 ms ×3.3
👁 test_no_chain 39.7 µs 1,184.1 µs -96.65%
test_chains_found 127,985.7 µs 84.8 µs ×1,500
test_no_chains 128.7 ms 1.2 ms ×110
test_build_django_from_cache 225.1 ms 135.5 ms +66.1%
test_copy_graph 80.4 ms 43.1 ms +86.64%
test_find_descendants 72.6 ms 41.4 ms +75.21%
test_find_downstream_modules 2,487,587.1 µs 186.8 µs ×13,000
test_find_upstream_modules 1,828.3 ms 3.9 ms ×460
🆕 test_get_import_details N/A 249.7 µs N/A
👁 test_graph_contains_module 198.7 µs N/A N/A
👁 test_iterate_over_modules_in_graph 2.9 ms N/A N/A
🆕 test_modules_property_first_access N/A 34.3 ms N/A
🆕 test_modules_property_many_accesses N/A 34.7 ms N/A

@seddonym seddonym mentioned this pull request Feb 7, 2025
seddonym and others added 9 commits February 7, 2025 15:10
This is the main graph implemented in Rust - but it isn't plumbed into
Python yet.
This adds a wrapper around the Rust-graph in lib.rs which handles the
interface between Python and Rust. At the same time, it changes the
Python ImportGraph so it uses Rust instead.
This is faster, but not cryptographically safe. That shouldn't matter for our
purposes.
pytest-benchmark will actually determine a suitable number
of rounds+iterations automatically (typically much more than 3!).
With only 3 rounds the results are quite noisy.
Accessing graph.modules is more expensive than it looks.
@seddonym seddonym marked this pull request as ready for review February 7, 2025 15:35
@seddonym seddonym merged commit 50d04f4 into master Feb 7, 2025
17 checks passed
@seddonym seddonym deleted the rust-graph branch February 7, 2025 15:35
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants