Speed up the Python filtering using Rust

[liuchengxu]

Original info from output of pytest:

----------------------------------------------------------------------------------------------- benchmark: 8 tests ----------------------------------------------------------------------------------------------
Name (time in ms)                   Min                    Max                   Mean              StdDev                 Median                   IQR            Outliers      OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_rust_10000                 11.7424 (1.0)          35.8978 (1.0)          14.2472 (1.0)        5.0495 (2.42)         12.1977 (1.0)          2.3190 (1.0)           5;9  70.1893 (1.0)          82           1
test_rust_100000               184.3735 (15.70)       220.7831 (6.15)        212.0999 (14.89)     15.5814 (7.46)        218.9250 (17.95)       11.6435 (5.02)          1;1   4.7148 (0.07)          5           1
test_pure_python_10000         335.1105 (28.54)       340.1328 (9.48)        337.8211 (23.71)      2.0885 (1.0)         338.7337 (27.77)        3.3208 (1.43)          2;0   2.9601 (0.04)          5           1
test_rust_500000             1,506.0994 (128.26)    1,566.3961 (43.63)     1,529.0764 (107.32)    24.7463 (11.85)     1,530.7926 (125.50)      36.2058 (15.61)         1;0   0.6540 (0.01)          5           1
test_rust_800000             2,750.9194 (234.27)    2,977.6952 (82.95)     2,829.1714 (198.58)    88.7958 (42.52)     2,812.0944 (230.54)      99.7731 (43.02)         1;0   0.3535 (0.01)          5           1
test_pure_python_100000      5,294.6214 (450.90)    5,356.2496 (149.21)    5,316.4504 (373.16)    23.6550 (11.33)     5,309.3389 (435.27)      24.1679 (10.42)         1;0   0.1881 (0.00)          5           1
test_pure_python_500000     38,645.1174 (>1000.0)  39,222.1991 (>1000.0)  39,011.7326 (>1000.0)  249.1471 (119.29)   39,114.7954 (>1000.0)    398.7475 (171.95)        1;0   0.0256 (0.00)          5           1
test_pure_python_800000     71,624.3134 (>1000.0)  73,436.3243 (>1000.0)  72,491.7539 (>1000.0)  781.9669 (374.41)   72,649.2563 (>1000.0)  1,370.8810 (591.14)        2;0   0.0138 (0.00)          5           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------- benchmark: 8 tests ----------------------------------------------------------------------------------------------

Name (time in ms)                   Min                    Max                   Mean              StdDev                 Median                   IQR            Outliers      OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_rust_10000                 11.7424 (1.0)          35.8978 (1.0)          14.2472 (1.0)        5.0495 (2.42)         12.1977 (1.0)          2.3190 (1.0)           5;9  70.1893 (1.0)          82           1
test_pure_python_10000         335.1105 (28.54)       340.1328 (9.48)        337.8211 (23.71)      2.0885 (1.0)         338.7337 (27.77)        3.3208 (1.43)          2;0   2.9601 (0.04)          5           1

test_rust_100000               184.3735 (15.70)       220.7831 (6.15)        212.0999 (14.89)     15.5814 (7.46)        218.9250 (17.95)       11.6435 (5.02)          1;1   4.7148 (0.07)          5           1
test_pure_python_100000      5,294.6214 (450.90)    5,356.2496 (149.21)    5,316.4504 (373.16)    23.6550 (11.33)     5,309.3389 (435.27)      24.1679 (10.42)         1;0   0.1881 (0.00)          5           1

test_rust_500000             1,506.0994 (128.26)    1,566.3961 (43.63)     1,529.0764 (107.32)    24.7463 (11.85)     1,530.7926 (125.50)      36.2058 (15.61)         1;0   0.6540 (0.01)          5           1
test_pure_python_500000     38,645.1174 (>1000.0)  39,222.1991 (>1000.0)  39,011.7326 (>1000.0)  249.1471 (119.29)   39,114.7954 (>1000.0)    398.7475 (171.95)        1;0   0.0256 (0.00)          5           1

test_rust_800000             2,750.9194 (234.27)    2,977.6952 (82.95)     2,829.1714 (198.58)    88.7958 (42.52)     2,812.0944 (230.54)      99.7731 (43.02)         1;0   0.3535 (0.01)          5           1
test_pure_python_800000     71,624.3134 (>1000.0)  73,436.3243 (>1000.0)  72,491.7539 (>1000.0)  781.9669 (374.41)   72,649.2563 (>1000.0)  1,370.8810 (591.14)        2;0   0.0138 (0.00)          5           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The filtering function written in Rust is 20x faster the Python one.

[issue-label-bot]

Issue-Label Bot is automatically applying the label feature_request to this issue, with a confidence of 0.94. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.