[manifold] make partitioner faster.

[jhgg]

Partitioner Optimizations

• instead of Utils.group_by, we are using a new function called Utils.partition_pids which uses a tuple instead of a map internally. This ends up being roughly 2.2x faster than Utils.group_by in the benchmarks attached.

## GroupByBench
benchmark name     iterations   average time
partition_pids 8         5000   340.75 µs/op
partition_pids 24        5000   382.38 µs/op
partition_pids 48        5000   432.45 µs/op
group by 8               5000   600.24 µs/op
group by 24              2000   730.41 µs/op
group by 48              1000   1154.78 µs/op

• make state not need to be modified during iteration. (pre-spawn the workers) in the Partitioner. This means we don't have to use Enum.reduce and can instead use a more optimized code path for this, do_send, which operates on a tuple of lists, and sends to the respective worker. Turns out this isn't really any faster...

## WorkerSendBenches
benchmark name    iterations   average time
enum reduce send       50000   65.56 µs/op
do_send send           50000   68.85 µs/op

• add a specific case for sending to a single pid (which we do quite often, when doing manifold sends to a single pid). There is no practical speedup in avoiding the group by operation, but the resultant send operation ends up being faster, as it doesn't have to start an Enum.reducer, or iterate over the result of Utils.partition_pids, so that avoids ~0.15µs/op + ~0.70-1.5 µs/op. additionally, this function consumes less reductions as it is doing less things (which we want!)

## GroupByOneBench
benchmark name     iterations   average time
partition_pids 8     10000000   0.11 µs/op
partition_pids 24    10000000   0.14 µs/op
group by 48          10000000   0.14 µs/op
group by 8           10000000   0.14 µs/op
group by 24          10000000   0.16 µs/op
partition_pids 48    10000000   0.20 µs/op

## WorkerSendOneBenches (worker count=48)
benchmark name    iterations   average time
enum reduce send    10000000   0.70 µs/op
do_send send         1000000   1.47 µs/op

worker gained some optimizations too:

• use list comprehension instead of Enum.each (it's a little bit faster), here's sending to 200 pids:

## SendBench
benchmark name     iterations   average time
send list comp           2000   961.87 µs/op
send fast reducer        2000   982.81 µs/op
send enum each           1000   1153.00 µs/op

• add a special case for sending to a single pid:

## SendBenchOne
benchmark name     iterations   average time
send one             10000000   0.22 µs/op
send fast reducer    10000000   0.27 µs/op
send enum each       10000000   0.34 µs/op
send list comp       10000000   0.34 µs/op

Manifold.send

• make a specialized code path for sending to a single pid.
• use list comprehension instead of Enum.each

[vishnevskiy]

It is probably a fine change. But this no longer supports respawning a worker if it somehow crashes. If you want to make this change you should remove :trap_exit and the :EXIT match.

[jhgg]

fixed.

[jhgg]

this is clearly faster, i'm wanting to merge this then work on sharding partitions in #4. wdyt?

[vishnevskiy]

Only change that seems worth it is the group by optimization.