Parallelization for SWIFTGalaxies

[kyleaoman]

There's an obvious use-case for tools to iterate over SWIFTGalaxy objects in parallel. This was omitted from the initial SWIFTGalaxies PR for simplicity but implementing some parallelization tools on top of SWIFTGalaxies should be high on the to-do list. Some initial thoughts carried over from the SWIFTGalaxies development process:

• SWIFTGalaxies.split(n) to return a list of n SWIFTGalaxies instances each with a fraction approx 1/n of the ensemble of target regions (where each region could contain many target galaxies). This can easily be farmed out to multiple processes (maybe a SWIFTGalaxies.sub(n, N) approach makes more sense so that each process gets a copy-on-write instance and then picks out its own share?). Some cookbook examples useful here.
• Could combine map with split (or sub) to offer some parallelization built in.
• Consider analogous tools to other tools from the multiprocessing in addition to map. Things like some kind of queue, or asynchronous access to results as they arrive.
• Beware nested parallelism.