Things Todo

Major items

These items are fairly fundamental issues with the code that will need to be addressed.

• Distributed Memory - At the moment we are limited to analysis where we can fit the KL-mode calculation on a single node. This means we need to be able to fit roughly four covariance matrices within memory at any one time. If the number of degrees of freedom at each m grows too much, then we run out of memory.

  The solution is to allow the computations to be distributed using Scalapack. We've started on this route by creating a Python wrapper around it: PyScalapack.

• Reduced Memory Footprint - A potential intermediate is to reduce the memory footprint. Currently we store roughly the whole of four covariance matrices. One foreground, one signal, one for the eigenvectors and one for the workspace (using divide and conquer). By generating only half of each of the S and F matrices, and using the MTRR routine (small workspace), we can probably cut this in two.

Both of these can be tackled reasonably independently of the working code base and then integrated back in when finished, so I think they're fair game for everyone to have a go at.

Pipeline

Eventually all the analysis code needs to be turned into a realistic pipeline. At the moment there are many things blocking this, but both for CHIME and GMRT we will need to be sorting these in the enar future.

• SVD Map Making - After changing to the new SVD code I think the map making code is broken. The pseudo-inverse of the new beam matrices is the obvious way to go, however this doesn't seem to have the nice Bayesian interpretation that the pre-SVD approach had.

• Wiener Filtering - The maps made by the map maker tend to be full of instrumental noise, unless some kind of filtering has been applied. Obviously a noise only KL would work, but this is expensive. I think a simple Wiener filter before map making is the obvious option.

• Real Power Spectrum Estimation - At the moment the power spectrum code can only predict Fisher matrices. This isn't the end of the world as to generate this we essentially have to estimate the power spectrum of simulated data, but there's quite a fair bit of work required to hook things up.

• 2D Power Spectrum Estimation - Currently we can only deal with estimating the radially averaged powerspectrum. In theory it shouldn't be too hard to extend this a 2D estimation, by generating an extended range of basis functions.

These are all priorities for the GMRT analysis and most of them aren't too hard to do but they will require a little work to integrate with the code base. For the moment, these should probably be done by Richard.

Things to test

There's a great many things we need to test:

The first class are essentially all corruptions of the pure time stream. Most can be simulated by directly applying to a mock timestream.

• How well do we need to know our beams?
  • Randomized beam shapes
• How well do we need to calibrate?
  • Gain errors
• Non-sidereal signals
  • The Sun
  • RFI
• Ionosphere
  • Scintillation
  • Birefringence (rotation and splitting)

The beam errors are a little more complicated but can be generated by taking random linear combinations of basis beam shapes.

The second class are a little more interesting, however in many cases I'm not really clear what Kris is talking about :)

• Solve the incompletness data problem.
• Nonlinear Power Spectrum (Zahn)

Sky Simulation

Simulation of polarised skies will be needed for detailed testing and forecasting of the analysis code. At present the only simulations are unpolarised, but reasonably realistic.

• Polarised Point Sources - Draw a random polarisation for each source, and then rotate with a Faraday screen (currently thinking of using the Opperman map).

• Polarised Galactic Synchrotron - This is much trickier. Currently I think this can be done by simulating the polarised component as a function of Faraday depth. The width of this distribution should be around that of the Opperman map.