Implementation of refactored rates in plasma

[andrewfullard]

📝 Description

Type: 🚀 feature

Initial implementation of the new rates solver, level pop solver, and atomic data parsing into the plasma.

🚦 Testing

How did you test these changes?

• Testing pipeline
• Other method (describe)
• My changes can't be tested (explain why)

☑️ Checklist

• I requested two reviewers for this pull request
• I updated the documentation according to my changes
• I built the documentation by applying the build_docs label

Note: If you are not allowed to perform any of these actions, ping (@) a contributor.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[tardis-bot]

*beep* *bop*
Hi human,
I ran ruff on the latest commit (7216bae).
Here are the outputs produced.
Results can also be downloaded as artifacts here.
Summarised output:

5	G004  	[ ] Logging statement uses f-string
2	W605  	[*] Invalid escape sequence: `\A`
1	RET505	[ ] Unnecessary `elif` after `return` statement

Complete output(might be large):

docs/workflows/simple_workflow_equilibrium.ipynb:cell 8:10:26: W605 [*] Invalid escape sequence: `\A`
docs/workflows/simple_workflow_equilibrium.ipynb:cell 8:11:40: W605 [*] Invalid escape sequence: `\A`
tardis/plasma/properties/partition_function.py:154:9: RET505 Unnecessary `elif` after `return` statement
tardis/plasma/properties/partition_function.py:190:25: G004 Logging statement uses f-string
tardis/workflows/simple_tardis_workflow.py:230:17: G004 Logging statement uses f-string
tardis/workflows/simple_tardis_workflow.py:450:17: G004 Logging statement uses f-string
tardis/workflows/standard_tardis_workflow.py:172:13: G004 Logging statement uses f-string
tardis/workflows/standard_tardis_workflow.py:212:17: G004 Logging statement uses f-string
Found 8 errors.
[*] 2 fixable with the `--fix` option.

[andrewfullard]

This requires updated default atomic data to function correctly, using the full set of Chianti collisional strength values (and not the Carsus pre-calculated values).

[tardis-bot]

*beep* *bop*

Hi, human.

The docs workflow has succeeded ✔️

Click here to see your results.

[tardis-bot]

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (cc0b125) and the latest commit (524968d).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

Significantly changed benchmarks:

| Change   | Before [5e2d0bb3] <master>   | After [524968d7]    |   Ratio | Benchmark (Parameter)                                                                             |
|----------|------------------------------|---------------------|---------|---------------------------------------------------------------------------------------------------|
| +        | 4.09±0.01ms                  | 6.27±0.01ms         |    1.53 | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('macroatom') |

All benchmarks:

Benchmarks that have stayed the same:

| Change   | Before [5e2d0bb3] <master>   | After [524968d7]    | Ratio   | Benchmark (Parameter)                                                                                                               |
|----------|------------------------------|---------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------|
|          | 1.31±0.3μs                   | 1.89±0.4μs          | ~1.44   | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_line                    |
|          | 591±200ns                    | 782±300ns           | ~1.32   | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation                                      |
|          | 3.08±0.01ms                  | 3.56±0.03ms         | ~1.16   | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('scatter')                                     |
|          | 2.02±1μs                     | 2.34±1μs            | ~1.16   | transport_montecarlo_estimators_radfield_estimator_calcs.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators       |
|          | 3.05±0.5μs                   | 3.52±0.2μs          | ~1.15   | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_within_shell                              |
|          | 2.62±0.5ms                   | 2.86±0.4ms          | 1.09    | transport_montecarlo_single_packet_loop.BenchmarkTransportMontecarloSinglePacketLoop.time_single_packet_loop                        |
|          | 43.8±20μs                    | 46.8±20μs           | 1.07    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_emission                                         |
|          | 45.2±20μs                    | 46.3±20μs           | 1.02    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_scatter                                          |
|          | 63.4±0.1ms                   | 64.5±0.2ms          | 1.02    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_rpacket_trackers_to_dataframe                  |
|          | 38.9±0.3s                    | 39.3±0.1s           | 1.01    | run_tardis.BenchmarkRunTardis.time_run_tardis                                                                                       |
|          | 3.19±0.5μs                   | 3.23±0.7μs          | 1.01    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_bad_vpacket                                       |
|          | 592±200ns                    | 591±500ns           | 1.00    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation                                |
|          | 2.08±0m                      | 2.08±0m             | 1.00    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_FormalIntegrator_functions                                 |
|          | 203±0.2ns                    | 203±0.03ns          | 1.00    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_intensity_black_body                                       |
|          | 1.69±0.2ms                   | 1.69±0.01ms         | 1.00    | transport_montecarlo_main_loop.BenchmarkTransportMontecarloMontecarloMainLoop.time_montecarlo_main_loop                             |
|          | 6.32±0.8μs                   | 6.35±1μs            | 1.00    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket                                           |
|          | 1.05±0.01m                   | 1.04±0m             | 0.99    | run_tardis.BenchmarkRunTardis.time_run_tardis_rpacket_tracking                                                                      |
|          | 735±0.8ns                    | 726±0.08ns          | 0.99    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_thomson_scatter                                       |
|          | 39.4±0.02μs                  | 38.5±0.02μs         | 0.98    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_tracker_list                  |
|          | 601±100ns                    | 581±200ns           | 0.97    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation                              |
|          | 21.2±5μs                     | 20.6±5μs            | 0.97    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_last_interaction_tracker_list |
|          | 1.27±0μs                     | 1.22±0μs            | 0.96    | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_boundary                |
|          | 7.97±2μs                     | 7.38±2μs            | 0.93    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_volley                                    |

Benchmarks that have got worse:

| Change   | Before [5e2d0bb3] <master>   | After [524968d7]    |   Ratio | Benchmark (Parameter)                                                                             |
|----------|------------------------------|---------------------|---------|---------------------------------------------------------------------------------------------------|
| +        | 4.09±0.01ms                  | 6.27±0.01ms         |    1.53 | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('macroatom') |

If you want to see the graph of the results, you can check it here

[andrewfullard]

We need to change the new atomic data (version 2.0) to have .format_latest appended before the file type. This should be propagated to all sources of the data and all readers of the data. In the future an automated append of .format_latest would be nice. Then old versions would be renamed to .format_1.0 before the file type.

[andrewfullard]

Double check where interpolation is happening

[andrewfullard]

Double check where interpolation is happening

Interpolation is only happening for Chianti data in the UpsilonChiantiSolver class. The old NLTEData class remains for testing purposes.

[andrewfullard]

TODO:

• Update regression data to use the raw Chianti values
• Decide on a name for the new/old atomic data
• Determine which data need updating
  • Standard data
  • Helium data
  • Others?
• Update paths to new data
• Fix any broken tests