Skip to content

Future Testing Concept

Jump to bottom
richardgilham edited this page Jan 15, 2024 · 4 revisions

Future testing of Simulation Models

The impending transition of our simulations to GitHub and easier access to cloud platforms presents challenges and opportunities for transforming the approach to regression testing, trunk progression and release cycle management. Here we develop a vision, requirements and potential solutions around this topic with the ultimate aim of using it as evidence towards commissioning a project.

The sub-headings below capture top level goals, requirements and possible solutions. Sub-pages contain more detail.

Top-level Goals

This is our vision of what we want to achieve by 202N

  • Federate responsibility for advancing the trunks comprising the simulation model components of Momentum to the Momentum Partnership
  • Provide equitable regression testing for any developer worldwide
  • Leverage GitHub functionality to maximise the opportunities for Continuous Integration
  • Leverage Cloud platforms to maximise portability through access to diverse hardware and software
  • Minimise the barriers to using cloud and on-premise platforms for regression and compute performance testing

Historically, management of the Unified Model (UM) release cycle was Met Office-centric. This proved to by highly successful and provided strong assurance of the UM for its operational weather forecasting role. However, the UM ecosystem was not well-posed to evolve with trends in technology and software engineering. Its successor, based around the LFRic infrastructure, attempted to catch up with these trends but was limited in its ability to achieve this. With the planned migration of the codebase from the fac/svn-based Met Office Shared Repository Service (MOSRS) to GitHub, and changes to the effective access to cloud platforms, catching up is now a more tractable proposition.

Top-level Requirements

  • Overall comparable level of protection as afforded by the existing UM approach
  • Perform compute-intensive regression testing using a cloud platform
  • Perform compute-intensive regression testing using on-premise platforms
  • Continuous Integration testing for low-expense regression testing
  • Capability to synthesise testing results from all platforms for decision-making purposes

Top-level Potential Solutions

  • Repository mirrors
  • Self-hosted GitHub runners
  • GitLab CI
Clone this wiki locally