From d8eec05d72366b814289e9c950969421f77af10b Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 12:18:57 +0100 Subject: [PATCH 01/29] :truck: Refactored capabilities --- docs/images/Capability_Map/full.drawio.svg | 2 +- docs/source/pillars/computing_technology.md | 265 ++++++++++++-------- 2 files changed, 165 insertions(+), 102 deletions(-) diff --git a/docs/images/Capability_Map/full.drawio.svg b/docs/images/Capability_Map/full.drawio.svg index dca739f3..fd107c52 100644 --- a/docs/images/Capability_Map/full.drawio.svg +++ b/docs/images/Capability_Map/full.drawio.svg @@ -1,4 +1,4 @@ -
2. Computing Technology
2. Computing Technology
1. Information Governance
1. Information Governance
3. Data Management
3. Data Management

Member Accreditation
 Member Accreditation
Risk Management
Risk Management

Infrastructure Lifecycle Management 
Infrastructure Lifecycle Manage...
Network Management
Network Management
End User Computing
End User Computing
Output Management
Output Management
Identity and Access Management
Identity and Access Management
Data Lifecycle Management
Data Lifecycle Management

Information Security
 Information Security
Governance requirements
Governance requirements

Quality management
 Quality management
Infrastructure Analytics
Infrastructure Analytics
Availability Management
Availability Management
Information search and discovery
Information search and discovery
4. Supporting Capabilities
4. Supporting Capabilities
Project and Programme Management
Project and Programme Management
Business Continuity Management
Business Continuity Management
Financial Management
Financial Management
Relationship Management
Relationship Management
Legal Services
Legal Services
Procurement
Procurement
IT Service Management
IT Service Management
Knowledge Management
Knowledge Management
Security Levels and Tiering
Security Levels and Tiering
Study Management
Study Management
Public involvement and engagement
Public involvement and engageme...Text is not SVG - cannot display \ No newline at end of file +
2. Computing Technology
2. Computing Technology
1. Information Governance
1. Information Governance
3. Data Management
3. Data Management

Member Accreditation
 Member Accreditation
Risk Management
Risk Management

Infrastructure Management 
 Infrastructure Management 
Capacity Management
Capacity Management
End User Computing
End User Computing
Output Management
Output Management
Identity and Access Management
Identity and Access Management
Data Lifecycle Management
Data Lifecycle Management

Information Security
 Information Security
Governance requirements
Governance requirements

Quality management
 Quality management
Information search and discovery
Information search and discovery
4. Supporting Capabilities
4. Supporting Capabilities
Project and Programme Management
Project and Programme Management
Business Continuity Management
Business Continuity Management
Financial Management
Financial Management
Relationship Management
Relationship Management
Legal Services
Legal Services
Procurement
Procurement
IT Service Management
IT Service Management
Knowledge Management
Knowledge Management
Security Levels and Tiering
Security Levels and Tiering
Study Management
Study Management
Public involvement and engagement
Public involvement and engageme...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 57f8569a..055522f5 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -2,7 +2,6 @@ # Computing technology -This capability concerns what the TRE operator does to manage systems for storing, retrieving, analysing and sending information. ```{figure} ../../images/Capability_Map/full.drawio.svg :alt: SATRE Pillars Capability Map @@ -11,38 +10,40 @@ This capability concerns what the TRE operator does to manage systems for storin SATRE Pillars Capability Map ``` +This pillar concerns actions taken by the TRE operator to manage systems for storing, retrieving, analysing and sending information. + +Each TRE operator will have its own computing technology requirements. +Requirements are drawn from information governance requirements which primarily specify the controls needed within the technical and computing infrastructure. +Researcher personas influence the according to the technical knowledge and experience of researchers along with the work they need to perform within the system, a data scientist will have very different requirements to a clinician for example. +The required compute resources will vary according to the scale of data and computational techniques employed during research. + ## End user computing _The ability of the TRE operator to provide and manage devices, workspaces, interfaces and applications used by researchers to interact with underlying systems and data._ -### User interface +### End user computing interfaces -_The interfaces used for interacting with the TRE management system and the TRE workspace._ +Software or system that allows people to interact with computers, applications, and services. +This application group includes desktop, command line and code submission interface. ```{list-table} :header-rows: 1 -:name: tab-end-user-user-interface +:name: tab-end-user-computing-interfaces * - Statement - Guidance - Importance -* - Your TRE should be accessed via a user interface accessible using commonly available applications. - - TREs which allow users to connect from their own devices should not require the installation of any bespoke TRE application on the user's device. - In practice a web browser is the most common way to achieve this. + +* - You should disable the ability to copy data out of your TRE via the system clipboard. + - A TRE user must not be able to copy sensitive data out of a workspace using the system clipboard. + A TRE may allow user to paste text into a workspace. - Recommended * - Your TRE workspace should provide an environment familiar to your users. - This may take the form of a virtual Windows or Linux desktops, non-desktop interfaces such as JupyterLab and other web applications, or a terminal. Bespoke TRE-specific software should be avoided when widely used alternatives already exist. - Recommended -* - Your TRE should take accessibility for users with disabilities into account. - - The restricted nature of TREs means many assistive tools such as screenreaders in a virtual desktop may not be allowed, but other options such as colour schemes, font sizes, and resizing user interface elements, should be supported. - - Recommended -* - You should disable the ability to copy data out of your TRE via the system clipboard. - - A TRE user must not be able to copy sensitive data out of a workspace using the system clipboard. - A TRE may allow user to paste text into a workspace. - - Recommended * - A TRE could restrict data access from researchers entirely and provide an interface for submitting code. - For example, you might use a system where users submit jobs that run over the data and return results without allowing direct data access. - Optional @@ -50,7 +51,7 @@ _The interfaces used for interacting with the TRE management system and the TRE ### Software tools -_The tools used by researchers inside a TRE, such as programming languages, IDEs and desktop applications._ +The tools used by researchers inside a TRE, such as programming languages, IDEs and desktop applications. ```{list-table} :header-rows: 1 @@ -59,20 +60,14 @@ _The tools used by researchers inside a TRE, such as programming languages, IDEs * - Statement - Guidance - Importance -* - Your TRE must provide software applications that are relevant to working with the data in the TRE. - - The tools provided will depend on the types of data in the TRE, and the expectations of users of the TRE. - For users working in a TRE via a virtual desktop, this may include programming languages such as Python and R, integrated development environments, Jupyter notebooks, office type applications such as word processors and spreadsheets, command line tools, etc. - TREs with non-desktop interfaces should similarly consider carefully which applications are best suited for the researchers needs when interacting with the data, for example "point and click" GUI tools for querying a database and generating plots of data. - The set of tools should be reviewed regularly to ensure they are up to date. - - Mandatory +* - Your TRE should be accessed via a user interface accessible using commonly available applications. + - TREs which allow users to connect from their own devices should not require the installation of any bespoke TRE application on the user's device. + In practice a web browser is the most common way to achieve this. + - Recommended * - Your TRE must provide clear guidance on how to use software tools and work with data in the TRE. - TREs that provide a virtual desktop environment for researchers to work in should provide documentation detailing the available tools. TREs where the analysis code is developed on the access machine (as opppose to within the TRE) should provide documentation detailing the mechanism by which code is submitted to the TRE. - Mandatory -* - Your TRE should provide tools to encourage best-practice in reproducibly analysing data. - - Reproducibility of analyses improves auditability and accountability of how data has been used, as well as being best-practice in research. - This may include version control software, and tools for developing and running data analysis pipelines. - - Recommended * - Your TRE should, where possible, automatically apply security related updates for user software. - Reducing the risk of exploitable vulnerabilities in installed software will increase the security of your TRE. - Recommended @@ -80,18 +75,57 @@ _The tools used by researchers inside a TRE, such as programming languages, IDEs - This may include shared file storage, databases, collaborative writing, and other web applications. This must only be shared amongst users within the same project. - Optional +* - Your TRE could include licenced commercial software if required by researchers, but additional risks must be recorded and mitigated where neccesary. + - For example, if an application must connect to an external licensing server, you must be confident that only licensing information is sent to this server, and that any network connections are secure. + - Optional +* - Your TRE must provide software applications that are relevant to working with the data in the TRE. + - The tools provided will depend on the types of data in the TRE, and the expectations of users of the TRE. + For users working in a TRE via a virtual desktop, this may include programming languages such as Python and R, integrated development environments, Jupyter notebooks, office type applications such as word processors and spreadsheets, command line tools, etc. + TREs with non-desktop interfaces should similarly consider carefully which applications are best suited for the researchers needs when interacting with the data, for example "point and click" GUI tools for querying a database and generating plots of data. + The set of tools should be reviewed regularly to ensure they are up to date. + - Mandatory +* - Your TRE should take accessibility for users with disabilities into account. + - The restricted nature of TREs means many assistive tools such as screenreaders in a virtual desktop may not be allowed, but other options such as colour schemes, font sizes, and resizing user interface elements, should be supported. + - Recommended +``` + +### Code Version Control System + +An application that provides version control and collaboration features for code developed inside the TRE. + +```{list-table} +:header-rows: 1 +:name: tab-end-user-code-vcs + +* - Statement + - Guidance + - Importance +* - Your TRE should provide tools to encourage best-practice in reproducibly analysing data. + - Reproducibility of analyses improves auditability and accountability of how data has been used, as well as being best-practice in research. + This may include version control software, and tools for developing and running data analysis pipelines. + - Recommended +``` + +### Artefact management + +A service that manages and organizes software artefacts such as code libraries, dependencies, and build outputs in a centralised repository. + +```{list-table} +:header-rows: 1 +:name: tab-end-user-artefact-management + +* - Statement + - Guidance + - Importance * - Your TRE could provide limited access to some public software repositories or container registries. - For example, a TRE may allow installation of packages from Python or R repositories, or provide an internal mirror with approved packages. Similarly a subset of public containers could be made available, or individual container images via an internal container registry. - Optional -* - Your TRE could include licenced commercial software if required by researchers, but additional risks must be recorded and mitigated where neccesary. - - For example, if an application must connect to an external licensing server, you must be confident that only licensing information is sent to this server, and that any network connections are secure. - - Optional ``` ### Advanced or cluster computing -_The ability to run analyses requiring more compute resources, or more specialised hardware, than is present in the user's workspace._ +Advanced, powerful computer resources to solve complex problems and process large amounts of data, possibly using specialised hardware. ```{list-table} :header-rows: 1 @@ -104,6 +138,10 @@ _The ability to run analyses requiring more compute resources, or more specialis - If a TRE supports users conducting computationally intensive research it should provide access to dynamically scaleable compute or the equivalent. For example this may be in the form of a batch scheduler on a HPC cluster, or a dynamically created compute nodes on a cloud platform. - Recommended +* - Your TRE could make data available to researchers using common database systems such as PostgreSQL, MSSQL or MongoDB. + - Databases must be secured and only accessible to users within the same project. + If shared (multi-tenant) database servers are used, database administrators must ensure that the database server enforces segregation of users and databases belonging to different projects. + - Optional * - Your TRE should be able to provide access to accelerators such as GPUs if required by users. - GPUs and other accelerators are commonly used in machine learning and other computationally intensive research. TREs should make it clear to users whether GPUs and other resources are available whilst projects are being assessed. @@ -114,10 +152,6 @@ _The ability to run analyses requiring more compute resources, or more specialis For example, when using physical compute resources, all sensitive data could be securely wiped before another user is given access to that same node. In a cloud hosted TRE virtual machines could be destroyed and recreated. - Mandatory -* - Your TRE could make data available to researchers using common databases such as PostgreSQL, MSSQL or MongoDB. - - Databases must be secured and only accessible to users within the same project. - If shared (multi-tenant) database servers are used, database administrators must ensure that the database server enforces segregation of users and databases belonging to different projects. - - Optional * - Your TRE could integrate with large-scale data analytics tools for working with large datasets. - For example, Spark and Hadoop can be used for distributed computing across a cluster. This may be an advantage where a TRE is using an amount of data that is too large for single-machine computing to be practical. @@ -128,35 +162,83 @@ _The ability to run analyses requiring more compute resources, or more specialis - Optional ``` -## Infrastructure analytics +## Infrastructure management + +The ability of the TRE operator to instantiate, deploy, change or remove physical or virtual infrastructure. + +### Infrastructure deployment -_The ability of the TRE operator to record and analyse data about the usage of the TRE._ +The process of setting up and configuring infrastructure components and resources to support applications or services. +This involves development, installation, configuration, and validation. ```{list-table} :header-rows: 1 -:name: tab-end-user-infrastructure-analytics +:name: tab-infrastructure-deployment * - Statement - Guidance - Importance -* - Your TRE must record usage data. - - This may include the number of users, number of projects, the amount of data stored, number of datasets, the number of workspaces, etc. +* - You must have a documented procedure for deploying infrastructure. + - This might, for instance, be a handbook that is followed or a set of automated scripts. - Mandatory -* - Your TRE should record which datasets are accessed, when and by who. - - This helps maintain auditability of how sensitive data has been used. +* - You must have a documented procedure for making changes to deployed infrastructure. + - This refers both to changes that might be expected in the course of normal operation and emergency changes that might be needed. + Your change management process may form part of a wider accreditation such as ISO 27001. + - Mandatory +* - You should, where possible, automate any repeatable aspects of your deployment. + - This might involve using infrastructure-as-code tools or simply a series of scripts. - Recommended -* - Your TRE should record computational resource usage at the user or aggregate level. - - This is useful for optimising allocation of resources, and managing costs. +* - You must test changes before they are used in production. + - This might involve a separate development environment or another system for testing. + - Mandatory +* - You could test changes in a development environment that mirrors your production system. + - Consider the costs and practicality of whether this will work for your situation. + - Optional +``` + +### Infrastructure decommissioning + +The process of retiring or removing infrastructure assets that are no longer needed or outdated, ensuring proper data handling and disposal. + +```{list-table} +:header-rows: 1 +:name: tab-infrastructure-decommissioning + +* - Statement + - Guidance + - Importance +* - You must have a documented procedure for removing infrastructure when it is no longer needed + - Removing unused infrastructure not only reduces costs and management burden but also reduces the attack surface of a TRE and reduces the risk of unaddressed vulnerabilities. + - Mandatory +``` + +### Availability management + +The process of ensuring all IT infrastructure meets the agreed levels of availability. + +```{list-table} +:header-rows: 1 +:name: tab-infrastructure-availability + +* - Statement + - Guidance + - Importance +* - You should understand the availability and uptime guarantees of any providers that you rely on. + - For remote TREs this might include your cloud provider(s) and/or data centre operators. + For on-premises TREs, it might be worth considering your ISP and electricity provider. + - Recommended +* - You should develop an availability target or statement and share this with your users. + - Understanding how and when the TRE might be unavailable will help your projects in planning their work. - Recommended ``` -## Network management +### Network management -_The ability of the TRE operator to administer and secure network infrastructure using applications, tools and processes._ +An application used to manage and control network devices, ensuring proper functioning, security, and performance of the network. ```{list-table} :header-rows: 1 -:name: tab-end-user-network-management +:name: tab-infrastructure-network * - Statement - Guidance @@ -165,60 +247,47 @@ _The ability of the TRE operator to administer and secure network infrastructure - Network infrastructure must prevent unauthorised access to resources on the network. This may include firewalls, network segmentation, and restricting connections to the network. - Mandatory -* - You must monitor the network configuration of your TRE to check for misconfigurations and vulnerabilities. - - This may include regular vulnerability scanning, and penetration testing. - - Mandatory * - Your TRE must not allow connectivity between users in different projects, or with access to different datasets. - Connectivity between users in the same project may be allowed, for example to support shared network services within the project. - Mandatory * - Your TRE must block outbound connections to the internet by default. - Limited outbound connectivity may be allowed for some services. - Mandatory +* - You must monitor the network configuration of your TRE to check for misconfigurations and vulnerabilities. + - This may include regular vulnerability scanning, and penetration testing. + - Mandatory ``` -## Infrastructure lifecycle management - -_The ability of the TRE operator to manage necessary physical or virtual infrastructure._ - -### Deployment management +### Infrastructure analytics -_The ability of the TRE operator to instantiate, deploy, change or remove deployed infrastructure._ +The ability of the TRE operator to record and analyse data about the usage of the TRE. ```{list-table} :header-rows: 1 -:name: tab-deployment-management +:name: tab-end-user-infrastructure-analytics * - Statement - Guidance - Importance -* - You must have a documented procedure for deploying infrastructure. - - This might, for instance, be a handbook that is followed or a set of automated scripts. +* - Your TRE must record usage data. + - This may include the number of users, number of projects, the amount of data stored, number of datasets, the number of workspaces, etc. - Mandatory -* - You should, where possible, automate any repeatable aspects of your deployment. - - This might involve using infrastructure-as-code tools or simply a series of scripts. +* - Your TRE should record which datasets are accessed, when and by who. + - This helps maintain auditability of how sensitive data has been used. + - Recommended +* - Your TRE should record computational resource usage at the user or aggregate level. + - This is useful for optimising allocation of resources, and managing costs. - Recommended -* - You must have a documented procedure for making changes to deployed infrastructure. - - This refers both to changes that might be expected in the course of normal operation and emergency changes that might be needed. - Your change management process may form part of a wider accreditation such as ISO 27001. - - Mandatory -* - You must test changes before they are used in production. - - This might involve a separate development environment or another system for testing. - - Mandatory -* - You could test changes in a development environment that mirrors your production system. - - Consider the costs and practicality of whether this will work for your situation. - - Optional -* - You must have a documented procedure for removing infrastructure when it is no longer needed - - Removing unused infrastructure not only reduces costs and management burden but also reduces the attack surface of a TRE and reduces the risk of unaddressed vulnerabilities. - - Mandatory ``` -### Capacity management +## Capacity management +### Capacity planning -_The ability of the TRE operator to ensure the right amount of resources are available at the right time to provide a service._ +The process of forecasting and determining the resources required to meet the demands of an application or system, ensuring that adequate resources are available when needed. ```{list-table} :header-rows: 1 -:name: tab-capacity-management +:name: tab-infrastructure-capacity * - Statement - Guidance @@ -229,10 +298,6 @@ _The ability of the TRE operator to ensure the right amount of resources are ava * - You should ensure that the anticipated needs of projects can be satisfied using available resources. - Note that this does not require you to accept requests for additional resources, but rather that promises made about resource availability before a project starts should be honoured wherever possible. - Recommended -* - You must ensure that the anticipated resource requirements will not result in overspending by the TRE. - - For cloud-based TREs this may involve budgeting and/or restricting resource consumption on a project-by-project basis. - For on-premises TREs this may involve managing expectations to match the available resource. - - Mandatory * - You must have a procedure for increasing/decreasing available resources. - For cloud-based TREs this may involve scaling resources, such as virtual machines or databases, or deploying additional resources. For on-premises TREs this may involve a procurement process to ensure that necessary resources are available. @@ -242,13 +307,31 @@ _The ability of the TRE operator to ensure the right amount of resources are ava - Mandatory ``` +### Billing + +The process of generating and managing invoices and bills for projects within the TRE. +It involves calculation, issuance, and recording of payments and receipts. + +```{list-table} +:header-rows: 1 +:name: tab-infrastructure-billing + +* - Statement + - Guidance + - Importance +* - You must ensure that the anticipated resource requirements will not result in overspending by the TRE. + - For cloud-based TREs this may involve budgeting and/or restricting resource consumption on a project-by-project basis. + For on-premises TREs this may involve managing expectations to match the available resource. + - Mandatory +``` + ### Configuration management -_The ability of the TRE operator to identify, maintain, and verify information on IT assets and configurations in the TRE operator._ +The ability of the TRE operator to identify, maintain, and verify information on IT assets and configurations in the TRE operator. ```{list-table} :header-rows: 1 -:name: tab-configuration-management +:name: tab-infrastructure-configuration * - Statement - Guidance @@ -275,23 +358,3 @@ _The ability of the TRE operator to identify, maintain, and verify information o - Virus and malware scans will help identify malicious code which may compromise the security, or correct operation, of the TRE. - Mandatory ``` - -## Availability management - -_The ability of the TRE operator to ensure all IT infrastructure, processes, tools, roles etc. are appropriate for the agreed availability targets._ - -```{list-table} -:header-rows: 1 -:name: tab-availability-management - -* - Statement - - Guidance - - Importance -* - You should understand the availability and uptime guarantees of any providers that you rely on. - - For remote TREs this might include your cloud provider(s) and/or data centre operators. - For on-premises TREs, it might be worth considering your ISP and electricity provider. - - Recommended -* - You should develop an availability target or statement and share this with your users. - - Understanding how and when the TRE might be unavailable will help your projects in planning their work. - - Recommended -``` From 5252988fbe3cf0762649a0a5351c391c664bd101 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 12:31:00 +0100 Subject: [PATCH 02/29] :sparkles: Add/combine statements as suggested by @machintim --- docs/source/pillars/computing_technology.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 055522f5..790baa79 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -2,7 +2,6 @@ # Computing technology - ```{figure} ../../images/Capability_Map/full.drawio.svg :alt: SATRE Pillars Capability Map :align: center @@ -121,6 +120,10 @@ A service that manages and organizes software artefacts such as code libraries, - For example, a TRE may allow installation of packages from Python or R repositories, or provide an internal mirror with approved packages. Similarly a subset of public containers could be made available, or individual container images via an internal container registry. - Optional +* - You could choose to tightly control access to public repositories. + - This might involve only allowing a limited selection of allowed repositories. + You might also choose to scan for malicious packages and/or go through an approval process before allowing code into the technical environment. + - Optional ``` ### Advanced or cluster computing @@ -191,9 +194,10 @@ This involves development, installation, configuration, and validation. * - You must test changes before they are used in production. - This might involve a separate development environment or another system for testing. - Mandatory -* - You could test changes in a development environment that mirrors your production system. - - Consider the costs and practicality of whether this will work for your situation. - - Optional +* - You should have separate technical environments for development and testing infrastructure changes before they are committed to production. + - If possible, you should automate application of changes between development and production environments. + Consider the costs and practicality of whether this will work for your situation. + - Recommended ``` ### Infrastructure decommissioning @@ -281,6 +285,7 @@ The ability of the TRE operator to record and analyse data about the usage of th ``` ## Capacity management + ### Capacity planning The process of forecasting and determining the resources required to meet the demands of an application or system, ensuring that adequate resources are available when needed. @@ -301,9 +306,7 @@ The process of forecasting and determining the resources required to meet the de * - You must have a procedure for increasing/decreasing available resources. - For cloud-based TREs this may involve scaling resources, such as virtual machines or databases, or deploying additional resources. For on-premises TREs this may involve a procurement process to ensure that necessary resources are available. - - Mandatory -* - You must have a procedure to decide when to change capacity. - - Not all requests for capacity increase must necessarily be granted, but having a clear process will help projects understand when/why/how they can make use of additional capacity. + Not all requests for capacity increase must necessarily be granted, but having a clear process will help projects understand when/why/how they can make use of additional capacity. - Mandatory ``` From b02291c646dd1903764948fdde1d795835f3aa33 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 12:45:09 +0100 Subject: [PATCH 03/29] :coffin: Remove 'Your TRE should take accessibility for users with disabilities into account.' --- docs/source/pillars/computing_technology.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 790baa79..23931b4c 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -83,9 +83,6 @@ The tools used by researchers inside a TRE, such as programming languages, IDEs TREs with non-desktop interfaces should similarly consider carefully which applications are best suited for the researchers needs when interacting with the data, for example "point and click" GUI tools for querying a database and generating plots of data. The set of tools should be reviewed regularly to ensure they are up to date. - Mandatory -* - Your TRE should take accessibility for users with disabilities into account. - - The restricted nature of TREs means many assistive tools such as screenreaders in a virtual desktop may not be allowed, but other options such as colour schemes, font sizes, and resizing user interface elements, should be supported. - - Recommended ``` ### Code Version Control System From 1988a366e111f9bdc85660204b42b3a38ac2a397 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 12:45:25 +0100 Subject: [PATCH 04/29] :coffin: Remove 'Your TRE could integrate with cloud-native managed services.' --- docs/source/pillars/computing_technology.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 23931b4c..fd1d2ef7 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -156,10 +156,6 @@ Advanced, powerful computer resources to solve complex problems and process larg - For example, Spark and Hadoop can be used for distributed computing across a cluster. This may be an advantage where a TRE is using an amount of data that is too large for single-machine computing to be practical. - Optional -* - Your TRE could integrate with cloud-native managed services. - - Cloud providers supply many different managed services. - Although the cloud provider is responsible for managing the configuration of these services, the TRE operator must ensure that using them does not compromise the security of the TRE. - - Optional ``` ## Infrastructure management From a08505cb485c24e62b721e35e5a2c89250148787 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 12:46:02 +0100 Subject: [PATCH 05/29] :coffin: Remove 'Your TRE must provide software applications that are relevant to working with the data in the TRE.' --- docs/source/pillars/computing_technology.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index fd1d2ef7..e0486e53 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -77,12 +77,6 @@ The tools used by researchers inside a TRE, such as programming languages, IDEs * - Your TRE could include licenced commercial software if required by researchers, but additional risks must be recorded and mitigated where neccesary. - For example, if an application must connect to an external licensing server, you must be confident that only licensing information is sent to this server, and that any network connections are secure. - Optional -* - Your TRE must provide software applications that are relevant to working with the data in the TRE. - - The tools provided will depend on the types of data in the TRE, and the expectations of users of the TRE. - For users working in a TRE via a virtual desktop, this may include programming languages such as Python and R, integrated development environments, Jupyter notebooks, office type applications such as word processors and spreadsheets, command line tools, etc. - TREs with non-desktop interfaces should similarly consider carefully which applications are best suited for the researchers needs when interacting with the data, for example "point and click" GUI tools for querying a database and generating plots of data. - The set of tools should be reviewed regularly to ensure they are up to date. - - Mandatory ``` ### Code Version Control System From 38d7785ddce8bf8f8d01535a212e92fe44a07b26 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 12:46:34 +0100 Subject: [PATCH 06/29] :coffin: Remove 'Your TRE should be able to provide access to accelerators such as GPUs if required by users.' --- docs/source/pillars/computing_technology.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index e0486e53..7ab2dc6f 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -136,10 +136,6 @@ Advanced, powerful computer resources to solve complex problems and process larg - Databases must be secured and only accessible to users within the same project. If shared (multi-tenant) database servers are used, database administrators must ensure that the database server enforces segregation of users and databases belonging to different projects. - Optional -* - Your TRE should be able to provide access to accelerators such as GPUs if required by users. - - GPUs and other accelerators are commonly used in machine learning and other computationally intensive research. - TREs should make it clear to users whether GPUs and other resources are available whilst projects are being assessed. - - Recommended * - Your TRE must maintain segregation of users and data from different projects when using non-standard compute. - High performance or specialist compute is often shared amongst multiple users. Users and data must remain segregated at all times. From 826ff45ee52c8ce7e6fa45bdb67ec138f6e3e264 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 12:46:59 +0100 Subject: [PATCH 07/29] :coffin: Remove 'Your TRE must maintain segregation of users and data from different projects when using non-standard compute.' --- docs/source/pillars/computing_technology.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 7ab2dc6f..2a6412f5 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -136,12 +136,6 @@ Advanced, powerful computer resources to solve complex problems and process larg - Databases must be secured and only accessible to users within the same project. If shared (multi-tenant) database servers are used, database administrators must ensure that the database server enforces segregation of users and databases belonging to different projects. - Optional -* - Your TRE must maintain segregation of users and data from different projects when using non-standard compute. - - High performance or specialist compute is often shared amongst multiple users. - Users and data must remain segregated at all times. - For example, when using physical compute resources, all sensitive data could be securely wiped before another user is given access to that same node. - In a cloud hosted TRE virtual machines could be destroyed and recreated. - - Mandatory * - Your TRE could integrate with large-scale data analytics tools for working with large datasets. - For example, Spark and Hadoop can be used for distributed computing across a cluster. This may be an advantage where a TRE is using an amount of data that is too large for single-machine computing to be practical. From 546e5d05ffe4ea5824f51e7698d869416e65e469 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 12:47:23 +0100 Subject: [PATCH 08/29] :coffin: 'You must have a documented procedure for making changes to deployed infrastructure.' --- docs/source/pillars/computing_technology.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 2a6412f5..73fba3b0 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -161,10 +161,6 @@ This involves development, installation, configuration, and validation. * - You must have a documented procedure for deploying infrastructure. - This might, for instance, be a handbook that is followed or a set of automated scripts. - Mandatory -* - You must have a documented procedure for making changes to deployed infrastructure. - - This refers both to changes that might be expected in the course of normal operation and emergency changes that might be needed. - Your change management process may form part of a wider accreditation such as ISO 27001. - - Mandatory * - You should, where possible, automate any repeatable aspects of your deployment. - This might involve using infrastructure-as-code tools or simply a series of scripts. - Recommended From 9e8702b66b20e981a906a81061daf6361090fd15 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 13:50:25 +0100 Subject: [PATCH 09/29] :coffin: Remove 'You must monitor the network configuration of your TRE to check for misconfigurations and vulnerabilities.' --- docs/source/pillars/computing_technology.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 73fba3b0..5935c439 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -230,9 +230,6 @@ An application used to manage and control network devices, ensuring proper funct * - Your TRE must block outbound connections to the internet by default. - Limited outbound connectivity may be allowed for some services. - Mandatory -* - You must monitor the network configuration of your TRE to check for misconfigurations and vulnerabilities. - - This may include regular vulnerability scanning, and penetration testing. - - Mandatory ``` ### Infrastructure analytics From d3cb03f7bb8ddd94acc918e20821d69d4595e8a5 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 7 Aug 2023 13:51:12 +0100 Subject: [PATCH 10/29] :coffin: Remove 'Infrastructure Analytics' --- docs/source/pillars/computing_technology.md | 22 --------------------- 1 file changed, 22 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 5935c439..ad2ce3b2 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -232,28 +232,6 @@ An application used to manage and control network devices, ensuring proper funct - Mandatory ``` -### Infrastructure analytics - -The ability of the TRE operator to record and analyse data about the usage of the TRE. - -```{list-table} -:header-rows: 1 -:name: tab-end-user-infrastructure-analytics - -* - Statement - - Guidance - - Importance -* - Your TRE must record usage data. - - This may include the number of users, number of projects, the amount of data stored, number of datasets, the number of workspaces, etc. - - Mandatory -* - Your TRE should record which datasets are accessed, when and by who. - - This helps maintain auditability of how sensitive data has been used. - - Recommended -* - Your TRE should record computational resource usage at the user or aggregate level. - - This is useful for optimising allocation of resources, and managing costs. - - Recommended -``` - ## Capacity management ### Capacity planning From 128419ce01aa0788a0eae1cae6c23cc806ec659d Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 11:50:01 +0100 Subject: [PATCH 11/29] :memo: Apply rewording suggestions from review Co-authored-by: Jim Madge --- docs/source/pillars/computing_technology.md | 22 ++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index ad2ce3b2..4666940e 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -9,7 +9,7 @@ SATRE Pillars Capability Map ``` -This pillar concerns actions taken by the TRE operator to manage systems for storing, retrieving, analysing and sending information. +This pillar concerns actions taken by the TRE operator to manage TRE computing systems. Each TRE operator will have its own computing technology requirements. Requirements are drawn from information governance requirements which primarily specify the controls needed within the technical and computing infrastructure. @@ -20,12 +20,12 @@ The required compute resources will vary according to the scale of data and comp ## End user computing -_The ability of the TRE operator to provide and manage devices, workspaces, interfaces and applications used by researchers to interact with underlying systems and data._ +The ability of the TRE operator to provide and manage devices, workspaces, interfaces and applications used by researchers to interact with underlying systems and data. ### End user computing interfaces -Software or system that allows people to interact with computers, applications, and services. -This application group includes desktop, command line and code submission interface. +Software or systems that allows people to interact with the TRE. +This may include desktop, command-line and/or code-submission interfaces. ```{list-table} :header-rows: 1 @@ -81,7 +81,7 @@ The tools used by researchers inside a TRE, such as programming languages, IDEs ### Code Version Control System -An application that provides version control and collaboration features for code developed inside the TRE. +Systems and tools providing version control and collaboration features for code developed inside the TRE. ```{list-table} :header-rows: 1 @@ -144,7 +144,7 @@ Advanced, powerful computer resources to solve complex problems and process larg ## Infrastructure management -The ability of the TRE operator to instantiate, deploy, change or remove physical or virtual infrastructure. +The ability of the TRE operator to deploy, change or remove physical or virtual infrastructure. ### Infrastructure deployment @@ -162,29 +162,29 @@ This involves development, installation, configuration, and validation. - This might, for instance, be a handbook that is followed or a set of automated scripts. - Mandatory * - You should, where possible, automate any repeatable aspects of your deployment. - - This might involve using infrastructure-as-code tools or simply a series of scripts. + - This might involve using infrastructure-as-code tools or a series of scripts. - Recommended * - You must test changes before they are used in production. - This might involve a separate development environment or another system for testing. - Mandatory -* - You should have separate technical environments for development and testing infrastructure changes before they are committed to production. +* - You should have separate environments for development and testing infrastructure changes before they are committed to production. - If possible, you should automate application of changes between development and production environments. Consider the costs and practicality of whether this will work for your situation. - Recommended ``` -### Infrastructure decommissioning +### Infrastructure removal The process of retiring or removing infrastructure assets that are no longer needed or outdated, ensuring proper data handling and disposal. ```{list-table} :header-rows: 1 -:name: tab-infrastructure-decommissioning +:name: tab-infrastructure-removal * - Statement - Guidance - Importance -* - You must have a documented procedure for removing infrastructure when it is no longer needed +* - You must have a documented procedure for removing infrastructure when it is no longer needed. - Removing unused infrastructure not only reduces costs and management burden but also reduces the attack surface of a TRE and reduces the risk of unaddressed vulnerabilities. - Mandatory ``` From aba156f2268577c3b1208781c02faaec1a0edff3 Mon Sep 17 00:00:00 2001 From: Ed Chalstrey Date: Tue, 8 Aug 2023 14:05:11 +0100 Subject: [PATCH 12/29] Update docs/source/pillars/computing_technology.md Co-authored-by: James Robinson --- docs/source/pillars/computing_technology.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 4666940e..960e3205 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -251,7 +251,7 @@ The process of forecasting and determining the resources required to meet the de * - You should ensure that the anticipated needs of projects can be satisfied using available resources. - Note that this does not require you to accept requests for additional resources, but rather that promises made about resource availability before a project starts should be honoured wherever possible. - Recommended -* - You must have a procedure for increasing/decreasing available resources. +* - You must have a procedure for allocating available resources among projects. - For cloud-based TREs this may involve scaling resources, such as virtual machines or databases, or deploying additional resources. For on-premises TREs this may involve a procurement process to ensure that necessary resources are available. Not all requests for capacity increase must necessarily be granted, but having a clear process will help projects understand when/why/how they can make use of additional capacity. From 4f1f89c3a6f82aa803b8377020a36a59d5d544cd Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:13:48 +0100 Subject: [PATCH 13/29] :memo: Apply rewording suggestions from @craddm Co-authored-by: Matt Craddock --- docs/source/pillars/computing_technology.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 960e3205..e21be81b 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -12,8 +12,9 @@ SATRE Pillars Capability Map This pillar concerns actions taken by the TRE operator to manage TRE computing systems. Each TRE operator will have its own computing technology requirements. -Requirements are drawn from information governance requirements which primarily specify the controls needed within the technical and computing infrastructure. -Researcher personas influence the according to the technical knowledge and experience of researchers along with the work they need to perform within the system, a data scientist will have very different requirements to a clinician for example. +The security controls needed by the computing infrastructure will depend on information governance requirements. +Other computing requirements will be influenced by the technical knowledge and experience of those using the TRE, along with the work they need to perform within the system. +For example, a data scientist will have very different requirements to a clinician. The required compute resources will vary according to the scale of data and computational techniques employed during research. From e2246a39fcc677b7c506cd488bdc362380f3e4ca Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:16:27 +0100 Subject: [PATCH 14/29] :wrench: Make disable copy required. --- docs/source/pillars/computing_technology.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index e21be81b..35599bff 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -36,10 +36,11 @@ This may include desktop, command-line and/or code-submission interfaces. - Guidance - Importance -* - You should disable the ability to copy data out of your TRE via the system clipboard. +* - You must not allow users to copy data out of your TRE via the system clipboard. - A TRE user must not be able to copy sensitive data out of a workspace using the system clipboard. A TRE may allow user to paste text into a workspace. - - Recommended + This might not be relevant to your TRE, for example if your user interface does not have a clipboard. + - Required * - Your TRE workspace should provide an environment familiar to your users. - This may take the form of a virtual Windows or Linux desktops, non-desktop interfaces such as JupyterLab and other web applications, or a terminal. Bespoke TRE-specific software should be avoided when widely used alternatives already exist. From be1f53a7fcb3ead2456c22624d195d91cee1cfca Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:18:22 +0100 Subject: [PATCH 15/29] :rewind: Revert ":coffin: Remove 'Your TRE must provide software applications that are relevant to working with the data in the TRE.'" This reverts commit a08505cb485c24e62b721e35e5a2c89250148787. --- docs/source/pillars/computing_technology.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 35599bff..6619a31b 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -79,6 +79,12 @@ The tools used by researchers inside a TRE, such as programming languages, IDEs * - Your TRE could include licenced commercial software if required by researchers, but additional risks must be recorded and mitigated where neccesary. - For example, if an application must connect to an external licensing server, you must be confident that only licensing information is sent to this server, and that any network connections are secure. - Optional +* - Your TRE must provide software applications that are relevant to working with the data in the TRE. + - The tools provided will depend on the types of data in the TRE, and the expectations of users of the TRE. + For users working in a TRE via a virtual desktop, this may include programming languages such as Python and R, integrated development environments, Jupyter notebooks, office type applications such as word processors and spreadsheets, command line tools, etc. + TREs with non-desktop interfaces should similarly consider carefully which applications are best suited for the researchers needs when interacting with the data, for example "point and click" GUI tools for querying a database and generating plots of data. + The set of tools should be reviewed regularly to ensure they are up to date. + - Mandatory ``` ### Code Version Control System From 72140c2e65af09aa8ab480aea9dd7f07824604ff Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:21:45 +0100 Subject: [PATCH 16/29] :rewind: Revert ':coffin: Remove 'Your TRE should be able to provide access to accelerators such as GPUs if required by users.'' --- docs/source/pillars/computing_technology.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 6619a31b..85740ca2 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -13,7 +13,7 @@ This pillar concerns actions taken by the TRE operator to manage TRE computing s Each TRE operator will have its own computing technology requirements. The security controls needed by the computing infrastructure will depend on information governance requirements. -Other computing requirements will be influenced by the technical knowledge and experience of those using the TRE, along with the work they need to perform within the system. +Other computing requirements will be influenced by the technical knowledge and experience of those using the TRE, along with the work they need to perform within the system. For example, a data scientist will have very different requirements to a clinician. The required compute resources will vary according to the scale of data and computational techniques employed during research. @@ -140,6 +140,10 @@ Advanced, powerful computer resources to solve complex problems and process larg - If a TRE supports users conducting computationally intensive research it should provide access to dynamically scaleable compute or the equivalent. For example this may be in the form of a batch scheduler on a HPC cluster, or a dynamically created compute nodes on a cloud platform. - Recommended +* - Your TRE should be able to provide access to accelerators such as GPUs if required by users. + - GPUs and other accelerators are commonly used in machine learning and other computationally intensive research. + TREs should make it clear to users whether GPUs and other resources are available whilst projects are being assessed. + - Recommended * - Your TRE could make data available to researchers using common database systems such as PostgreSQL, MSSQL or MongoDB. - Databases must be secured and only accessible to users within the same project. If shared (multi-tenant) database servers are used, database administrators must ensure that the database server enforces segregation of users and databases belonging to different projects. From 59c90cf1bc343a6eea9e0b727090a30a10a045e2 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:24:02 +0100 Subject: [PATCH 17/29] :truck: Move accessibility into principles --- docs/source/specification.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/source/specification.md b/docs/source/specification.md index 59ee415e..28729c32 100644 --- a/docs/source/specification.md +++ b/docs/source/specification.md @@ -66,6 +66,7 @@ In addition, we also describe some {ref}`roles ` that are necessary The SATRE specification has been developed based on the following principles: - TREs should be as easy as possible for end-users to use (_e.g._ researchers) whilst still remaining secure. +- TREs should take accessibility for all users with disabilities into account - TRE deployments should be offered that support data of different levels of sensitivity (_e.g._ through a tiered system of technical controls and policies). - TREs conforming to the specification should be interoperable and provide a familiar end-user experience. - The specification will be managed and updated following an open, community-driven process, and will not be tied to a single vendor or implementation. From 3e872dff411e1e6796430cbbc4444cdf5591f035 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:36:31 +0100 Subject: [PATCH 18/29] :memo: Disentangle software repository options --- docs/source/pillars/computing_technology.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 85740ca2..7c86d8b5 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -106,7 +106,7 @@ Systems and tools providing version control and collaboration features for code ### Artefact management -A service that manages and organizes software artefacts such as code libraries, dependencies, and build outputs in a centralised repository. +A service that manages and organizes third-party software artefacts such as packaged code libraries or containers in a centralised repository. ```{list-table} :header-rows: 1 @@ -115,12 +115,11 @@ A service that manages and organizes software artefacts such as code libraries, * - Statement - Guidance - Importance -* - Your TRE could provide limited access to some public software repositories or container registries. - - For example, a TRE may allow installation of packages from Python or R repositories, or provide an internal mirror with approved packages. - Similarly a subset of public containers could be made available, or individual container images via an internal container registry. +* - Your TRE could provide access to some public software repositories or container registries. + - For example, a TRE may allow direct installation of packages from Python or R repositories, or provide an internal mirror. - Optional -* - You could choose to tightly control access to public repositories. - - This might involve only allowing a limited selection of allowed repositories. +* - Your TRE could tightly control which packages are available. + - For example, a TRE may only allow installation of a pre-defined set of approved packages. You might also choose to scan for malicious packages and/or go through an approval process before allowing code into the technical environment. - Optional ``` From 89083973496c81c54532a6315736448f0b921760 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:41:08 +0100 Subject: [PATCH 19/29] :rewind: Reverts 'Remove 'Your TRE must maintain segregation of users and data from different projects when using non-standard compute.'' --- docs/source/pillars/computing_technology.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 7c86d8b5..ea9e9e0b 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -135,6 +135,12 @@ Advanced, powerful computer resources to solve complex problems and process larg * - Statement - Guidance - Importance +* - Your TRE must maintain segregation of users and data from different projects when using non-standard compute. + - High performance or specialist compute is often shared amongst multiple users. + Users and data must remain segregated at all times. + For example, when using physical compute resources, all sensitive data could be securely wiped before another user is given access to that same node. + In a cloud hosted TRE virtual machines could be destroyed and recreated. + - Mandatory * - Your TRE should be able to provide access to high performance computing or other scaleable compute resource if required by users. - If a TRE supports users conducting computationally intensive research it should provide access to dynamically scaleable compute or the equivalent. For example this may be in the form of a batch scheduler on a HPC cluster, or a dynamically created compute nodes on a cloud platform. From 39cbbe0ee840ef3a5d8f7109e918f4f2f4cd7026 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:42:27 +0100 Subject: [PATCH 20/29] :rewind: Revert ":coffin: Remove 'You must monitor the network configuration of your TRE to check for misconfigurations and vulnerabilities.'" This reverts commit 9e8702b66b20e981a906a81061daf6361090fd15. --- docs/source/pillars/computing_technology.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index ea9e9e0b..e5933dc7 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -247,6 +247,9 @@ An application used to manage and control network devices, ensuring proper funct * - Your TRE must block outbound connections to the internet by default. - Limited outbound connectivity may be allowed for some services. - Mandatory +* - You must monitor the network configuration of your TRE to check for misconfigurations and vulnerabilities. + - This may include regular vulnerability scanning, and penetration testing. + - Mandatory ``` ## Capacity management From 75baaf5c93e42c19ab43e86fa821d73e7ae12254 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:42:59 +0100 Subject: [PATCH 21/29] :rewind: Revert ":coffin: Remove 'Infrastructure Analytics'" This reverts commit d3cb03f7bb8ddd94acc918e20821d69d4595e8a5. --- docs/source/pillars/computing_technology.md | 22 +++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index e5933dc7..8a2b7240 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -252,6 +252,28 @@ An application used to manage and control network devices, ensuring proper funct - Mandatory ``` +### Infrastructure analytics + +The ability of the TRE operator to record and analyse data about the usage of the TRE. + +```{list-table} +:header-rows: 1 +:name: tab-end-user-infrastructure-analytics + +* - Statement + - Guidance + - Importance +* - Your TRE must record usage data. + - This may include the number of users, number of projects, the amount of data stored, number of datasets, the number of workspaces, etc. + - Mandatory +* - Your TRE should record which datasets are accessed, when and by who. + - This helps maintain auditability of how sensitive data has been used. + - Recommended +* - Your TRE should record computational resource usage at the user or aggregate level. + - This is useful for optimising allocation of resources, and managing costs. + - Recommended +``` + ## Capacity management ### Capacity planning From a942cc6c4c5fe68ac8b08c40e08dfacd98d9397b Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:44:57 +0100 Subject: [PATCH 22/29] :rewind: Revert ':coffin: 'You must have a documented procedure for making changes to deployed infrastructure'' --- docs/source/pillars/computing_technology.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 8a2b7240..a3946856 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -181,6 +181,10 @@ This involves development, installation, configuration, and validation. * - You should, where possible, automate any repeatable aspects of your deployment. - This might involve using infrastructure-as-code tools or a series of scripts. - Recommended +* - You must have a documented procedure for making changes to deployed infrastructure. + - This refers both to changes that might be expected in the course of normal operation and emergency changes that might be needed. + Your change management process may form part of a wider accreditation such as ISO 27001. + - Mandatory * - You must test changes before they are used in production. - This might involve a separate development environment or another system for testing. - Mandatory From 19a2a2d06c15fe6c51afb9a2fc5fea628bcc5ee7 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:47:12 +0100 Subject: [PATCH 23/29] :memo: Drop electricity provider suggestion --- docs/source/pillars/computing_technology.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index a3946856..9549d251 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -223,7 +223,7 @@ The process of ensuring all IT infrastructure meets the agreed levels of availab - Importance * - You should understand the availability and uptime guarantees of any providers that you rely on. - For remote TREs this might include your cloud provider(s) and/or data centre operators. - For on-premises TREs, it might be worth considering your ISP and electricity provider. + For on-premises TREs, it might be worth using an uninterruptable power supply (UPS) and planning how you would deal with internet outages. - Recommended * - You should develop an availability target or statement and share this with your users. - Understanding how and when the TRE might be unavailable will help your projects in planning their work. From 731356b9374e38c7e1b5f32f6168a1c71294fdb3 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 15:48:25 +0100 Subject: [PATCH 24/29] :memo: Update network management --- docs/source/pillars/computing_technology.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 9549d251..32167e20 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -232,7 +232,7 @@ The process of ensuring all IT infrastructure meets the agreed levels of availab ### Network management -An application used to manage and control network devices, ensuring proper functioning, security, and performance of the network. +An application used to manage network infrastructure, ensuring proper functioning, security, and performance. ```{list-table} :header-rows: 1 From 8665ded8cbfd2d0dbbc657b597ec0614724d65b4 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 16:20:25 +0100 Subject: [PATCH 25/29] :memo: Reword artefact management guidance. Co-authored-by: Jim Madge --- docs/source/pillars/computing_technology.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/pillars/computing_technology.md b/docs/source/pillars/computing_technology.md index 32167e20..905e1fb3 100644 --- a/docs/source/pillars/computing_technology.md +++ b/docs/source/pillars/computing_technology.md @@ -106,7 +106,7 @@ Systems and tools providing version control and collaboration features for code ### Artefact management -A service that manages and organizes third-party software artefacts such as packaged code libraries or containers in a centralised repository. +A service that manages and organises third-party software artefacts such as packaged code libraries or containers. ```{list-table} :header-rows: 1 From 83773841101f9997c3a206dc2304e4f935f93004 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 8 Aug 2023 16:21:17 +0100 Subject: [PATCH 26/29] :memo: Better accessibility wording Co-authored-by: Jim Madge --- docs/source/specification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/specification.md b/docs/source/specification.md index 28729c32..6bc63807 100644 --- a/docs/source/specification.md +++ b/docs/source/specification.md @@ -66,7 +66,7 @@ In addition, we also describe some {ref}`roles ` that are necessary The SATRE specification has been developed based on the following principles: - TREs should be as easy as possible for end-users to use (_e.g._ researchers) whilst still remaining secure. -- TREs should take accessibility for all users with disabilities into account +- TREs should take accessibility for all users, including those with disabilities, into account. - TRE deployments should be offered that support data of different levels of sensitivity (_e.g._ through a tiered system of technical controls and policies). - TREs conforming to the specification should be interoperable and provide a familiar end-user experience. - The specification will be managed and updated following an open, community-driven process, and will not be tied to a single vendor or implementation. From b0ac2590ac15cc9d9f709348ca80ca4fe89e73c5 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Thu, 10 Aug 2023 09:32:50 +0100 Subject: [PATCH 27/29] :memo: Update glossary --- docs/source/glossary.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/docs/source/glossary.md b/docs/source/glossary.md index 980082b3..bd7d603d 100644 --- a/docs/source/glossary.md +++ b/docs/source/glossary.md @@ -10,7 +10,8 @@ Definition of an actor ## Application component -Definition of a application component +An encapsulation of application functionality which is modular and replaceable. +For example: to perform work within a TRE a researcher might need access to a Desktop or command line interface application component. (def_application_service)= @@ -25,11 +26,12 @@ Definition of a application service Fundamental guidelines that inform the design, decision making and implementation of a TRE. These principles provide a framework to ensure that the design of the underlying components of a TRE are aligned to consistent goals, values and best practices. -(def_business_data_object)= +(def_business_process)= -## Business data object +## Business process -Definition of decomposition +A set of actions which produce a specific desired outcome. +For example: to access the TRE a researcher needs to complete an onboarding business process. (def_capability)= @@ -54,11 +56,14 @@ In addition to the components realising the capability, a catalogue of standards Definition of a component -(def_process)= +(def_data_object)= -## Process +## Data Object + +A store of data or information. +For example: to know what data is stored within the TRE a study database data object is needed. +This contains information on the data assets within the TRE, who owns them and other compliance information. -Definition of a process (def_role)= From 6bb6d28032d864b5e796c2256df6f9565403e7a8 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Thu, 10 Aug 2023 09:46:58 +0100 Subject: [PATCH 28/29] :recycle: Switch to native MyST glossary format for the glossary --- docs/source/glossary.md | 131 +++++++++++++---------------------- docs/source/specification.md | 13 ++-- 2 files changed, 55 insertions(+), 89 deletions(-) diff --git a/docs/source/glossary.md b/docs/source/glossary.md index bd7d603d..d78d3e75 100644 --- a/docs/source/glossary.md +++ b/docs/source/glossary.md @@ -1,85 +1,50 @@ # Commonly used terms -(def_actor)= - -## Actor - -Definition of an actor - -(def_application_component)= - -## Application component - -An encapsulation of application functionality which is modular and replaceable. -For example: to perform work within a TRE a researcher might need access to a Desktop or command line interface application component. - -(def_application_service)= - -## Application service - -Definition of a application service - -(def_architectural_principle)= - -## Architectural principle - -Fundamental guidelines that inform the design, decision making and implementation of a TRE. -These principles provide a framework to ensure that the design of the underlying components of a TRE are aligned to consistent goals, values and best practices. - -(def_business_process)= - -## Business process - -A set of actions which produce a specific desired outcome. -For example: to access the TRE a researcher needs to complete an onboarding business process. - -(def_capability)= - -## Capability - -An ability that a system possesses. -Capabilities are typically expressed in general and high-level terms and typically require a combination of organisation, people, processes, and technology to achieve. - -(def_capability_decomposition)= - -## Capability decomposition - -A set of components that realise a capability. -These components will vary depending on the nature of the capability. -Business-focused capabilities will be realised by business processes, roles and services. -Technology-focused capabilities will be realised by applications, application services and interfaces. -In addition to the components realising the capability, a catalogue of standards, frameworks and controls linked to the capabilities will provide guidance on how to implement the capabilities safely. - -(def_component)= - -## Component - -Definition of a component - -(def_data_object)= - -## Data Object - -A store of data or information. -For example: to know what data is stored within the TRE a study database data object is needed. -This contains information on the data assets within the TRE, who owns them and other compliance information. - - -(def_role)= - -## Role - -Definition of a role - -(def_specification_pillar)= - -## Specification pillar - -A specification pillar is a group of related capabilities. -SATRE has four specification pillars: Information governance, Computing technology, Data management and Supporting Capabilities. - -(def_tre_organisation)= - -## TRE organisation - -A TRE organisation is the set of people, processes and technology that operate and use a particular TRE. +```{glossary} +Actor + Definition of an actor + +Application component + An encapsulation of application functionality which is modular and replaceable. + For example: to perform work within a TRE a researcher might need access to a Desktop or command line interface application component. + +Application service + Definition of a application service + +Architectural principle + Fundamental guidelines that inform the design, decision making and implementation of a TRE. + These principles provide a framework to ensure that the design of the underlying components of a TRE are aligned to consistent goals, values and best practices. + +Business process + A set of actions which produce a specific desired outcome. + For example: to access the TRE a researcher needs to complete an onboarding business process. + +Capability + An ability that a system possesses. + Capabilities are typically expressed in general and high-level terms and typically require a combination of organisation, people, processes, and technology to achieve. + +Capability decomposition + A set of {term}`components ` that realise a capability. + These components will vary depending on the nature of the capability. + Business-focused capabilities will be realised by business processes, roles and services. + Technology-focused capabilities will be realised by applications, application services and interfaces. + In addition to the components realising the capability, a catalogue of standards, frameworks and controls linked to the capabilities will provide guidance on how to implement the capabilities safely. + +Component + The statements concerning processes, controls, practices and applications that make up a {term}`capability `, together with an importance label. + +Data Object + A store of data or information. + For example: to know what data is stored within the TRE a study database data object is needed. + This contains information on the data assets within the TRE, who owns them and other compliance information. + +Role + Definition of a role + +Specification pillar + A specification pillar is a group of related capabilities. + SATRE has four specification pillars: Information governance, Computing technology, Data management and Supporting Capabilities. + +TRE organisation + A TRE organisation is the set of people, processes and technology that operate and use a particular TRE. +``` diff --git a/docs/source/specification.md b/docs/source/specification.md index 6bc63807..e09209e0 100644 --- a/docs/source/specification.md +++ b/docs/source/specification.md @@ -46,18 +46,19 @@ SATRE Specification Architecture ``` {ref}`Architectural Principles ` -: The {ref}`principles ` that all TRE operators looking to use the specification should hold themselves accountable to. +: The {term}`principles ` that all TRE operators looking to use the specification should hold themselves accountable to. {ref}`Specification Pillars ` : The broad areas of TRE provisioning the specification covers. -TRE Capabilities -: The capabilities within these pillars that TRE operators can measure themselves against. +Each pillar is broken down into several {term}`TRE Capabilities `. -TRE Capability Components -: The statements concerning processes, controls, practices and applications that make up a capability, together with an importance label. +Each capability consists of one or more {term}`TRE Capability Components `. -In addition, we also describe some {ref}`roles ` that are necessary for the operation and use of a TRE. +Together, these provide a framework that TRE operators can measure themselves against. + +{ref}`Roles ` +: In addition, we also describe some {term}`roles ` that are necessary for the operation and use of a TRE. (satre_principles)= From 81621d6df2fad17bde5c723af9ce0c5f81d7b198 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Thu, 10 Aug 2023 09:52:14 +0100 Subject: [PATCH 29/29] :memo: Add missing definitions --- docs/source/glossary.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/source/glossary.md b/docs/source/glossary.md index d78d3e75..c49a96da 100644 --- a/docs/source/glossary.md +++ b/docs/source/glossary.md @@ -2,14 +2,12 @@ ```{glossary} Actor - Definition of an actor + A person, organization, or system that has one or more roles that initiates or interacts with activities. + Example: _The SATRE architecture needs actors such as researchers and internal auditors._ Application component An encapsulation of application functionality which is modular and replaceable. - For example: to perform work within a TRE a researcher might need access to a Desktop or command line interface application component. - -Application service - Definition of a application service + Example: _To perform work within a TRE a researcher might need access to a Desktop or command line interface application component._ Architectural principle Fundamental guidelines that inform the design, decision making and implementation of a TRE. @@ -17,17 +15,18 @@ Architectural principle Business process A set of actions which produce a specific desired outcome. - For example: to access the TRE a researcher needs to complete an onboarding business process. + Example: _to access the TRE a researcher needs to complete an onboarding business process._ Capability An ability that a system possesses. - Capabilities are typically expressed in general and high-level terms and typically require a combination of organisation, people, processes, and technology to achieve. + Capabilities are typically expressed in general and high-level terms. + Achieving a capability typically requires a combination of organisation, people, processes, and technology. Capability decomposition A set of {term}`components ` that realise a capability. These components will vary depending on the nature of the capability. - Business-focused capabilities will be realised by business processes, roles and services. - Technology-focused capabilities will be realised by applications, application services and interfaces. + Business-focused capabilities will be realised by {term}`business processes `, {term}`roles ` and services. + Technology-focused capabilities will be realised by {term}`applications `, services and interfaces. In addition to the components realising the capability, a catalogue of standards, frameworks and controls linked to the capabilities will provide guidance on how to implement the capabilities safely. Component @@ -39,7 +38,8 @@ Data Object This contains information on the data assets within the TRE, who owns them and other compliance information. Role - Definition of a role + A role is a set of connected behaviors, rights, obligations and norms within a TRE system. + Roles are occupied by individuals, who are called {term}`actors `. Specification pillar A specification pillar is a group of related capabilities.