Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

AKS Baseline regulated - Bicep migration - Networking files #63

Merged
merged 23 commits into from
Sep 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
aa489c3
Hubs and Spokes - Bicep migration
magrande Apr 25, 2022
b827d02
Bug fixes
magrande Apr 29, 2022
00b95ad
Comments added, and existing resources moved to the right section
magrande Apr 29, 2022
26442dc
[deploy steps] prescribe bit more precise the regulatory compliance s…
ferantivero Aug 24, 2022
e1689bc
[hubv1] remove reference usage for getting img builder snet address p…
ferantivero Aug 24, 2022
e8b3570
[hubv1] disable no hardcoded url warn
ferantivero Aug 24, 2022
91dc537
[hubv1] fix sintax issue
ferantivero Aug 24, 2022
45560fe
[spokev1] fix policy deinition names to avoid not founds
ferantivero Aug 24, 2022
e8f2459
[hubv2] remove references usage for getting snet address prefixes
ferantivero Aug 24, 2022
c75a484
[hubv2] disable no hardcoded url warn
ferantivero Aug 24, 2022
4904a06
[hubv2] fix sintax issue
ferantivero Aug 24, 2022
9e0faa4
Address PR Feedback: keep ARM naming convetion
ferantivero Sep 2, 2022
7be26b8
Address PR Feedback: restore uniqueifying sufix for la
ferantivero Sep 2, 2022
8c15583
Address PR Feedback: add zones to bastion pip
ferantivero Sep 2, 2022
b818451
Address PR Feedback: uniqueness wo/ depedency
ferantivero Sep 6, 2022
64609b2
Address PR Feedback: restore Network.DNS.EnableProy
ferantivero Sep 6, 2022
4ec6009
Address PR Feedback: use an existing resource for reference the snet
ferantivero Sep 13, 2022
4213dd6
Address PR Feedback: typos
ferantivero Sep 13, 2022
afea6e0
Address PR Feedback: change vnet link description
ferantivero Sep 13, 2022
a169d8f
Address PR Feedback: add zone to az appgw
ferantivero Sep 13, 2022
c94e849
Address PR Feedback: add dependency to builder-hub peering
ferantivero Sep 13, 2022
0b9fd70
Address PR Feedback: reverse dependency
ferantivero Sep 13, 2022
c6f169b
bug fix: la name generation
ferantivero Sep 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions docs/deploy/04-subscription.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,9 +118,8 @@ It is recommended that your Azure _subscription_ have the **Azure Security Bench

1. Open the [**Regulatory Compliance** screen in Microsoft Defender for Cloud](https://portal.azure.com/#blade/Microsoft_Azure_Security/SecurityMenuBlade/22)
1. Click on **Manage compliance policies**
1. Click on your subscription
1. Ensure the **Azure Security Benchmark** is applied as the **Microsoft Defender for Cloud default policy**.
1. You'll want to ensure all relevant standards (e.g. **PCI DSS 3.2.1**) are **Enabled** under **Industry & regulatory standards**
1. Expand the tree, and click on your subscription
1. Navigate to **Security policy** blade since you'll want to ensure the **Azure Security Benchmark** and all relevant standards (e.g. **PCI DSS 3.2.1**) are **Enabled** under **Industry & regulatory standards**
1. The **Regulatory Compliance** dashboard in Microsoft Defender for Cloud might take an hour or two to reflect any modifications you've made.

**None of the above is required for this walkthrough**, but we want to ensure you're aware of these subscription-level policies and how you can enable them for your final implementation. All subscriptions containing PCI workloads should have the **PCI DSS 3.2.1** Industry & regulatory standards reports enabled, which _requires_ that the **Azure Security Benchmark** is applied as the default policy.
Expand Down
4 changes: 2 additions & 2 deletions docs/deploy/05-networking-hub.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Egressing your spoke traffic through a hub network (following the hub-spoke mode

After executing these steps you'll have the `rg-enterprise-networking-hubs` resource group populated with a regional virtual network (vnet), Azure Firewall, Azure Bastion, and Azure Monitor & Flow Log storage for network observability. No spokes will have been created yet, so the default firewall rules are maximally restrictive, as there is no expected outflow of traffic so none is allowed. We'll open up access on an as-needed bases throughout this walk through.

Specifically, you'll see networking/hub-region.v​_n_.json referenced a couple times in this process. Think of this as an evolution of a _single_ ARM template as the number and needs of the connected spokes change over time. You can diff the v​_n_ and v​_n+1_ ARM templates to see this progression over time. Typically your network team would have encapsulated this hub in a file named something like `hub-eastus2.json` and updated it in their source control as dependencies/requirements dictate. It likely would have not taken as many parameters as either, as those would be constants that could be easily defined directly in the template as the file would be specific to the region's spokes. To keep this reference implementation more flexible on what region you select, you'll be asked to provide deployment parameters and the filename can remain the generic name of hub-​_region_.
Specifically, you'll see networking/hub-region.v​_n_.bicep referenced a couple times in this process. Think of this as an evolution of a _single_ ARM template as the number and needs of the connected spokes change over time. You can diff the v​_n_ and v​_n+1_ ARM templates to see this progression over time. Typically your network team would have encapsulated this hub in a file named something like `hub-eastus2.bicep` and updated it in their source control as dependencies/requirements dictate. It likely would have not taken as many parameters as either, as those would be constants that could be easily defined directly in the template as the file would be specific to the region's spokes. To keep this reference implementation more flexible on what region you select, you'll be asked to provide deployment parameters and the filename can remain the generic name of hub-​_region_.

The examples that follow use `eastus2` as the primary region. You're welcome to change this in the ARM template parameters throughout this walkthrough. Clusters are regional resources; and the expectation is that your regional hub, regional spoke, and regional workload are all sharing the same region. So if you make a change to the region, be sure you change it in _all_ places along the way. For a reference architecture of a general-purpose, multi-region cluster, see [AKS Baseline for Multi-Region Topology](https://github.com/mspnp/aks-baseline-multi-region).

Expand All @@ -29,7 +29,7 @@ Since this walkthrough is expected to be deployed isolated from existing infrast

```bash
# [This takes about eight minutes to run.]
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v0.json -p location=eastus2
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v0.bicep -p location=eastus2
```

The hub deployment will output the following:
Expand Down
8 changes: 4 additions & 4 deletions docs/deploy/06-aks-jumpboximage.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,20 +40,20 @@ You are going to be using Azure Image Builder to generate a Kubernetes-specific
RESOURCEID_VNET_HUB=$(az deployment group show -g rg-enterprise-networking-hubs -n hub-region.v0 --query properties.outputs.hubVnetId.value -o tsv)

# [This takes about one minute to run.]
az deployment group create -g rg-enterprise-networking-spokes -f networking/spoke-BU0001A0005-00.json -p location=eastus2 hubVnetResourceId="${RESOURCEID_VNET_HUB}"
az deployment group create -g rg-enterprise-networking-spokes -f networking/spoke-BU0001A0005-00.bicep -p location=eastus2 hubVnetResourceId="${RESOURCEID_VNET_HUB}"
```

1. Update the regional hub deployment to account for the requirements of the spoke.

Now that the first spoke network is created, the hub network's firewall needs to be updated to support the Azure Image Builder process that will execute in there. The hub firewall does NOT have any default permissive egress rules, and as such, each needed egress endpoint needs to be specifically allowed. This deployment builds on the prior with the added allowances in the firewall.

> :eyes: If you're curious to see what changed in the regional hub, [view the diff](https://diffviewer.azureedge.net/?l=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v0.json&r=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v1.json).
> :eyes: If you're curious to see what changed in the regional hub, [view the diff](https://diffviewer.azureedge.net/?l=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v0.bicep&r=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v1.bicep).

```bash
RESOURCEID_SUBNET_AIB=$(az deployment group show -g rg-enterprise-networking-spokes -n spoke-BU0001A0005-00 --query properties.outputs.imageBuilderSubnetResourceId.value -o tsv)

# [This takes about five minutes to run.]
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v1.json -p location=eastus2 aksImageBuilderSubnetResourceId="${RESOURCEID_SUBNET_AIB}"
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v1.bicep -p location=eastus2 aksImageBuilderSubnetResourceId="${RESOURCEID_SUBNET_AIB}"
```

### Build and deploy the jump box image
Expand All @@ -62,7 +62,7 @@ Now that we have our image building network created, egressing through our hub,

1. Download the ARM templates from the AKS Jump Box Image Builder repository.

Ideally core templates like this would be part of your private Bicep registry. For this walk through, we are simply downloading the remote Bicep templates locally for execution.
Ideally core templates like this would be part of your private Bicep registry. For this walk through, we are simply downloading the remote ARM templates locally for execution.

```bash
wget -B https://raw.githubusercontent.com/mspnp/aks-jumpbox-imagebuilder/main/ -x -nH --cut-dirs=3 -i jumpbox/jumpbox-bicep.txt -P jumpbox
Expand Down
6 changes: 3 additions & 3 deletions docs/deploy/08-cluster-networking.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,22 +23,22 @@ Your `rg-enterprise-networking-spokes` will be populated with the dedicated regi
RESOURCEID_VNET_HUB=$(az deployment group show -g rg-enterprise-networking-hubs -n hub-region.v0 --query properties.outputs.hubVnetId.value -o tsv)

# [This takes about five minutes to run.]
az deployment group create -g rg-enterprise-networking-spokes -f networking/spoke-BU0001A0005-01.json -p location=eastus2 hubVnetResourceId="${RESOURCEID_VNET_HUB}"
az deployment group create -g rg-enterprise-networking-spokes -f networking/spoke-BU0001A0005-01.bicep -p location=eastus2 hubVnetResourceId="${RESOURCEID_VNET_HUB}"
```

1. Update the regional hub deployment to account for the runtime requirements of the virtual network.

This is an evolution of same hub template you used before, but now updated with Azure Firewall rules specific to this AKS cluster infrastructure.

> :eyes: If you're curious to see what changed in the regional hub, [view the diff](https://diffviewer.azureedge.net/?l=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v1.json&r=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v2.json).
> :eyes: If you're curious to see what changed in the regional hub, [view the diff](https://diffviewer.azureedge.net/?l=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v1.bicep&r=https://raw.githubusercontent.com/mspnp/aks-baseline-regulated/main/networking/hub-region.v2.bicep).

```bash
RESOURCEID_SUBNET_AIB=$(az deployment group show -g rg-enterprise-networking-spokes -n spoke-BU0001A0005-00 --query properties.outputs.imageBuilderSubnetResourceId.value -o tsv)
RESOURCEID_SUBNET_NODEPOOLS="['$(az deployment group show -g rg-enterprise-networking-spokes -n spoke-BU0001A0005-01 --query "properties.outputs.nodepoolSubnetResourceIds.value | join ('\',\'',@)" -o tsv)']"
RESOURCEID_SUBNET_JUMPBOX=$(az deployment group show -g rg-enterprise-networking-spokes -n spoke-BU0001A0005-01 --query properties.outputs.jumpboxSubnetResourceId.value -o tsv)

# [This takes about seven minutes to run.]
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v2.json -p location=eastus2 aksImageBuilderSubnetResourceId="${RESOURCEID_SUBNET_AIB}" nodepoolSubnetResourceIds="${RESOURCEID_SUBNET_NODEPOOLS}" aksJumpboxSubnetResourceId="${RESOURCEID_SUBNET_JUMPBOX}"
az deployment group create -g rg-enterprise-networking-hubs -f networking/hub-region.v2.bicep -p location=eastus2 aksImageBuilderSubnetResourceId="${RESOURCEID_SUBNET_AIB}" nodepoolSubnetResourceIds="${RESOURCEID_SUBNET_NODEPOOLS}" aksJumpboxSubnetResourceId="${RESOURCEID_SUBNET_JUMPBOX}"
```

### Next step
Expand Down
4 changes: 2 additions & 2 deletions modules/subscriptionPolicyAssignment.bicep
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ param enforcementMode string = 'Default'
@minLength(4)
param location string

@description('The name of the policy set to assign.')
@description('The name of the policy or policy set to assign.')
@minLength(36)
@maxLength(36)
param policyDefinitionSetName string
Expand All @@ -33,7 +33,7 @@ var builtIntPolicyDefinitionSetId = subscriptionResourceId('Microsoft.Authorizat

@description('Assignment of policy')
resource policyAssignment 'Microsoft.Authorization/policyAssignments@2021-06-01' = {
name: guid(builtIntPolicyDefinitionSetId)
name: guid(builtIntPolicyDefinitionSetId, subscription().id)
identity: {
type: 'SystemAssigned'
}
Expand Down
10 changes: 5 additions & 5 deletions networking/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ This is part of the Azure Kubernetes Service (AKS) Baseline Cluster for Regulate

## Files

* [`hub-region.v0.json`](./hub-region.v0.json) is a file that defines a generic regional hub. All regional hubs can generally be considered a fork of this base template.
* [`hub-region.v1.json`](./hub-region.v1.json) is an updated version that defines a specific region's hub (for example, it might be named `hub-eastus2.json`). This is the long-lived template that defines this specific region's hub. This version has support for our image builder process.
* [`hub-region.v2.json`](./hub-region.v2.json) is an even more updated version that defines a specific region's hub (it would still be the same named `hub-eastus2.json` file). This version has support for our image builder process plus our AKS cluster's needs.
* [`spoke-BU0001A0005-00.json`](./spoke-BU0001A0005-00.json) is a file that defines a specific spoke in the topology. A spoke is created for each workload in a business unit, hence the naming pattern in the file name. This spoke contains the networking resources for the image builder process.
* [`spoke-BU0001A0005-01.json`](./spoke-BU0001A0005-01.json) is a file that defines a specific spoke in the topology. This spoke contains the networking resources for the AKS cluster.
* [`hub-region.v0.bicep`](./hub-region.v0.bicep) is a file that defines a generic regional hub. All regional hubs can generally be considered a fork of this base template.
* [`hub-region.v1.bicep`](./hub-region.v1.bicep) is an updated version that defines a specific region's hub (for example, it might be named `hub-eastus2.bicep`). This is the long-lived template that defines this specific region's hub. This version has support for our image builder process.
* [`hub-region.v2.bicep`](./hub-region.v2.bicep) is an even more updated version that defines a specific region's hub (it would still be the same named `hub-eastus2.bicep` file). This version has support for our image builder process plus our AKS cluster's needs.
* [`spoke-BU0001A0005-00.bicep`](./spoke-BU0001A0005-00.bicep) is a file that defines a specific spoke in the topology. A spoke is created for each workload in a business unit, hence the naming pattern in the file name. This spoke contains the networking resources for the image builder process.
* [`spoke-BU0001A0005-01.bicep`](./spoke-BU0001A0005-01.bicep) is a file that defines a specific spoke in the topology. This spoke contains the networking resources for the AKS cluster.

Your organization will likely have its own standards for their hub-spoke or vwan implementation. Be sure to follow your organizational guidelines.

Expand Down
Loading