EKS: Default Control Plane Security Group Doesn't Create Ingress Rules

[hakenmt]

Describe the bug

By default, an EKS Cluster does not create ingress rules for the security group applied to the control plane. This seems unintuitive as allowing inbound 443 from worker nodes feels like a sane default requirement for anything to work.

Expected Behavior

An ingress rule allowing 443 inbound from the VPC or worker node security group or some warning emitted that the security group has not been configured with inbound access.

Current Behavior

No ingress rule or warning emitted.

Reproduction Steps

Just create an EKS cluster.

Possible Solution

Automatically add an ingress rule or emit a warning during synth if the user has not created ingress.

Additional Information/Context

No response

CDK CLI Version

2.138.0

Framework Version

No response

Node.js Version

v20.9.0

OS

darwin

Language

.NET

Language Version

No response

Other information

No response

[pahud]

The cluster would share the same SG with the managed nodegroups and if you check that SG, it essentially allows ingress from the same SG as the source so I think it won't need additional ingress rule like that?

[hakenmt]

I don't see an ingress rule being created that allows from the SG itself, by default I don't see any ingress rule created at all.

[pahud]

If you create the eks.Cluster like this:

    const cluster = new eks.Cluster(scope, 'EksCluster', {
        version: eks.KubernetesVersion.V1_30,
        kubectlLayer: new KubectlLayer(scope, 'KubectlLayer'),
        mastersRole,
        vpc: ec2.Vpc.fromLookup(scope, 'Vpc', { isDefault: true }),
        defaultCapacity: defaultCapacity ?? 0,
        vpcSubnets: [
            { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS }
        ]
    });

After deployment, check the Cluster security group from the EKS console:

There is a Inbound rule that allows all traffic from another SG(sg-0c2b44e0469f0480d in my case)

If you check your managed nogegroup ASG or EC2 instance, that is the SG of the instance.

This means all traffic from the managed node is allowed to the control plane. You don't need to explicitly create an additional ingress for TCP 443.

[hakenmt]

Is this also the case when you set VpcSubnets = new SubnetSelection[] { new SubnetSelection() { SubnetType = SubnetType.PRIVATE_ISOLATED } }, ?

[pahud]

@hakenmt

It should be. Can you verify and let us know if it isn't?

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[hakenmt]

Create a cluster like this:

Cluster testcluster = new Cluster(new NestedStack(this, "EKS2Stack"), "TESTEKSCluster", new ClusterProps(){
                Vpc = this.NetworkStack.Vpc,
                VpcSubnets = new SubnetSelection[] { new SubnetSelection() { SubnetType = SubnetType.PRIVATE_ISOLATED } },
                DefaultCapacity = 0,
                Version =  KubernetesVersion.Of("1.30"),
                PlaceClusterHandlerInVpc = false
            });

And the resulting CFN looks like this:

{
 "Resources": {
  "TESTEKSClusterKubectlHandlerRole2A207518": {
   "Type": "AWS::IAM::Role",
   "Properties": {
    "AssumeRolePolicyDocument": {
     "Statement": [
      {
       "Action": "sts:AssumeRole",
       "Effect": "Allow",
       "Principal": {
        "Service": "lambda.amazonaws.com"
       }
      }
     ],
     "Version": "2012-10-17"
    },
    "ManagedPolicyArns": [
     {
      "Fn::Join": [
       "",
       [
        "arn:",
        {
         "Ref": "AWS::Partition"
        },
        ":iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
       ]
      ]
     },
     {
      "Fn::Join": [
       "",
       [
        "arn:",
        {
         "Ref": "AWS::Partition"
        },
        ":iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
       ]
      ]
     },
     {
      "Fn::Join": [
       "",
       [
        "arn:",
        {
         "Ref": "AWS::Partition"
        },
        ":iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
       ]
      ]
     },
     {
      "Fn::If": [
       "TESTEKSClusterHasEcrPublicC8F18CD5",
       {
        "Fn::Join": [
         "",
         [
          "arn:",
          {
           "Ref": "AWS::Partition"
          },
          ":iam::aws:policy/AmazonElasticContainerRegistryPublicReadOnly"
         ]
        ]
       },
       {
        "Ref": "AWS::NoValue"
       }
      ]
     }
    ]
   },
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/KubectlHandlerRole/Resource"
   }
  },
  "TESTEKSClusterKubectlHandlerRoleDefaultPolicyAA373831": {
   "Type": "AWS::IAM::Policy",
   "Properties": {
    "PolicyDocument": {
     "Statement": [
      {
       "Action": "eks:DescribeCluster",
       "Effect": "Allow",
       "Resource": {
        "Fn::GetAtt": [
         "TESTEKSClusterE78440B5",
         "Arn"
        ]
       }
      },
      {
       "Action": "sts:AssumeRole",
       "Effect": "Allow",
       "Resource": {
        "Fn::GetAtt": [
         "TESTEKSClusterCreationRole85D7122D",
         "Arn"
        ]
       }
      }
     ],
     "Version": "2012-10-17"
    },
    "PolicyName": "TESTEKSClusterKubectlHandlerRoleDefaultPolicyAA373831",
    "Roles": [
     {
      "Ref": "TESTEKSClusterKubectlHandlerRole2A207518"
     }
    ]
   },
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/KubectlHandlerRole/DefaultPolicy/Resource"
   }
  },
  "TESTEKSClusterRole1DF8E73F": {
   "Type": "AWS::IAM::Role",
   "Properties": {
    "AssumeRolePolicyDocument": {
     "Statement": [
      {
       "Action": "sts:AssumeRole",
       "Effect": "Allow",
       "Principal": {
        "Service": "eks.amazonaws.com"
       }
      }
     ],
     "Version": "2012-10-17"
    },
    "ManagedPolicyArns": [
     {
      "Fn::Join": [
       "",
       [
        "arn:",
        {
         "Ref": "AWS::Partition"
        },
        ":iam::aws:policy/AmazonEKSClusterPolicy"
       ]
      ]
     }
    ]
   },
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/Role/Resource"
   }
  },
  "TESTEKSClusterControlPlaneSecurityGroup15BB2198": {
   "Type": "AWS::EC2::SecurityGroup",
   "Properties": {
    "GroupDescription": "EKS Control Plane Security Group",
    "SecurityGroupEgress": [
     {
      "CidrIp": "0.0.0.0/0",
      "Description": "Allow all outbound traffic by default",
      "IpProtocol": "-1"
     }
    ],
    "VpcId": {
     "Ref": "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpc127ECD75Ref"
    }
   },
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/ControlPlaneSecurityGroup/Resource"
   }
  },
  "TESTEKSClusterCreationRole85D7122D": {
   "Type": "AWS::IAM::Role",
   "Properties": {
    "AssumeRolePolicyDocument": {
     "Statement": [
      {
       "Action": "sts:AssumeRole",
       "Effect": "Allow",
       "Principal": {
        "AWS": [
         {
          "Fn::GetAtt": [
           "TESTEKSClusterKubectlHandlerRole2A207518",
           "Arn"
          ]
         },
         {
          "Fn::GetAtt": [
           "awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454",
           "Outputs.multiazworkshopEKS2StackawscdkawseksClusterResourceProviderIsCompleteHandlerServiceRole7C95EC47Arn"
          ]
         },
         {
          "Fn::GetAtt": [
           "awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454",
           "Outputs.multiazworkshopEKS2StackawscdkawseksClusterResourceProviderOnEventHandlerServiceRoleEE000176Arn"
          ]
         }
        ]
       }
      }
     ],
     "Version": "2012-10-17"
    }
   },
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/Resource/CreationRole/Resource"
   }
  },
  "TESTEKSClusterCreationRoleDefaultPolicy5C09B2CC": {
   "Type": "AWS::IAM::Policy",
   "Properties": {
    "PolicyDocument": {
     "Statement": [
      {
       "Action": "iam:PassRole",
       "Effect": "Allow",
       "Resource": {
        "Fn::GetAtt": [
         "TESTEKSClusterRole1DF8E73F",
         "Arn"
        ]
       }
      },
      {
       "Action": [
        "eks:CreateCluster",
        "eks:CreateFargateProfile",
        "eks:DeleteCluster",
        "eks:DescribeCluster",
        "eks:DescribeUpdate",
        "eks:TagResource",
        "eks:UntagResource",
        "eks:UpdateClusterConfig",
        "eks:UpdateClusterVersion"
       ],
       "Effect": "Allow",
       "Resource": "*"
      },
      {
       "Action": [
        "eks:DeleteFargateProfile",
        "eks:DescribeFargateProfile"
       ],
       "Effect": "Allow",
       "Resource": "*"
      },
      {
       "Action": [
        "ec2:DescribeDhcpOptions",
        "ec2:DescribeInstances",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs",
        "iam:CreateServiceLinkedRole",
        "iam:GetRole",
        "iam:listAttachedRolePolicies"
       ],
       "Effect": "Allow",
       "Resource": "*"
      }
     ],
     "Version": "2012-10-17"
    },
    "PolicyName": "TESTEKSClusterCreationRoleDefaultPolicy5C09B2CC",
    "Roles": [
     {
      "Ref": "TESTEKSClusterCreationRole85D7122D"
     }
    ]
   },
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/Resource/CreationRole/DefaultPolicy/Resource"
   }
  },
  "TESTEKSClusterE78440B5": {
   "Type": "Custom::AWSCDK-EKS-Cluster",
   "Properties": {
    "ServiceToken": {
     "Fn::GetAtt": [
      "awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454",
      "Outputs.multiazworkshopEKS2StackawscdkawseksClusterResourceProviderframeworkonEventAF807DB4Arn"
     ]
    },
    "Config": {
     "version": "1.30",
     "roleArn": {
      "Fn::GetAtt": [
       "TESTEKSClusterRole1DF8E73F",
       "Arn"
      ]
     },
     "kubernetesNetworkConfig": {
      "ipFamily": "ipv4"
     },
     "resourcesVpcConfig": {
      "subnetIds": [
       {
        "Ref": "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet1Subnet4AAB0A9ARef"
       },
       {
        "Ref": "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet2Subnet3E55198ARef"
       },
       {
        "Ref": "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet3Subnet08F609EBRef"
       }
      ],
      "securityGroupIds": [
       {
        "Fn::GetAtt": [
         "TESTEKSClusterControlPlaneSecurityGroup15BB2198",
         "GroupId"
        ]
       }
      ],
      "endpointPublicAccess": true,
      "endpointPrivateAccess": true
     }
    },
    "AssumeRoleArn": {
     "Fn::GetAtt": [
      "TESTEKSClusterCreationRole85D7122D",
      "Arn"
     ]
    },
    "AttributesRevision": 2
   },
   "DependsOn": [
    "TESTEKSClusterCreationRoleDefaultPolicy5C09B2CC",
    "TESTEKSClusterCreationRole85D7122D"
   ],
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/Resource/Resource/Default"
   }
  },
  "TESTEKSClusterKubectlReadyBarrier7BB2051A": {
   "Type": "AWS::SSM::Parameter",
   "Properties": {
    "Type": "String",
    "Value": "aws:cdk:eks:kubectl-ready"
   },
   "DependsOn": [
    "TESTEKSClusterCreationRoleDefaultPolicy5C09B2CC",
    "TESTEKSClusterCreationRole85D7122D",
    "TESTEKSClusterE78440B5"
   ],
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/KubectlReadyBarrier"
   }
  },
  "awscdkawseksClusterResourceProviderNestedStackawscdkawseksClusterResourceProviderNestedStackResource9827C454": {
   "Type": "AWS::CloudFormation::Stack",
   "Properties": {
    "TemplateURL": {
     "Fn::Join": [
      "",
      [
       "https://s3.",
       {
        "Fn::Sub": "${AWS::Region}"
       },
       ".",
       {
        "Ref": "AWS::URLSuffix"
       },
       "/",
       {
        "Fn::Sub": "${AssetsBucket}"
       },
       "/",
       {
        "Fn::Sub": "${AssetsBucketPrefix}77c8cc1d139f1f7caf9cdf76c80b4c6da2dc64b419c0c14feb57c491da64ef6a.json"
       }
      ]
     ]
    }
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/@aws-cdk--aws-eks.ClusterResourceProvider.NestedStack/@aws-cdk--aws-eks.ClusterResourceProvider.NestedStackResource",
    "aws:asset:path": "multiazworkshopEKS2StackawscdkawseksClusterResourceProvider2A8E013A.nested.template.json",
    "aws:asset:property": "TemplateURL"
   }
  },
  "awscdkawseksKubectlProviderNestedStackawscdkawseksKubectlProviderNestedStackResourceA7AEBA6B": {
   "Type": "AWS::CloudFormation::Stack",
   "Properties": {
    "Parameters": {
     "referencetomultiazworkshopEKS2StackTESTEKSClusterKubectlHandlerRole2414B900Arn": {
      "Fn::GetAtt": [
       "TESTEKSClusterKubectlHandlerRole2A207518",
       "Arn"
      ]
     },
     "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet1Subnet4AAB0A9ARef": {
      "Ref": "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet1Subnet4AAB0A9ARef"
     },
     "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet2Subnet3E55198ARef": {
      "Ref": "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet2Subnet3E55198ARef"
     },
     "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet3Subnet08F609EBRef": {
      "Ref": "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet3Subnet08F609EBRef"
     },
     "referencetomultiazworkshopEKS2StackTESTEKSCluster4AB6851FClusterSecurityGroupId": {
      "Fn::GetAtt": [
       "TESTEKSClusterE78440B5",
       "ClusterSecurityGroupId"
      ]
     }
    },
    "TemplateURL": {
     "Fn::Join": [
      "",
      [
       "https://s3.",
       {
        "Fn::Sub": "${AWS::Region}"
       },
       ".",
       {
        "Ref": "AWS::URLSuffix"
       },
       "/",
       {
        "Fn::Sub": "${AssetsBucket}"
       },
       "/",
       {
        "Fn::Sub": "${AssetsBucketPrefix}515456f0aaeae9c0db04d592a127d12320ac1dea6b29c7af03f92ccbcb52b349.json"
       }
      ]
     ]
    }
   },
   "DependsOn": [
    "TESTEKSClusterKubectlHandlerRoleDefaultPolicyAA373831",
    "TESTEKSClusterKubectlHandlerRole2A207518"
   ],
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/@aws-cdk--aws-eks.KubectlProvider.NestedStack/@aws-cdk--aws-eks.KubectlProvider.NestedStackResource",
    "aws:asset:path": "multiazworkshopEKS2StackawscdkawseksKubectlProvider44A40627.nested.template.json",
    "aws:asset:property": "TemplateURL"
   }
  }
 },
 "Conditions": {
  "TESTEKSClusterHasEcrPublicC8F18CD5": {
   "Fn::Equals": [
    {
     "Ref": "AWS::Partition"
    },
    "aws"
   ]
  }
 },
 "Parameters": {
  "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpc127ECD75Ref": {
   "Type": "String"
  },
  "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet1Subnet4AAB0A9ARef": {
   "Type": "String"
  },
  "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet2Subnet3E55198ARef": {
   "Type": "String"
  },
  "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpcisolatedsubnetSubnet3Subnet08F609EBRef": {
   "Type": "String"
  }
 }
}

You can see there's only 1 security generated and has no ingress rules:

"TESTEKSClusterControlPlaneSecurityGroup15BB2198": {
   "Type": "AWS::EC2::SecurityGroup",
   "Properties": {
    "GroupDescription": "EKS Control Plane Security Group",
    "SecurityGroupEgress": [
     {
      "CidrIp": "0.0.0.0/0",
      "Description": "Allow all outbound traffic by default",
      "IpProtocol": "-1"
     }
    ],
    "VpcId": {
     "Ref": "referencetomultiazworkshopNetworkNestedStackNetworkNestedStackResourceD557F66AOutputsmultiazworkshopNetworkvpc127ECD75Ref"
    }
   },
   "Metadata": {
    "aws:cdk:path": "multi-az-workshop/EKS2Stack/TESTEKSCluster/ControlPlaneSecurityGroup/Resource"
   }
  },