Feb 14, 2024

Securing K8s Ingress Gateway Resource with ATLAS Certificate Manager Plugin

What is ingress?

Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster.
Traffic routing is controlled by rules defined on the Ingress resource.

Here is a simple example where an Ingress sends all its traffic to one service:                                                 

Prerequisites:

1. Unzip $sudo apt install unzip
2. AWS CLI
3. $curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"|
   
   unzip awscliv2.zip
   sudo ./aws/install
4. Create an S3 bucket.
5. Create one hosted zone from Route 53. (The name of the zone is either the same as the bucket name or should be the succeeding of the bucket name. For example, if the bucket name is example.com, the hosted zone name should be abc.example.com.)
6. Helm installation from the script.
   
   $curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
   $chmod 700 get_helm.sh
   $./get_helm.sh

There are a few policies required to run the cluster successfully, which should be assigned to a user and a role for the cluster:

#IAM URL: https://console.aws.amazon.com/iamv2/home#/home

#Create a user for the k8s cluster and assign the below policies:

1. Go to Users.
2. Then, add users.
3. Give a name to the user.
4. Select Access key: Programmatic Access
5. Permissions
6. Select Attach existing policies directly, and select the below policies for this user.

              a) VPC full access

              b) EC2 full access

              c) S3 full access

              d) Route53 full access

              e) IAM full access

#Create a role for the k8s cluster and assign the below policies:

1. Go to the IAM dashboard https://console.aws.amazon.com/iamv2/home#/home
2. Roles
3. Create role
4. Form the common use case, select EC2 and go to the next screen for permissions.
5. Select the below permissions for this user

              a) VPC full access

              b) EC2 full access

              c) S3 full access

              d) Route53 full access

              e) IAM full access

#Launch an Ec2-Instance and assign this role to the instance.

Installing kubectl and kops

kubectl:

#Download the latest release with the command:

$curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

#make the downloaded file executable

$chmod +x kubectl

#Move the executable to the /usr/local/bin

$sudo mv kubectl /usr/local/bin

kops:

#Download the latest release with the command:

$curl -LO https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-linux-amd64

#Make the binary executable

$chmod +x kops-linux-amd64

#Move the executable to /usr/local/bin

$sudo mv kops-linux-amd64 /usr/local/bin/kops

In order to work further need to login with the above created programmatic user

$aws configure

#enter the Access key ID and Secret access key.

#Provide the region details, i.e., US-East-1 or any other

#Give output format as "json".

#Create a S3 bucket through the S3 bucket console: https://s3.console.aws.amazon.com/s3/home?region=us-east-1

#Create a private hosted zone from the Route53 console: https://console.aws.amazon.com/route53/v2/home#Dashboard

#Generate public and private keys.

$ssh-keygen

#Everything is setup. Now let's begin creating the cluster with the below commands.

$export KOPS_STATE_STORE="s3://pki.atlasqa.co.uk"
$export MASTER_SIZE=${MASTER_SIZE:-m4.large}
$export NODE_SIZE=${NODE_SIZE:-m4.large}
$export ZONES="us-east-1a,us-east-1b,us-east-1c"

$kops create cluster pki.atlasqa.co.uk --node-count 3 --zones $ZONES --node-size $NODE_SIZE --master-size $MASTER_SIZE --master-zones $ZONES --dns public --dns-zone pki.atlasqa.co.uk --cloud aws

#It will describe everything that it will create within the cluster. In the next step, kops will update the cluster and resources will be created.

$kops update cluster --name pki.atlasqa.co.uk --yes --admin

#It will take around 20 minutes to get all the resources ready within the cluster.

#Check the cluster status after 20 minutes with the below command.

$kops validate cluster --name pki.atlasqa.co.uk

 

The cluster is setup; there are 3 node machines running in us-east-1 region and 3 master running in us-east-1 as per the availability zones.

• Install cert-manager

#Add the Jetstack Helm repository.

$helm repo add jetstack https://charts.jetstack.io

#Update your local Helm chart repository cache.

$helm repo update

#Install the CustomResourceDefinition resources separately.

$kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.8.0/cert-manager.crds.yaml

#Install the cert-manager using helm.

$helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.8.0

#Next, install the Atlas controller and CRDs:'

$kubectl apply -f https://github.com/globalsign/atlas-cert-manager/releases/download/v0.0.1/install.yaml

The controller is deployed and ready to handle Atlas requests.

#Label the cert-manager namespace to disable resource validation.

$kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true

• Install Nginx-ingress-controller in namespace cert-manager and create A record for your DNS Zone.

$helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace cert-manager

• Add an A record to your DNS zone.

$kubectl get svc -n cert-manager

Step 1. Set up ATLAS account as a Secret Within Kubernetes.

Create a secret to store the ATLAS account api_key, secrets along with the mTLS and private key:

kubectl create secret generic issuer-credentials --from-literal=apikey=$API_KEY --from-literal=apisecret=$API_SECRET --from-literal=cert="$(cat mTLS.pem)" --from-literal=certkey="$(cat privatekey.pem)" -n cert-manager

Step 2. Set up the GlobalSign as the issuer/ClusterIssuer.

$kubectl apply -f issuer.yaml

issuer.yaml
apiVersion: hvca.globalsign.com/v1alpha1
kind: Issuer
metadata:
  name: gs-issuer
  namespace: cert-manager
spec:
  authSecretName: "issuer-credentials"
  url: "https://emea.api.hvca.globalsign.com:8443/v2"

Step 3. Create a certificate resource.

$kubectl apply -f cert.yaml

cert.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: pki.atlasqa.co.uk
  namespace: cert-manager
spec:
  # Secret names are always required.
  secretName: www.atlasqa.co.uk
 
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  subject:
  #  organizations:
  #    - jetstack
  # The use of the common name field has been deprecated since 2000 and is
  # discouraged from being used.
  commonName: pki.atlasqa.co.uk
  isCA: false
  privateKey:
    algorithm: RSA
    encoding: PKCS1
    size: 2048
  usages:
    - server auth
    #- client auth
  # At least one of a DNS Name, URI, or IP address is required.
#  dnsNames:
#    -
#www.atlasqa.co.uk
  # Issuer references are always required.
  issuerRef:
    name: gs-issuer
    # We can reference ClusterIssuers by changing the kind here.
    # The default value is Issuer (i.e. a locally namespaced Issuer)
    kind: Issuer
    # This is optional since cert-manager will default to this value however
    # if you are using an external issuer, change this to that issuer group.
    group: hvca.globalsign.com

Step 4. Securing ingress resource.

$kubectl apply -f ingress.yaml

ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  namespace: cert-manager
  annotations:
    cert-manager.io/issuer: GS-issuer
    kubernetes.io/ingress.class: nginx
spec:
  tls:
    - hosts:
        - pki.atlasqa.co.uk
      secretName: www.atlasqa.co.uk
  rules:
    - host: pki.atlasqa.co.uk
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: example-service
                port:
                  number: 80