SSL and DNS setup in Azure
How to setup DNS and terminate the traffic via TLS.
In this section we will see how we can setup DNS using Azure DNS and terminate the ingress traffic using SSL from the certificate generated by cert-manager and Let's encrypt.
If you are using a different DNS provider then almost same changes will reflect in the DNS provider and cert-manager
settings. To perform all the below tasks cert-manager
and tfy-istio-ingress
must be installed already.
Setting up DNS
DNs settings can be done in two ways
Public load balancer
If you want your endpoints to be exposed to the outer world so that they can be accessed from the internet then the load balancer type must be public. For this no change needs to be made in the tfy-istio-ingress
. By default the application will create a public load balancer with public endpoint.
To get the IP of this public endpoint
kubectl get svc -n istio-system tfy-istio-ingress -ojsonpath='{.status.loadBalancer.ingress[0].ip}'
Once this endpoint is available make sure to point your domain name with A
record to the above public IP in your DNS provider.
Private load balancer
There are use cases when endpoints are to be kept private. For this we have to use a private load balancer in Azure. To get a private IP for load balancer
-
Go to Deployments page in the left panel.
-
Select your application name (
tfy-istio-ingress
) from the Helm section and click on Edit from the three dots in the right side. -
Edit the section with the below
gateway.annotations
gateway: annotations: service.beta.kubernetes.io/azure-load-balancer-internal: "true" tfyGateway: name: tfy-wildcard spec: servers: - tls: httpsRedirect: true port: name: http-tfy-wildcard number: 80 protocol: HTTP hosts: - "*.infra.truefoundry.com" - port: name: https-tfy-wildcard number: 443 protocol: HTTP hosts: - "*.infra.truefoundry.com" selector: istio: tfy-istio-ingress
-
Once done you can see your load balancer will have a private IP address
$ k get svc -n istio-system tfy-istio-ingress -ojsonpath='{.status.loadBalancer.ingress[0].ip}' 10.224.0.8
-
Point this private load balancer IP to the domain name in your favourite DNS provider. If you want to use Azure DNS and you don't have a DNS zone created follow the below document.
Setting up hosted Zone in Azure DNS
-
Authenticate to Azure cli
-
Create the DNS hosted zone
$ az network dns zone create \ --name demo2.truefoundry.com \ --resource-group tfy-datascience \ --query nameServers
-
The above command will output list of the name servers that must be pointed in your main DNS resolver as NS record. This is done so that any traffic coming to
demo2.truefoundry.com
(in this case) will come to the hosted zone in Azure DNS. -
To check if you name servers are pointed correctly, run the below command
dig -t NS +short demo2.truefoundry.com
-
Now we will create a record with the load balancer's IP so that request coming to
*.demo2.truefoundry.com
goes to the load balancer's IPexport LOAD_BALANCERIP="<LOADBALANCER IP HERE>" az network dns record-set a add-record \ --ipv4-address $LOAD_BALANCERIP \ --record-set-name "*" \ --resource-group tfy-datascience \ --zone-name demo2.truefoundry.com
-
To test the record
dig -t A something.demo2.truefoundry.com +short
SSL/TLS using cert-manager
Once we have our DNS set up we need to set up the cert-manager
to issue certificates. In this section we will use AzureDNS as the DNS provider to authenticate for DNS challenges that are created by Issuers
in cert-manager
. You can check Cert-manager DNS provider configuration to configure DNS challenges for your preferred DNS provider. Below documentation gives a brief overview on how to use Azure DNS to create DNS challenges and issue certificates.
-
If you don't have workload identity feature enabled for your AKS cluster use the command to enable it. If you have used Azure AKS page to create AKS cluster then it is enabled by default.
az aks update \ --name ${CLUSTER} \ --resource-group ${RESOURCE_GROUP} \ --enable-oidc-issuer \ --enable-workload-identity # ℹ️ This option is currently only available when using the aks-preview extension.
-
Make sure to create a managed identity. If you have used Azure AKS page to create AKS then we have already created a managed identity under the name
tfy-user-identity
.- To create a new managed identity use the below command and save the output Client ID
export IDENTITY_NAME=cert-manager PRINCIPAL_ID=$(az identity create --name "${IDENTITY_NAME}" --query principalId)
- If you have already created the managed identity or want to use an already existing one then run the following command
# As per the installations steps of AKS cluster PRINCIPAL_ID=$(az identity show \ --name tfy-user-identity \ --resource-group tfy-datascience \ --query principalId -otsv)
- To create a new managed identity use the below command and save the output Client ID
-
Get the ID of the DNS zone
DNS_ZONE_ID=$(az network dns zone show \ --name demo2.truefoundry.com \ --resource-group tfy-datascience \ --query id -otsv)
-
Assign the managed identity DNS contributor role to the DNS zone
az role assignment create \ --assignee $PRINCIPAL_ID \ --role "DNS Zone Contributor" \ --scope $DNS_ZONE_ID
-
Assigning the service account in kubernetes to use the managed identity. This is called federated identity where we are assigning the managed identity so that is can be used by
-
Set these variables with proper values
# service account of cert-manager SERVICE_ACCOUNT_NAME=cert-manager # namespace of cert-manager SERVICE_ACCOUNT_NAMESPACE=cert-manager # get the resource group # make sure to use your resource group if the value is different RESOURCE_GROUP=tfy-datascience # get the cluster name # make sure to use the right cluster name if the values is different CLUSTER_NAME=tfy-aks-cluster # if you are using identity created in azure aks installation page IDENTITY_NAME=tfy-user-identity # get the OIDC_ISSUER_URL OIDC_ISSUER_URL=$(az aks show \ --resource-group $RESOURCE_GROUP \ --name $CLUSTER_NAME \ --query "oidcIssuerProfile.issuerUrl" -o tsv)
-
Creating federated identity
az identity federated-credential create \ --name "cert-manager" \ --identity-name "${IDENTITY_NAME}" \ --issuer "${OIDC_ISSUER_URL}" \ --resource-group "${RESOURCE_GROUP}" \ --subject "system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}"
-
-
We will edit the
cert-manager
application that is installed in the cluster.- Go to Deployments section from the left panel and edit you application
cert-manager
from the Helm tab - Pass these values in the values section and click submit
installCRDs: true extraArgs: - --issuer-ambient-credentials podLabels: azure.workload.identity/use: "true" serviceAccount: labels: azure.workload.identity/use: "true"
--issuer-ambient-credentials
is a flag incert-manager
to use credentials from metadata- The labels passed above will help
cert-manager
to authenticate to Azure.
- Go to Deployments section from the left panel and edit you application
-
Create
Issuer
. Make sure to create it inistio-system
namespace-
Fill in these variables
# your mail ID MAIL_ID=<> # dns zone name DNS_ZONE_NAME="demo2.truefoundry.com" # subscription ID AZURE_SUBSCRIPTION_ID=<> # get managed identity client ID IDENTITY_CLIENT_ID=$(az identity show \ --name $IDENTITY_NAME \ --resource-group $RESOURCE_GROUP \ --query 'clientId' -otsv)
-
kubectl apply -f - <<EOF apiVersion: cert-manager.io/v1 kind: Issuer metadata: name: demo2-issuer namespace: istio-system spec: acme: # give your email ID which will get notificate when the certificate gets expired email: $MAIL_ID # staging server is used for testing, it will create certificate # which are not publicly accepted. For valid production certificates # use https://acme-v02.api.letsencrypt.org/directory as acme.server server: https://acme-staging-v02.api.letsencrypt.org/directory privateKeySecretRef: name: demo2-privkey solvers: - dns01: azureDNS: hostedZoneName: $DNS_ZONE_NAME resourceGroupName: $RESOURCE_GROUP subscriptionID: $AZURE_SUBSCRIPTION_ID environment: AzurePublicCloud managedIdentity: clientID: $IDENTITY_CLIENT_ID EOF
-
-
Create a certificate
- Make sure to point the issuer name from the above code correctly in
issuerRef.name
- Create the below certificate
kubectl apply -f - <<EOF apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: demo2-cert namespace: istio-system spec: # the secret in which the cert will be pasted once provisioned # it MUST BE PRESENT IN THE istio-system NAMESPACE secretName: demo2-tls issuerRef: # Issuer name from the previous step name: demo2-issuer dnsNames: - "demo2.truefoundry.com" - "*.demo2.truefoundry.com" EOF
- It will take sometime for certificate to get created
$ kubectl get cert -n istio-system NAME READY SECRET AGE demo2-cert True demo2-tls 4m29
- The secret
demo2-tls
will contain the certificates$ kubectl get secret -n istio-system -l controller.cert-manager.io/fao=true NAME TYPE DATA AGE demo2-tls kubernetes.io/tls 2 5m47s
- Make sure to point the issuer name from the above code correctly in
Adding the secret in ingress
-
Now the secret is created we need to add this secret to
tfy-istio-ingress
application for it terminate to SSL. -
Go to Deployments section from the left panel and edit application
tfy-istio-ingress
. -
Enter the secret in the values section at
tfyGateway.spec.servers[1].tls.credentialName
and click SubmittfyGateway: name: tfy-wildcard spec: servers: - tls: httpsRedirect: true port: name: http-tfy-wildcard number: 80 protocol: HTTP hosts: - "*.demo2.truefoundry.com" - "demo2.truefoundry.com" - port: name: https-tfy-wildcard number: 443 protocol: HTTPS hosts: - "*.demo2.truefoundry.com" - "demo2.truefoundry.com" tls: mode: SIMPLE credentialName: demo2-tls selector: istio: tfy-istio-ingress
-
Once the secret is added we need to enable the base domain URL from the cluster settings.
- Go to Integrations tab and click on Edit section of the cluster.
- Add the base domain URL
-
Now these domain URLs can be used to expose your endpoints in the deployments section.
- Go to deployments section and create a new service (let is be
nginx
) - Expose it at port 80.
- Create the service and access the URL. The format will be
https://nginx-<workspace-name>-80.demo2.truefoundry.com/
- Go to deployments section and create a new service (let is be
Updated 15 days ago