Ingress, cert-manager, and Let's Encrypt DNS-01 validation (DigitalOcean guide)

This article is going to be very specific — I will tell you a bit about how I deploy my web applications, and what strategies I find most convenient and easy.

It's worth saying that my technological stack is pretty easy and standard — I deploy a bunch of Docker containers, use Kubernetes to manage them all, and run Ingress to expose HTTP and HTTPS routes from outside the cluster to services within the cluster. All these things allow me to scale the application quite easily, manage it with almost no difficulties, and focus on the product itself.

However, there is one thing I found very challenging — SSL certificates. Of course, there is a cert-manager that can easily be installed with HELM, and in theory it should handle all the problems with ease. However, I struggled a bit with cert-manager, and I simply want to make your life easier by telling you how to issue certificates using the DNS-01 challenge provider.

Wait, what's that?

Everybody who builds a web application in 2019 needs to care about HTTPS. It means getting a certificate, updating it from time to time, and getting more and more certificates for all the subdomains.

Cert-manager is a beautiful Kubernetes add-on that is designed to automate the management and issuance of TLS certificates from various issuing sources. For example, you can get your certificate from Let's Encrypt, and cert-manager will handle all the steps of the process.

There are several guides on how to start using it, and the best ones are the official one and the one made by users of DigitalOcean. However, Let's Encrypt needs to validate that you control the domain names, and it uses ACME "challenges" for this purpose. There are basically two most commonly used types of challenges — HTTP-01 and DNS-01.

The idea of validation in short

Why are there at least two different ways to confirm that you own and control a domain? Well, because the situations can also be different.

In the case of HTTP-01 the validation is done in a pretty dumb way: ACME client puts a special file on your server at http://<YOUR_DOMAIN>/.well-known/acme-challenge/<TOKEN>, and Let's Encrypt tries to GET it. Usually it works, but I spend quite a lot of time struggling with this method — it turns out my configuration blocks port 80 (thus GETting is not possible), and you cannot issue the wildcard certificate (*.telescope.ac) anyway if you use HTTP-01.

That's why there is a DNS-01 challenge. In this case, the validation is done via a TXT record under the domain name. You can read more about different details of the validation process here: https://letsencrypt.org/docs/challenge-types

Using DNS-01 challenge

99% of all the guides focus on HTTP-01 — mostly because this validation method works in almost all cases. However, I used DNS-01, and I want to focus more on this method. It's also worth saying that my guide uses DigitalOcean, and cert-manager supports its API by default.

It means that you won't need to create TXT records manually if you use DigitalOcean — just give cert-manager an access the DO API, and it will do all these things automatically.

Of course, cert-manager also supports other DNS providers like AzureDNS, Cloudflare, Google CloudDNS, etc. However, don't worry if your provider is not supported — in this case you will just need to create all the TXT records manually.

1. An Issuer

apiVersion: certmanager.k8s.io/v1alpha1

kind: Issuer

metadata:

  name: letsencrypt-telescope-dns

spec:

  acme:

    server: https://acme-v02.api.letsencrypt.org/directory

    email: <EMAIL>

    privateKeySecretRef:

      name: letsencrypt-telescope-dns

    solvers:

    - dns01:

        cnameStrategy: Follow

        digitalocean:

          tokenSecretRef:

            name: telescope-digitalocean-dns

            key: access-token

(based on https://docs.cert-manager.io/en/latest/tasks/issuers/setup-acme/dns01/digitalocean.html).

Save it in issuer.yaml, and apply like that:

> kubectl apply -f issuer.yaml

This configuration defines an Issuer that contacts Let’s Encrypt in order to issue certificates. It uses an access token from DigitalOcean that we're now going to define.

2. DigitalOcean API access and a Secret

apiVersion: v1

kind: Secret

metadata:

  name: telescope-digitalocean-dns

data:

  access-token: <YOUR TOKEN IN BASE64>

> kubectl apply -f secret.yaml

This is how the definition of a DigitalOcean token looks like. But how can I get one?

Just go to https://cloud.digitalocean.com/account/api/tokens and generate a new personal access token. DigitalOcean will ask for a name, and then print you the new token. You have to transform it to base64 and then paste to the snippet above.

This is how it can be done on Mac/Linux:

> echo -n '<TOKEN>' | base64

3. Certificate definition

apiVersion: certmanager.k8s.io/v1alpha1

kind: Certificate

metadata:

  name: mydomain-com-certificate

spec:

  secretName: telescope-ac-tls

  issuerRef:

    name: letsencrypt-mydomain-dns

  commonName: telescope.ac

  dnsNames:

  - mydomain.com

  - "*.mydomain.com"

> kubectl apply -f certificate.yaml

4. Done! Let's now tell Ingress to use the certificate

Ok, we're done with all the cert-manager definitions. We've also told it how to generate certificates and what mechanism to use to validate domains. The last step is to create an Ingress that will use the Issuer we defined above.

apiVersion: extensions/v1beta1

kind: Ingress

metadata:

  name: mydomain-ingress

  annotations:

    kubernetes.io/ingress.class: nginx

    certmanager.k8s.io/issuer: letsencrypt-mydomain-dns

spec:

  tls:

  - hosts:

    - mydomain.com

    - subdomain.mydomain.com

    ...

    secretName: mydomain-ingress-tls

  rules:

  - host: mydomain.ac

    http:

      paths:

      - backend:

          serviceName: mydomain-backend-service

          servicePort: 80

  - host: subdomain.mydomain.com

    http:

      paths:

      - backend:

          serviceName: mydomain-backend-service

          servicePort: 80

  ...

4. DNS records

Cert-manager should have already created some orders and challenges (kubectl get orders). Unfortunately, all of them will fail because you have to create A records to your newly created LoadBalancer first.

First, get its IP address:

kubectl get services nginx-ingress-controller

NAME                       TYPE           CLUSTER-IP EXTERNAL-IP PORT(S)                   AGE

nginx-ingress-controller   LoadBalancer   x.x.x.x   x.x.x.x   80:31830/TCP,443:31137/TCP   65m

What you need is EXTERNAL-IP. You have to wait if kubectl shows <pending> there.

After you got the IP address, the important step is to create A records for all your domains and subdomains. Just got to the domain management page of DigitalOcean and add them there.

Woohoo!

Now you have to wait for some time (issuance takes some, but it shouldn't be longer than 3-5 minutes). After that, you will be able to access both your domain and all the subdomains. And, of course, you will see the issuance of new certificates if you change and apply (kubectl apply -f <filename>) the new definition of your Ingress.

Good luck!