TLS and wildcard certificates

If you are hosting a website at, you'd need to do three things:

  1. Get the domain
  2. Host a website somewhere, and point your domain at that webpage.
  3. Get a "TLS certificate," a cryptographic signature that stops others from pretending to be your website.

The third step is relatively new, having only been effectively required since 2015.

Sites without TLS certificates now show a scary warning and have lower search result rankings.

Recently, all three of these steps have become much simpler, with platforms like Cloudflare and Github Pages letting you host your sites while automatically getting TLS certificates.

How providers can offer free TLS certificates

When you use a managed website provider, usually you point one of your domains at their servers.

For example, you might make a DNS CNAME record pointing from to if you're using GitHub Pages.

When you tell GitHub about your domain, they're able to ask LetsEncrypt for a free certificate on your behalf:

  1. GitHub tells LetsEncrypt, "Please give me a certificate for"
  2. LetsEncrypt responds, "Prove you own it, place this file at"
  3. GitHub puts that file at
  4. LetsEncrypt queries, sees your CNAME record which says to check, and they see GitHub's record.

In all, this process is relatively easy, and this general mechanism is how almost all site hosting providers generate certificates for free for you.

The problem with wildcard certificates

Regular TLS certificates only work with one domain. If your site has pages at and, GitHub would need to go through the process above twice.

Similarly, if your app lets users create profiles, it might have a separate domain for every user:, etc.

Wildcard certificates are of the form *, they can be used for any of th epages above -,, etc.

The problem with wildcard certificates is that your hosting provider can't easily get them from LetsEncrypt. The process above (putting a file at a specific path) isn't possible, the only way to prove you control the entire wildcard domain is with another challenge:

DNS-01 and proving you own someone else's domain

To request a * certificate from letsencrypt, GitHub Pages would need to use DNS-01, which looks like this:

  1. GitHub tells LetsEncrypt, "Please give me a certificate for *"
  2. LetsEncrypt responds, "Prove you own it, create this DNS record: TXT (some value)
  3. The hard part: GitHub would need to create that record somehow
  4. LetsEncrypt queries, sees the record exists, and sends GitHub the * TLS certificate

Step 3 is the reason GitHub Pages doesn't support wildcard certificates - how could they create certificates within a domain that they don't own?

It's not impossible: Github could run their own DNS server

If you're building something like GitHub pages, and want your users to be able to create wildcard certificates, you'll need to run your own DNS server.

In GitHub's case, they want to work with two kinds of requests:

  1. Queries like "Which server is hosting the website for" should be responded to with "one of GitHub's webservers"
  2. Queries like "What is the value of the TXT record at should be responded to with "whatever letsencrypt wants"

Take a look at this example, where you've made * point to

  1. GitHub tells LetsEncrypt, "Please give me a certificate for *"
  2. LetsEncrypt responds, "Prove you own it, create this DNS record: TXT letsencryptpassword
  3. GitHub tells its DNS server to answer DNS queries for TXT with letsencryptpassword
  4. LetsEncrypt queries the TXT record at, is told "see" by your CNAME record. It dutifully queries's DNS server for the TXT record at and sees  letsencryptpassword, so it gives GitHub the certificate for *

A sample implementation

We had to solve this problem for, where users can add their own domains to host their preview environments.

For example, a user might want branch feature1 to be hosted at

This means we need to request the wildcard certificate * This example uses the popular miekg/dns go package.

The DNS handler function from the previous section would look like this:

dns.HandleFunc(".", func(writer dns.ResponseWriter, msg *dns.Msg) {
	respMsg := &dns.Msg{
		Answer: []dns.RR{},
	for _, q := range msg.Question {
		if q.Qtype == dns.TypeTXT {
			acmeResponse := getAcmeChallengeFor(q.Name)
			if acmeResponse == "" {
				respMsg.SetRcode(msg, dns.RcodeNameError)
			} else {
				respMsg.Answer = append(respMsg.Answer, &dns.TXT{
					Hdr: dns.RR_Header{
						Name:   q.Name,
						Rrtype: dns.TypeTXT,
						Class:  dns.ClassINET,
						Ttl:    0,
					Txt: []string{acmeResponse},
				respMsg.Authoritative = true
		} else if q.Qtype == dns.TypeA || q.Qtype == dns.TypeAAAA || q.Qtype == dns.TypeCNAME {
			matches := pattern.FindStringSubmatch(q.Name)
			if matches != nil {
				respMsg.SetRcode(msg, dns.RcodeNameError)
			respMsg.Answer = append(respMsg.Answer, &dns.CNAME{
				Hdr: dns.RR_Header{
					Name:   dns.Fqdn(q.Name),
					Rrtype: dns.TypeCNAME,
					Class:  dns.ClassINET,
					Ttl:    86400,
				Target: dns.Fqdn(""),
			respMsg.Authoritative = true
		err := writer.WriteMsg(respMsg)
	if err != nil {

To reiterate:

  • If someone asks for a TXT record, answer with what letsencrypt is expecting
  • If someone asks for a website, answer "check out this other page"

Wildcard certificates on-demand

There's one last related problem to solve: a wildcard certificate for * cannot be used for

That's where on-demand TLS, a concept popularized by the Caddy web server, comes in.

The idea is to generate the TLS certificate when a user visits the page for the first time:

  1. Someone requests
  2. We use DNS-01 with LetsEncrypt and our DNS server to create a record for *
  3. The user sees the newly issued certificate, and can successfully view the site.

Combining it with our wildcard TLS certificate flow from the previous section, the final process would look like this:

Graphic of generating wildcard TLS certificates on demand

Alternatives to LetsEncrypt

Some of the teams that use request over a hundred unique certificates per day. LetsEncrypt has limits on how many certificates you can issue, for that reason, we split our requests across LetsEncrypt, ZeroSSL, and We've had a particularly good experience with ZeroSSL, where we've issued many thousand certificates.