A Better Solution for Development Domains

There you are on the evening of December 5, 2017, working on your project. All of your services are running locally and you’re accessing them through Google Chrome.

Your environment is set up to use an address such as http://thenextbigthing.dev (using pow or via host entries). You commit your work and call it a night. You’ll finish tomorrow.

Little did you know that on December 6, 2017 Google Chrome 63 rolls out and breaks everything for you. You navigate to http://thenextbigthing.dev and get automatically redirected to https:// instead.

This has obviously devastating consequences since you don’t have a server listening on port 443 and even if you did, you don’t have a valid certificate for the domain. What’s going on?

Proposed as part of the gTLD expansion program, .dev (and about 100 other domains) belongs to Google for quite a while now and clearly, they must have some plans for it. The culprit is this tiny commit that is included in Chrome 63 that forces all requests for domains ending with .dev to be redirected to HTTPS via a preloaded HTTP Strict Transport Security (HSTS) header.

You cannot disable this behavior, there’s no workaround. Well, not using Chrome for development is one, but really, what else is there? The easiest way out is to change your setup to use a different gTLD, preferrably one of the protected ones such as .localhost, .test or .example

Or you could forget about this non-secure nonsense, embrace today’s available tools and bring your local setup closer to production by serving your website and APIs with a valid trusted SSL certificate.

Here’s what you’ll need to do so:

Domain Registration

You can either register a domain through Route53 Registration or point an existing domain’s DNS to Route53 if you have one available.

While there, set the following three DNS Records for the domain:

name type value
<DOMAIN NAME> A 127.0.0.1
<DOMAIN NAME> AAAA ::1
*.<DOMAIN NAME> CNAME <DOMAIN NAME>

After the registration or domain transfer is done, take note of its Hosted Zone ID for the next steps.

IAM User & Role

We’ll set up a specific IAM User with restricted privileges that can only manage this domain’s records. Do this from the AWS Console IAM Home. Add a user with a descriptive name such as dev-caddyserver and its Access type to Programmatic access so that you get AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY at the end of the process.

Next up are permissions for this user. Choose ‘Attach existing policies directly’ and then ‘Create policy’, use this JSON policy definition where you replace <Hosted Zone ID> with the one you got earlier, give your policy a name, optionally a description and create it.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "route53:ChangeResourceRecordSets",
      "Resource": "arn:aws:route53:::hostedzone/<Hosted Zone ID>"
    },
    {
      "Effect": "Allow",
      "Action": [
        "route53:GetChange",
        "route53:ListHostedZonesByName"
      ],
      "Resource": "*"
    }
  ]
}

This policy allows the user to Manage DNS Records for your domain, query the state of the specific change it is doing and list all hosted zones by name, no harm can be done to your other Route53 managed domains.

Attach the newly created policy to your IAM User and create it. When created, you should get yourself AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Take note of those too, you cannot get access to the secret later anymore.

Next up you’ll want to store your credentials in a place where all AWS SDKs look for them. This is going to ensure that the folder and files exist (without replacing existing ones) and appends your credentials to a specific profile we’ll be using.

$ mkdir -p ~/.aws
$ touch ~/.aws/credentials
$ echo "
[dev-caddyserver]
aws_access_key_id=<AWS_ACCESS_KEY_ID>
aws_secret_access_key=<AWS_SECRET_ACCESS_KEY>" >> ~/.aws/credentials

That’s about it for the AWS part.

Installing Caddy

You’ll need Go to install Caddy and you’ll need to include a specific plugin for seamless certificate management. Unfortunately the available homebrew recipe does not allow you to specify the list of plugins you wish to include but there’s a simple to use oneliner for installation provided by Caddy.

$ brew install go
$ curl https://getcaddy.com | bash -s personal tls.dns.route53

Setting up the first website

The following makes sure the Caddy folder structure is set up and that the first website configuration file is there. Replace <DOMAIN NAME> with the domain you’ve registered.

$ mkdir -p ~/.caddy/sites
$ cd ~/.caddy
$ echo "import sites/test.conf" >> Caddyfile
$ echo "https://test.<DOMAIN NAME> {
  tls {
    dns route53
  }

  proxy / localhost:9292 {
    transparent
  }
}" > "sites/test.conf"

Let’s run a simple webserver on port 9292 with this one liner (preferrably in a new terminal tab).

$ node -e "require('http').createServer(function (req, res) { res.end('OK CADDY'); }).listen(9292);"

Start Caddy

sudo AWS_PROFILE=dev-caddyserver caddy --conf ~/.caddy/Caddyfile

The first time Caddy is started it will prompt you for your email address but you may just leave it empty. Caddy will now attempt to retrieve certificates from Let’s Encrypt and will retain them for 60 days. It will also automatically renew the certificates when necessary.

Activating privacy features... done.
https://test.<DOMAIN NAME>
http://test.<DOMAIN NAME>

Open your broken .dev Chrome and navigate to http://test.<DOMAIN NAME>. You should be automatically redirected to the secure website and see a valid trusted certificate being presented to Chrome now.

How the … ?

Caddy is using the tls.dns.route53 plugin to push DNS entries that Let’s Encrypt is looking for when confirming the domain ownership. By default this would be done using a challenge exposed via http but your dev host is not really reachable for them. Hence the DNS way of verifying.

Other providers are available

There are alternative DNS providers and plugins to choose from, should the monthly 50¢ Route53 DNS management charge be an issue. I haven’t tested them myself but I’m sure you will be able to figure them out if you follow their documentation. At the time of writing these are

And you can read up and inspect the source for each here.

Trust your TLS offloading Caddy

Be sure to consult the documentation of your web application frameworks to ensure they take the proxy-added headers (e.g. X-Real-IP, X-Forwarded-For, X-Forwarded-Proto) into consideration when resolving current URLs and similar. This varies from framework to framework. A few examples for popular NodeJS frameworks

// Koa http://koajs.com
app.proxy = true;

// Express https://expressjs.com
app.enable('trust proxy');

Resources