This post goes into the details of how to make websites you run in your home, such as Home Assistant, exposed to the public securely and (almost) for free.
The post details how to set up a free GCE instance, TLS certificates via LetsEncrypt, a Treafik proxy with Google authentication, and automatic VPNs built using Wireguard. For brevity, I called this setup Wiregate, i.e. a wire-guard gateway.
Overal diagram giving an idea of how Wiregate works.
In this post we will walk through the set up of the VM, configuration of DNS, Letsencrypt TLS for wildcard DNS certificates, configuration of Traefik and google Auth, and set up of Wiregate DNS. We’ll finish with configuring a Plex media server to be Google-auth accessible via plex.example.com. If you want to continue to the Home Assistant guide, see another post.
Getting a Google Cloud project
Google Cloud is now an umbrella for multiple projects. Not only does it allow you to add additional Google Authentication endpoints (which we will need), control your domain (Google DNS), but it also give you a free tier that includes a f1-micro instance in one of the US west zones.
Unfortunately, not everything is included in the free tier, and in order to be able to use it, you will need a credit card to set up billing. However, don’t fret, as the whole thing is pretty cheap to run. I pay ~0.05USD a month for the whole setup below.
For the rest of the post, I’ll be using <myproject> whenever you should use your own project name.
Getting a domain
First, you’ll need a domain name. You can get a relatively good name either at Uniregistry or Google Domains for pretty cheap. For example mylittlecutehouse.uk costs ~10GBP a year.
To make things easier, you should transfer your domain for control to Google Cloud DNS, follow these instructions
For the rest of the post, I’ll use example.com for the suffix.
Setting a GCE instance
Using the f1-micro instances of GCE free tier, we’ll set up an Ubuntu 20.04 LTS instance in the cloud acting as our Cloud endpoint.
Yes, this is not a particularly HA set up, but with GCE’s live-migration downtime is not an issue for this non-bussiness critical setup.
We’ll chose us-east1 as it is the closest free tier region to Europe. Since f1-micro is extremely limited in terms of RAM available (600MB), we will only be running a minimal instance with Go-based programms that don’t require a lot of memory.
Reserve the public IP address
Public IP addresses, wether static or ephemeral cost nothing when used. Let’s get a regional one.
gcloud compute addresses create wiregate
--project=<myproject> --description="Public IP for wiregate." --network-tier=STANDARD --region=us-east1
Take not of the IP address as you will be referring to it later on. For our purposes let’s assume its 35.207.1.1.
Now, update the A record of *.example.com in your Google DNS to point to that IP address.
Create the instance
We’ll create an instance with a relatively large disk (25GB) and bound to the public IP --address=35.207.1.1.
gcloud beta compute --project=<myproject> instances create wiregate --zone=us-east1-c --machine-type=f1-micro --subnet=default --address=35.207.1.1 --network-tier=STANDARD --maintenance-policy=MIGRATE --tags=http-server,https-server --image= ubuntu-minimal-2004-focal-v20200423 --image-project=ubuntu-os-cloud --boot-disk-size=25GB --boot-disk-type=pd-standard --boot-disk-device-name=wiregate --no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring --reservation-affinity=any
Install all the other necessary tools.
As we’ve chosen the minimal install, there’s almost nothing on the system.
apt-get install iputils-ping
apt-get install vim
Set up etckeeper (optional)
Etckeeper allows you to keep all your etc configuration in git.
First, you need to use the root account, as all etc git changes will be triggered from the root account.
sudo su
apt-get -y install etckeeper
eval `ssh-agent -s`
Generage an SSH key for the root user and add it to your Github or Gitlab profile.
etckeeper init
git remote add origin git@<somewhere>
etckeeper commit "Initial commit."
git push --set-upstream origin master
Set up Docker
Since we’ll be running containers, getting Docker to run is a good idea.
sudo apt-get install docker.io
sudo docker version
Setting up Traefik
Traefik is a very flexible, yet lightweight, reverse proxy written in Go. Look out: a lot of example configuration for traefik is meant for v1. This writeup is aimed at v2.2.
Adding extra firewall rules in GCE
Traefik has a debug interface that is visible on 8080. While not strictly needed, it may be useful to have it available during setup. In order to allow 8080 in, we need to add a firewall rule for VM instances matching allow-tcp-8080 by creating the following.
gcloud compute --project=<myproject> firewall-rules create allow-debug-traefik --description="Allows debug Traefik port of 8080" --direction=INGRESS --priority=1000 --network=default --action=ALLOW --rules=tcp:8080 --source-ranges=0.0.0.0/0 --target-tags=allow-tcp-8080
Also we want to make sure to add the allow-tcp-8080 to “Network tags” of the instance (by editing it), alongside the project default http-server and https-server. Test the whole thing running
sudo nc -l 8080
and curling it from somewhere.
When you’re done, remove the 8080 firewall rule from the machine.
Simple Traefik with basic TLS
This section sets up Traefik as a systemd service and configures LetsEncrypt certificate autorenwal for a wiregate.example.com domain exposing the debug webpage of Traefik over TLS.
First, let’s create a config directory for Traefik.
sudo mkdir -p /etc/traefik
Static configuration file
Edit /etc/traefik/traefik.toml to create the static configuration. Static configuration is read at Treafik startup and configures mostly ports and other sources of configration (a.k.a. “providers”)
[log]
level = "DEBUG"
[api]
dashboard = true
debug = false
insecure = true # disable me
[ping]
entryPoint = "traefik"
[accessLog]
[entryPoints]
[entryPoints.web-insecure]
address = ":80"
[entryPoints.web-insecure.http]
[entryPoints.web-insecure.http.redirections]
[entryPoints.web-insecure.http.redirections.entryPoint]
to = "web-secure"
scheme = "https"
[entryPoints.web-secure]
address = ":443"
[entryPoints.traefik]
address = ":8080"
# Note: specifying the provider on the command line doesn't work. This needs to be a docker-local path.
[providers]
[providers.file]
filename = "/conf/traefik.dynamic.toml"
[certificatesResolvers]
[certificatesResolvers.simpletls.acme]
email = "myemail@example.com" # change me
storage = "acme.json"
[certificatesResolvers.simpletls.acme.tlsChallenge]
Dynamic configuration file
Now, edit /etc/traefik/traefik.dynamic.toml to create the dynamic file-based configuration “provider”. This will be read dynamically by a running Traefik. Look out: any errors in this will mean that none of your routes will work. If in doubt always visit wiregate.example.com to see whether that is working.
[http.routers]
[http.routers.simple]
entryPoints = ["web-secure"]
rule = "Host(`wiregate.example.com`)"
service = "api@internal"
[http.routers.simple.tls]
certresolver = "simpletls"
In this configuration we simply tell Traefik to allow wiregate.example.com and show the internal dashboard on it.
The systemd service file
In the file /etc/systemd/system/traefik.service
This mounts the whole /etc/traefik directory under /conf inside the docker container and also serves /etc/traefik/traefik.env` as the environment file.
[Unit]
Description=Traefik Frontend Server
After=docker.service network-online.target
[Service]
Restart=always
EnvironmentFile=/etc/traefik/traefik.env
ExecStart=/usr/bin/docker run \
--attach stderr --attach stdout \
--volume /etc/traefik:/conf \
--env-file=/etc/traefik/traefik.env \
--net=host \
traefik:v2.2 \
--configFile=/conf/traefik.toml
Restart=always
ExecStop=/usr/bin/docker stop -t 2 traefik
[Install]
WantedBy=multi-user.target
Please note the explicit Docker tag of :v2.2 to avoid unexpected auto-updates.
Run
sudo systemctl daemon-reload
sudo systemctl start traefik
sudo systemctl enable traefik
Setting wildcard DNS for Let’s Encrypt using Google DNS
In order to support wildcard certificates, Traefik needs to be be able to modify DNS entries via Google API calls.
Typically in order to use DNS-01 with Google DNS, you need to provide a Google service account via GCE_SERVICE_ACCOUNT_FILE env variable. However, since we’re running a VM on GCE, we can use the Metadata service to serve a service account that is appropriate.
First we need to set up a new service account that will have access to DNS configuration. For that, click here. Select a role: DNS Administrator.
This will create a service account that has DNS writing permissions. Now, edit VM’s Service Account to select the one you just created.
Since we’re running in the same project, the GCE Metadata service should populate both the GCE_PROJECT and GCE_SERVICE_ACCOUNT_FILE of thee configuration
Traefik automatically discovers what DNS names it needs by inspecting the Routes, which are stored in the dynamic configuration file. As such we will need to change both the static and dynamic config files.
Change the static traefik.toml TLS-related section to:
[certificatesResolvers]
[certificatesResolvers.wildcardtls.acme]
email = "myemail@example.com" # changeme
storage = "/conf/acme.json"
[certificatesResolvers.wildcardtls.acme.dnsChallenge]
resolvers = ["8.8.8.8:53", "1.1.1.1:53"]
provider = "gcloud"
# Note: normally we would need to provide ENV variables GCE_PROJECT and GCE_SERVICE_ACCOUNT_FILE.
# However we are running on GCE with a service account that has DNS permissions on the VM, so these will
# be sorted out automatically from GCE Metadata service.
Note, the /etc/traefik/acme.json contains secrets. Add it to .gitignore in the same dir so that etckeeper won’t be checking them in.
Now we need to specify the DNS addressess we want. For that, we need to use a SANS field inside a route. Modify the dynamic profier file of traefik.dynamic.toml to:
[http.routers]
[http.routers.simple]
rule = "Host(`wiregate.example.com`)"
service = "api@internal"
[http.routers.simple.tls]
certresolver = "wildcardtls"
[[http.routers.simple.tls.domains]]
main = "example.com"
sans = ["*.example.com"]
Google Authentication proxy using traefik-forward-auth
Traefik is very flexible, and one of its capabilities is to have pluggable authentication systems. One of them is traefik-forward-auth. It can intercept any browser-based session and require the user to log in with a whitelisted Google account (or any other OpenID Connect provider).
The setup is relatively complicated: it requires traefik-forwad-auth to run alongside Traefik, with the latter having a middleware configured for all routes that are meant to be authenticated.
OAuth2 credentials
As with any OAuth2 flow, we need to tell the upstream authentication provider who we are.
For that we can use the IAM configuration panel of or Google Cloud project. We’ll need to create a OAuth consent screen and new set of credentials. Follow the upstream instructions on how to do this.
Configuring traefik-forward-auth
We will use a file0based configuration placing it in /etc/traefik/traefik-forward-auth.ini
Please read the below file carefuly, as it really has things left out.
# Run in separate-host configuration, issuing auth for all of *.example.com.
auth-host=auth.example.com
cookie-domain=example.com
default-provider=google
providers.google.client-id= # what Google Console told you
providers.google.client-secret= #what Google console told you
# Repeated list of allowed Google accounts.
whitelist= #email @gmail.com
whitelist= #email @gmail.com
# Secret for signing cookies.
secret= # generate using `openssl rand -hex 16`
Configuring the systemd service
The service file should be put in /etc/systemd/system/traefik-forward-auth.service
[Unit]
Description=Traefik Forward Auth Server
After=docker.service network-online.target
[Service]
Restart=always
EnvironmentFile=/etc/traefik/traefik-forward-auth.env
ExecStart=/usr/bin/docker run \
--attach stderr --attach stdout \
--volume /etc/traefik:/conf \
--env-file=/etc/traefik/traefik-forward-auth.env \
--net=host \
thomseddon/traefik-forward-auth:2.1 \
--config=/conf/traefik-forward-auth.ini
Restart=always
ExecStop=/usr/bin/docker stop -t 2 traefik-forward-auth
[Install]
WantedBy=multi-user.target
Please note the explicit Docker tag of :2.1 to avoid unexpected auto-updates.
In order to start it up, just run:
sudo systemctl daemon-reload
sudo touch /etc/traefik/traefik-forward-auth.env
sudo systemctl start traefik-forward-auth
sudo systemctl enable traefik-forward-auth
Simple auth example
The below new traefik.dynamic.toml configuration will set up the auth.example.com endpoint that will be used by Traefik Auth for the Google authentication dance, and set up a middleware that will require users to authenticate whenever visiting wiregate.example.com
[http.routers]
[http.routers.simple]
rule = "Host(`wiregate.example.com`)"
service = "api@internal"
middlewares = ["google-forward-auth"]
[http.routers.simple.tls]
certresolver = "wildcardtls"
[[http.routers.simple.tls.domains]]
main = "example.com"
sans = ["*.example.com"]
[http.routers.auth]
rule = "Host(`auth.example.com`)"
service = "forward-auth"
middlewares = ["google-forward-auth"]
[http.routers.auth.tls]
certresolver = "wildcardtls"
[[http.routers.auth.tls.domains]]
main = "example.com"
sans = ["*.example.com"]
[http.middlewares]
[http.middlewares.google-forward-auth.forwardAuth]
address = "http://127.0.0.1:4181/"
trustForwardHeader = true
authResponseHeaders = ["X-Forwarded-User"]
[http.services]
# Traefik Forward Auth server running locally.
[http.services.forward-auth.loadBalancer]
[[http.services.forward-auth.loadBalancer.servers]]
url = "http://127.0.0.1:4181/"
At this point, if everything went right, you should be able to access the wiregate.example.com and see a login consent screen from Google.
Setting up WireGuard VPN
Wireguard is a new VPN protocol that is now part of the mainline Linux kernel. Unlike anything else, it is very easy to set up, and automatically resumes in cases of disconnects.
It is idea for our use case: a single Cloud VM that home devices (such as a Rapsberry Pi) can connect to automatically from behind a NAT.
We’ll use 10.212.0.0/24 as the subnet for the private VPN, as it is rarely used by home networks. We will use wiregate to denote the Cloud VM, and mediaserver to represent the home device. All IP addresses will be statically configured, and chosen as 10.212.0.1 and 10.212.0.11 respectively.
Note: This set up is different from your typical “I want to browse the internet over VPN”. The home devices will not route their traffic via wiregate instance, instead this acts as a way for the Traefik proxy to reach services hosted on home devices.
Add GCE firewall entry for Wireguard
Wireguard usually operates over UDP, over the 51820 port.
cloud compute --project=<myproject>
firewall-rules create allow-wireguard --description="Allows wireguard on standard 51820 port." --direction=INGRESS --priority=1000 --network=default --action=ALLOW --rules=udp:51820 --source-ranges=0.0.0.0/0 --target-tags=allow-wireguard
Now edit the VM instance and add a network tag of allow-wireguard next to https-server.
Installing Wireguard
On both machines machines install wireguard:
sudo add-apt-repository ppa:wireguard/wireguard
sudo apt install wireguard
We’ll be roughly following the Linode guide with extra info from ArchLinux and the wg-quick setup from Debian.
Generating secrets
On both machines you need to generate a private and a public key.
umask 077
wg genkey | tee privatekey | wg pubkey > publickey
This will create privatekey and publickey files, with obvious content. We will use wiregate_private, mediaserver_public etc. to represent the content of these files in subsequent configuration files.
Configure wiregate side
The wiregate side of file /etc/wireguard/wg0.conf:
# Interface for *non internet* VPN setup from remote devices. Each new one needs to added as a peer.
[Interface]
Address = 10.212.0.1/24
ListenPort = 51820
MTU = 1380 # working around GCE MTU limits
PrivateKey = # wiregate_privatekey
[Peer]
# mediaserver
PublicKey = # mediaserver_publickey
AllowedIPs = 10.212.0.11/32
If you want to add any additional peers, just repeate the [Peer] section with a different IP.
Run:
sudo systemctl enable wg-quick@wg0
sudo systemctl start wg-quick@wg0
This will make wiregate configure wg0 interface at startup.
Configure mediaserver side
The mediaserver side of the file /etc/wireguard/wg0.conf:
[Interface]
Address = 10.212.0.11/24
MTU = 1380 # same as cloud
PrivateKey = # mediaserver_privatekey
[Peer]
PublicKey = # wiregate_publickey
AllowedIPs = 10.212.0.0/24
Endpoint = wiregate.example.com:51820
PersistentKeepalive = 25
After set up you should see on wiregate something as follows:
$ sudo wg show
interface: wg0
public key: # wireguard_public_key
private key: (hidden)
listening port: 51820
peer: # mediaserver_public_key
endpoint: 83.115.194.221:22531
allowed ips: 10.212.0.11/32
latest handshake: 51 seconds ago
transfer: 13.91 KiB received, 5.32 KiB sent
This means that the tunnel is working, and you should easily be able to ping 10.212.0.1 and ping 10.212.0.11 from the different sides of the tunnel.
Add HTTP routes
Now that we have a wireguard connection, and Traefik configured with authentication, we can configure an HTTP route. We’ll use Plex Media server as an example.
Add hosts entries
Add each peer to /etc/hosts:
# Wireguard hosts
10.212.0.11 mediaserver.vpn.local mediaserver
This will allow you to easily represent hosts in your Traefik configuration.
Traefik rules
We have the on-server interface of Plex Media server running on mediaserver at its default port 32400. The final dynamic config file, including the above configuration should look as follows:
[http.routers]
[http.routers.simple]
rule = "Host(`wiregate.example.com`)"
service = "api@internal"
middlewares = ["google-forward-auth"]
[http.routers.simple.tls]
certresolver = "wildcardtls"
[[http.routers.simple.tls.domains]]
main = "example.com"
sans = ["*.example.com"]
[http.routers.auth]
rule = "Host(`auth.example.com`)"
service = "forward-auth"
middlewares = ["google-forward-auth"]
[http.routers.auth.tls]
certresolver = "wildcardtls"
[[http.routers.auth.tls.domains]]
main = "example.com"
sans = ["*.example.com"]
[http.routers.plex]
rule = "Host(`plex.example.com`)"
service = "plex"
middlewares = ["google-forward-auth"]
[http.routers.plex.tls]
certresolver = "wildcardtls"
[[http.routers.plex.tls.domains]]
main = "example.com"
sans = ["*.example.com"]
[http.middlewares]
[http.middlewares.google-forward-auth.forwardAuth]
address = "http://127.0.0.1:4181/"
trustForwardHeader = true
authResponseHeaders = ["X-Forwarded-User"]
[http.services]
# Traefik Forward Auth server running locally.
[http.services.forward-auth.loadBalancer]
[[http.services.forward-auth.loadBalancer.servers]]
url = "http://127.0.0.1:4181/"
# Plex server running on Media server.
[http.services.plex.loadBalancer]
[[http.services.plex.loadBalancer.servers]]
url = "http://mediaserver.vpn.local:32400/"
Note the repetition of domains. If one moved to a YAML file, this would be less repetetive by using YAML anchors.
What’s next
Now that you have a (almost free) Cloud endpoint for your home devices, you can easily build automations that implement web hooks for your home automations.
The primary reason why I set it up was to have a public-facing endpoint for my Home Assistant. The guide on how to set up Home Assistant to work wtih Google Authentication and Google Assistant can be found in a further post.