I've always known I could do this, but I didn't realize it was apparently now easy for people and their phones to use client certs! I will definitely be using this to share some of my services with friends! Thanks!
Great video! I use mTLS all the time for deploying IoT devices, it's a great way to ensure only authorized devices can connect to your services and it's easily scalable. A small note on extended key usage: You may not want your issued client certs to allow for server usage, as anyone with a client cert could potentially impersonate your server. SNI can protect against this as well.
They can't exactly impersonate the server, they are still limited by CN/SAN to their own name(s). Presumably the server would not have the same FQDN as the client.
At work, we've been using this for years for one of our products. To connect to the API, you have to field a TLS user certificate to validate your connection, and that you have permission to operate as the user you're requesting to connect as.
Pretty cool, didn't know you could do that. I may use this for any web apps instead of needing to constantly VPN in. Makes it a lot easier to setup for any users.
Probably beyond the scope of mTLS, but could this be done via passkeys? Rather than generating the .p12 package for a phone's browser, somehow enroll a passkey?
Most/all mobile devices support device keys (which can be used for mTLS) via MDM for enterprise setups, but without an MDM it's a bit more tricky. I've done a little bit with Apple mobileconfig's, but those are a bit of a process (client gets instructed on how to send a CSR for its OS-bound key to a CA using the SCEP protocol) mTLS also supports true hardware keys, and I have used it with a Yubikey, which is just a little more detail than I wanted to go into in this video.
Mutual TLS will improve security at some level, but it won't protect you from vulnerabilities in the server software itself. So if Caddy or Nginx server has problems not related to the apps you are exposing with it, you are still vulnerable. Always use separate vlan for your servers, proper WAF with properly configured rules and other security software like Crowdsec to keep a bit more protected if you expose your services from home.
You’re only vulnerable to caddy or Nginx bugs which can occur in the tls phase, since we never get to sending http traffic. You’re protected from http / web application bugs (at least if your authorized clients aren’t malicious) which are the vast majority of security issues. I’m perfectly happy to trust OpenSSL’s code, much more than basically any Python or node web app.
Daily auto-updates of Nginx should be a great way to mitigate this risk. I never had an Nginx update break something this simple. The biggest risk with this is a configuration mistake, but you run a similar risk with a VPN.
Coud this work with apps that don't use HTTPS as their mode of communication? Because web apps are not the only thing that can be self hosted. The video was a great explanation, that takes skill!
It works for TLS, which is more than just HTTPS, but certainly not everything. Short list is HTTP, MQTT, WebRTC, FTP, IMAP, XMPP, and DNS. SSH also has key-based auth which is also very robust (when properly configured!) and also supports certificate authorities, but it's not based on x.509 certificates.
If you run a wrapper program like "stunnel" on the client you can tunnel Minecraft's TCP protocol over TLS which will let you use certs like this. On the server side I think Caddy will work to convert TLS back to plain TCP, or you can use another copy of stunnel, or probably other programs.
Why would we not just use a RADIUS server with EAP-TLS, since it can also perform LDAP lookups? Why didn't you explain the difference, and when to use one versus the other? Basically, your system merely verifies that a valid cert is being used, but doesn't check whether the user is authorized to use that particular cert. There's nothing in there to stop a rogue user from making a valid cert with a forged user ID. The server needs to verify whether that particular user is authorized to use that particular cert. With OCP stapling etc, it's also useful to the user to verify that a server is authorized to use a particular server cert, as well. This can be necessary for a number of reasons.
Mutual TLS is exactly what EAP-TLS is, but EAP-TLS is encapsulated in a ton of junk. It's also how radsec works. RADIUS is also generally a poor protocol to choose for proxy authentication. With mTLS, server validation (OSCP stapling etc) is as usual, there is no change here. Client validates as they always did, in my tests using Let's Encrypt. Server has already proven itself to LE using ACME. Client TLS validation proves that the cert was issued by our internal authority, so it's therefore a known client. That is enough to go ahead with the TLS session, inside which you could have the user login with a username+password or any other method if you want (especially if mTLS authenticates the *device* and then username+password the user) The username is stored in the CN of the cert and signed by the CA, so it can't be forged. The CN can be passed by the proxy as a header, and if the backend app supports proxy authentication, the app can reliably use that username in its user db to validate the user's permissions in the apps own auth db. If the backend app doesn't support proxy authentication, you could either turn off auth on the backend app (and use mTLS exclusively) or still use usernames+passwords inside the app. You can also do any validation for client certs that you would for server certs, like OCSP or CRLs. You could also have a policy of only issuing hardware-bound certs, such as to Yubikeys or TPMs, all of which is very common with mTLS but too much for one video.
Thanks for the tutorial. I also did a video on it a while ago but seems TH-cam algorithm doesn't like it too much. th-cam.com/video/8DWcMbgQSZg/w-d-xo.html Glad to see bigger channels like yours covering it since it's just really cool TLS feature yet few people use it compared to something like tailscale or wireguard.
I'm not a fan of adding private CAs to browsers. Each such a CA is a huge liability: it has a power of creating valid certificates for any website on the internet, allowing for easy impersonation of any website you wish. The security protocol for a CA whom even a single person trusts has to be very strict, and it's not something I want to deal with. You mention an air-gaped system for the top-level CA, but such system also has to be very carefully protected. I would rather see authentication by adding to the server public keys from self-signed certificates, where public keys are passed in person, and each cert is verified by checking for a match with a corresponding public key. Scalability of such a solution is almost non-existent, so I don't expect it to see any time soon (if ever), but it would make live much easier for a small scale service.
Client certs don't require the client to actually trust their own cert though, only the server needs to trust it. I'm not adding the root CA to the client in this case, the client will not trust other client certs from this CA.
I've always known I could do this, but I didn't realize it was apparently now easy for people and their phones to use client certs!
I will definitely be using this to share some of my services with friends! Thanks!
Thanks for hosting a site with the written instructions!
Thanks!
Thanks!
Great video! I use mTLS all the time for deploying IoT devices, it's a great way to ensure only authorized devices can connect to your services and it's easily scalable. A small note on extended key usage: You may not want your issued client certs to allow for server usage, as anyone with a client cert could potentially impersonate your server. SNI can protect against this as well.
They can't exactly impersonate the server, they are still limited by CN/SAN to their own name(s). Presumably the server would not have the same FQDN as the client.
@@apalrdsadventures I agree, it's definitely an edge case that you shouldn't have to worry about will properly configured clients.
Danke! 👍
Thanks!
Amazing vlog entry, thanks for all the information shared.
Danke!
Thanks!
At work, we've been using this for years for one of our products. To connect to the API, you have to field a TLS user certificate to validate your connection, and that you have permission to operate as the user you're requesting to connect as.
Great practical info and examples, thanks.
Did you up your camera game? The colors look better. Also that blurry background effect. Looks very professional 💯
I'm getting a lot better at using the Nikon
I just use sshuttle to tunnel in. I used to have mTLS but it's such a hassle that i stopped using it.
The one thing I took away from this video is... I need that shirt!
It's from Jeff Geerling
How long did you focus for that shallow depth of field?
I finally got it to stop focusing on Kevin
This setup also works really well with my goverment provided Estonian ID card
Unfortunately our government technology is still in the stone age of COBOL here
Pretty cool, didn't know you could do that. I may use this for any web apps instead of needing to constantly VPN in. Makes it a lot easier to setup for any users.
This was great! Gotta up my Caddy game...
Probably beyond the scope of mTLS, but could this be done via passkeys? Rather than generating the .p12 package for a phone's browser, somehow enroll a passkey?
Most/all mobile devices support device keys (which can be used for mTLS) via MDM for enterprise setups, but without an MDM it's a bit more tricky. I've done a little bit with Apple mobileconfig's, but those are a bit of a process (client gets instructed on how to send a CSR for its OS-bound key to a CA using the SCEP protocol)
mTLS also supports true hardware keys, and I have used it with a Yubikey, which is just a little more detail than I wanted to go into in this video.
@@apalrdsadventures Oh wow, more involved than I thought it would be. Appreciate the details. Thanks!
Nice explanation, thanks. Will this work with iOS Apps though or only via webbrowser?
Depends on the app. Some apps (like the Home Assistant app) specifically support it. Many do not.
Can you do the same, but for eap-tls, specifically over wpa3 in 192 bit mode!
I actually have an eap-tls video already, although I used Smallstep’s saas radius instead of dealing with freeradius.
doesnt android have like the worst support for cert encyption? how are u getting around that
I think you forgot to link to the Diffie-Hellman explainer
You’re right I watched the video myself and got distracted. It’s there now.
Mutual TLS will improve security at some level, but it won't protect you from vulnerabilities in the server software itself. So if Caddy or Nginx server has problems not related to the apps you are exposing with it, you are still vulnerable. Always use separate vlan for your servers, proper WAF with properly configured rules and other security software like Crowdsec to keep a bit more protected if you expose your services from home.
You’re only vulnerable to caddy or Nginx bugs which can occur in the tls phase, since we never get to sending http traffic.
You’re protected from http / web application bugs (at least if your authorized clients aren’t malicious) which are the vast majority of security issues. I’m perfectly happy to trust OpenSSL’s code, much more than basically any Python or node web app.
Daily auto-updates of Nginx should be a great way to mitigate this risk. I never had an Nginx update break something this simple.
The biggest risk with this is a configuration mistake, but you run a similar risk with a VPN.
14:08 haha... I feel your pain when rookie sysadmins copy my server configs and send me their LE renewal notifications.
Great content. I'll be testing and implementing this config in my lab.
Thank you !!! 👍🏻
Coud this work with apps that don't use HTTPS as their mode of communication? Because web apps are not the only thing that can be self hosted.
The video was a great explanation, that takes skill!
It works for TLS, which is more than just HTTPS, but certainly not everything. Short list is HTTP, MQTT, WebRTC, FTP, IMAP, XMPP, and DNS.
SSH also has key-based auth which is also very robust (when properly configured!) and also supports certificate authorities, but it's not based on x.509 certificates.
Nice! Thanks for video
I personally like free Ocserv SSL VPN server.
Could you use this for something like a minecraft server?
Minecraft doesn't use TLS, they do RSA handshaking but not via certificates, via bare keys
If you run a wrapper program like "stunnel" on the client you can tunnel Minecraft's TCP protocol over TLS which will let you use certs like this. On the server side I think Caddy will work to convert TLS back to plain TCP, or you can use another copy of stunnel, or probably other programs.
I’d use haproxy for layer 4. Caddy can’t do layer 4, haproxy is really good at it, and nginx can also do it.
Why would we not just use a RADIUS server with EAP-TLS, since it can also perform LDAP lookups? Why didn't you explain the difference, and when to use one versus the other? Basically, your system merely verifies that a valid cert is being used, but doesn't check whether the user is authorized to use that particular cert. There's nothing in there to stop a rogue user from making a valid cert with a forged user ID. The server needs to verify whether that particular user is authorized to use that particular cert. With OCP stapling etc, it's also useful to the user to verify that a server is authorized to use a particular server cert, as well. This can be necessary for a number of reasons.
Mutual TLS is exactly what EAP-TLS is, but EAP-TLS is encapsulated in a ton of junk. It's also how radsec works. RADIUS is also generally a poor protocol to choose for proxy authentication.
With mTLS, server validation (OSCP stapling etc) is as usual, there is no change here. Client validates as they always did, in my tests using Let's Encrypt. Server has already proven itself to LE using ACME.
Client TLS validation proves that the cert was issued by our internal authority, so it's therefore a known client. That is enough to go ahead with the TLS session, inside which you could have the user login with a username+password or any other method if you want (especially if mTLS authenticates the *device* and then username+password the user)
The username is stored in the CN of the cert and signed by the CA, so it can't be forged. The CN can be passed by the proxy as a header, and if the backend app supports proxy authentication, the app can reliably use that username in its user db to validate the user's permissions in the apps own auth db. If the backend app doesn't support proxy authentication, you could either turn off auth on the backend app (and use mTLS exclusively) or still use usernames+passwords inside the app.
You can also do any validation for client certs that you would for server certs, like OCSP or CRLs. You could also have a policy of only issuing hardware-bound certs, such as to Yubikeys or TPMs, all of which is very common with mTLS but too much for one video.
Thanks for the tutorial. I also did a video on it a while ago but seems TH-cam algorithm doesn't like it too much.
th-cam.com/video/8DWcMbgQSZg/w-d-xo.html
Glad to see bigger channels like yours covering it since it's just really cool TLS feature yet few people use it compared to something like tailscale or wireguard.
I'm not a fan of adding private CAs to browsers. Each such a CA is a huge liability: it has a power of creating valid certificates for any website on the internet, allowing for easy impersonation of any website you wish. The security protocol for a CA whom even a single person trusts has to be very strict, and it's not something I want to deal with. You mention an air-gaped system for the top-level CA, but such system also has to be very carefully protected.
I would rather see authentication by adding to the server public keys from self-signed certificates, where public keys are passed in person, and each cert is verified by checking for a match with a corresponding public key. Scalability of such a solution is almost non-existent, so I don't expect it to see any time soon (if ever), but it would make live much easier for a small scale service.
Client certs don't require the client to actually trust their own cert though, only the server needs to trust it. I'm not adding the root CA to the client in this case, the client will not trust other client certs from this CA.