Added support for custom SSL certificate #969

kpumuk · 2024-09-24T18:41:39Z

Following the change in kamal-proxy, this MR introduces a configuration option to load custom SSL certificate and the corresponding private key from disk:

proxy:
  ssl: true
  ssl_certificate_path: /data/cert/foo.example.com/fullchain.pem
  ssl_private_key_path: /data/cert/foo.example.com/privkey.pem

Documentation preview:

yogeshjain999 · 2024-09-30T06:28:25Z

I'm a newbie to kamal (and docker) and was wondering what's the recommended way to get those custom PEM files on server.

Copy using kamal+Dockerfile or do it manually ? Maybe better alternative would be to load the values via ENV variables (given kamal-proxy could support it) ?

kpumuk · 2024-09-30T18:51:02Z

I'm a newbie to kamal (and docker) and was wondering what's the recommended way to get those custom PEM files on server.

The easiest way would be to use pre-proxy-reboot. For example, if you use 1Password to manage secrets:

Put both cert.pem and key.pem under the item you use for secrets

Create .kamal/hooks/pre-proxy-reboot:

#!/bin/sh

set -euo pipefail

KAMAL_PROXY_TLS_CERT=$(op read "op://Private/Kamal Demo/cert.pem")
KAMAL_PROXY_TLS_PRIVATE_KEY=$(op read "op://Private/Kamal Demo/key.pem")

for ip in ${KAMAL_HOSTS//,/ }; do
  ssh -q -T -o BatchMode=yes ubuntu@"${ip}" bash --noprofile <<-EOF
    mkdir -p .kamal/apps/${KAMAL_SERVICE}/tls
    echo '${KAMAL_PROXY_TLS_CERT}' > .kamal/apps/${KAMAL_SERVICE}/tls/cert.pem
    echo "${KAMAL_PROXY_TLS_PRIVATE_KEY}" > .kamal/apps/${KAMAL_SERVICE}/tls/key.pem
EOF
done

Edit config/deploy.yml to mount TLS certificates to Kamal's image and then enable them:

proxy:
  ssl: true
  host: app.example.com
  ssl_certificate_path: /home/kamal-proxy/.config/certs/cert.pem
  ssl_private_key_path: /home/kamal-proxy/.config/certs/key.pem
  volumes:
    - "/home/ubuntu/.kamal/apps/demo/certs:/home/kamal-proxy/.config/certs"

Run kamal proxy reboot to deploy

Copy using kamal+Dockerfile or do it manually ? Maybe better alternative would be to load the values via ENV variables (given kamal-proxy could support it) ?

kamal-proxy does not support environment variables.

TODO: Add volumes support to proxy.

yogeshjain999 · 2024-10-01T06:13:13Z

Cool, setting it up via pre-proxy-reboot sounds good approach. Thanks!!

kpumuk · 2024-10-02T15:27:32Z

@djmb to follow up on the thread in kamal-proxy, here is the documentation update + support for the recent custom TLS cert changes.

agu-z · 2024-10-09T10:52:20Z

Is it possible to specify a client certificate too? I need this in order to enable CloudFlare's Authenticated Origin Pulls

mtmckenna · 2024-10-14T21:07:52Z

With this change, would it be possible to remove the ensure_one_host_for_ssl requirement when providing a cert and key?

Context: I'm looking for a way to have end-to-end in-transit encryption in a regulated environment that requires TLS between the load balancer and server node. I'd also like to be able to use multiple app servers.

By providing my own cert to kamal-proxy, I was thinking I should be able to have the load balancer terminate SSL and then re-encrypt the traffic to kamal-proxy, which would use my supplied cert. wdyt?

Thank you!

kpumuk · 2024-10-14T23:03:37Z

By providing my own cert to kamal-proxy, I was thinking I should be able to have the load balancer terminate SSL and then re-encrypt the traffic to kamal-proxy, which would use my supplied cert. wdyt?

Yep, that's exactly how it would work with custom certificates, and it removes the limitation of one host behind the load balancer.

mtmckenna · 2024-10-14T23:31:45Z

Yep, that's exactly how it would work with custom certificates, and it removes the limitation of one host behind the load balancer.

That's great! Would ensure_one_host_for_ssl be updated in a separate PR?

kamal/lib/kamal/configuration/role.rb

Line 152 in 6073681

def ensure_one_host_for_ssl

Thank you!

kpumuk · 2024-10-14T23:45:25Z

That's great! Would ensure_one_host_for_ssl be updated in a separate PR?

Great catch, I missed this change in the upstream :-) Added a commit to this MR

qinmingyuan · 2024-10-17T16:01:47Z

I have used with wildcard domain and multi hosts, Works fine. Please merge this PR.

christo-ph · 2024-10-22T13:45:39Z

I am looking forward to this feature. Could you kindly explain how I can test this without being merged and released?

qinmingyuan · 2024-10-22T14:31:17Z

I am looking forward to this feature. Could you kindly explain how I can test this without being merged and released?

I have override the kamal command like this, and add the github repo to Gemfile:

Thread.report_on_exception = false
require_relative '../config/boot'
require 'kamal'
require 'kamal_override'

begin
  Kamal::Cli::Main.start(ARGV)
rescue SSHKit::Runner::ExecuteError => e
  puts "  \e[31mERROR (#{e.cause.class}): #{e.message}\e[0m"
  puts e.cause.backtrace if ENV["VERBOSE"]
  exit 1
rescue => e
  puts "  \e[31mERROR (#{e.class}): #{e.message}\e[0m"
  puts e.backtrace if ENV["VERBOSE"]
  exit 1
end

djmb

Hi @kpumuk!

Thanks for putting this together and sorry for the delay in responding - I've been away.

I've added some comments on this - the main things are:

providing a single way to map files into the proxy container
read the certs from the Kamal secrets
get kamal-proxy to infer the cert locations

I'll need to tackle the volume/file mounting work as part of the maintenance mode work, so it might be easier to wait until that's out and then this can piggy back on that.

djmb · 2024-10-23T08:38:30Z

lib/kamal/configuration/docs/proxy.yml

+  #
+  # The certificate must be in PEM format and contain the full chain. The private key
+  # must also be in PEM format.
+  ssl_certificate_path: /data/cert/foo.example.com/fullchain.pem


I think we should read these directly from the Kamal secrets. Then here we can set the name of the secret rather than specifying a path.

We can also nest the pems under the ssl key. A hash for the ssl key would imply ssl: true.

E.g:

ssl: certificate_pem: CERTIFICATE_PEM private_key_pem: PRIVATE_KEY_PEM

Then in your secrets file:

# .kamal/secrets CERTIFICATE_PEM=$(kamal secrets extract ...) PRIVATE_KEY_PEM=$(kamal secrets extract ...)

djmb · 2024-10-23T08:40:19Z

lib/kamal/configuration/role.rb

-    if running_proxy? && proxy.ssl? && hosts.size > 1
-      raise Kamal::ConfigurationError, "SSL is only supported on a single server, found #{hosts.size} servers for role #{name}"
+    if running_proxy? && proxy.ssl? && hosts.size > 1 && !proxy.custom_ssl_certificate?
+      raise Kamal::ConfigurationError, "SSL is only supported on a single server or with custom SSL certificates, found #{hosts.size} servers for role #{name}"


Maybe unless you provide custom certificates?

djmb · 2024-10-23T09:09:48Z

lib/kamal/configuration/proxy.rb

@@ -30,6 +34,8 @@ def deploy_options
    {
      host: hosts,
      tls: proxy_config["ssl"].presence,
+      "tls-certificate-path": proxy_config["ssl_certificate_path"],


We should mount a single volume into the proxy when we boot it.

This avoids having to set application level configuration for proxy volumes, which we want to avoid because multiple applications can share the same proxy.

We can then re-use that volume for any other features that need to provide files to the proxy (such as custom error pages for maintenance mode).

Also ideally we wouldn't need to pass paths around and kamal-proxy should be able to infer the location of the certificate.

I think it would mean that kamal-proxy would need to assume that app supplied files live in a different subdirectory to /home/kamal-proxy/.config/kamal-proxy so we can map the volume in separately - e.g. $(PWD)/.kamal/proxy/apps:/home/kamal-proxy/.config/kamal-proxy-apps.

/home/kamal-proxy/.config/kamal-proxy is then writable for the proxy, while /home/kamal-proxy/.config/kamal-proxy-apps is read only.

Then we can have /home/kamal-proxy/.config/kamal-proxy-apps/#{service}-#{role}-#{destination}/tls/ and store the certs in there. #{service}-#{role}-#{destination} will match the service name passed to the proxy, so kamal-proxy can infer where the certs are stored.

What to do think @kevinmcconnell?

We'll also need to update kamal app remove to scrub the /home/kamal-proxy/.config/kamal-proxy-apps/#{service}-#{role}-#{destination} directory.

The kamal-proxy-apps plan sounds good to me. The proxy won't really care how it's organized, as we can give it absolute paths for the files that it needs (certs, error pages, etc). But as a scheme for organizing the per-app resources I think it makes sense 👍

Rukamakama · 2024-11-12T16:20:12Z

Looking forward to having this functionality available, we are deploying multiple nodes behind a load balancer and would like to secure the transactions between the load balancer and application servers.
Thanks !

junket · 2024-11-26T14:43:08Z

This seems really close! Thank you @kpumuk ❤️ Anything you need to get this over the finish line?

kpumuk mentioned this pull request Sep 24, 2024

Documentation for custom SSL certificates basecamp/kamal-site#100

Merged

kpumuk force-pushed the custom-ssl branch from a64fd3d to c074dfc Compare September 24, 2024 18:51

kpumuk mentioned this pull request Sep 24, 2024

Added option to use custom TLS certificates instead of ACME basecamp/kamal-proxy#17

Merged

kpumuk force-pushed the custom-ssl branch 2 times, most recently from 834797a to 281e88f Compare September 30, 2024 00:54

kpumuk force-pushed the custom-ssl branch from 281e88f to 1291342 Compare September 30, 2024 16:16

kpumuk force-pushed the custom-ssl branch from 1291342 to a7d3f97 Compare October 2, 2024 15:25

kpumuk force-pushed the custom-ssl branch from a7d3f97 to 874053b Compare October 7, 2024 17:34

Added support for custom SSL certificate

81d96bd

kpumuk force-pushed the custom-ssl branch from 874053b to 28d802e Compare October 14, 2024 23:43

SSL is only supported on a single server or with custom SSL certificates

ff32dcb

kpumuk force-pushed the custom-ssl branch from 28d802e to ff32dcb Compare October 14, 2024 23:44

dhh requested a review from djmb October 18, 2024 22:17

djmb requested changes Oct 23, 2024

View reviewed changes

viktorianer mentioned this pull request Nov 24, 2024

Wildcard for 3rd level domains #1194

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support for custom SSL certificate #969

Added support for custom SSL certificate #969

kpumuk commented Sep 24, 2024 •

edited

Loading

yogeshjain999 commented Sep 30, 2024

kpumuk commented Sep 30, 2024 •

edited

Loading

yogeshjain999 commented Oct 1, 2024

kpumuk commented Oct 2, 2024

agu-z commented Oct 9, 2024

mtmckenna commented Oct 14, 2024

kpumuk commented Oct 14, 2024

mtmckenna commented Oct 14, 2024

kpumuk commented Oct 14, 2024

qinmingyuan commented Oct 17, 2024 •

edited

Loading

christo-ph commented Oct 22, 2024

qinmingyuan commented Oct 22, 2024 •

edited

Loading

djmb left a comment

djmb Oct 23, 2024

djmb Oct 23, 2024

djmb Oct 23, 2024

djmb Oct 23, 2024

kevinmcconnell Nov 26, 2024

Rukamakama commented Nov 12, 2024

junket commented Nov 26, 2024

Added support for custom SSL certificate #969

Are you sure you want to change the base?

Added support for custom SSL certificate #969

Conversation

kpumuk commented Sep 24, 2024 • edited Loading

yogeshjain999 commented Sep 30, 2024

kpumuk commented Sep 30, 2024 • edited Loading

yogeshjain999 commented Oct 1, 2024

kpumuk commented Oct 2, 2024

agu-z commented Oct 9, 2024

mtmckenna commented Oct 14, 2024

kpumuk commented Oct 14, 2024

mtmckenna commented Oct 14, 2024

kpumuk commented Oct 14, 2024

qinmingyuan commented Oct 17, 2024 • edited Loading

christo-ph commented Oct 22, 2024

qinmingyuan commented Oct 22, 2024 • edited Loading

djmb left a comment

Choose a reason for hiding this comment

djmb Oct 23, 2024

Choose a reason for hiding this comment

djmb Oct 23, 2024

Choose a reason for hiding this comment

djmb Oct 23, 2024

Choose a reason for hiding this comment

djmb Oct 23, 2024

Choose a reason for hiding this comment

kevinmcconnell Nov 26, 2024

Choose a reason for hiding this comment

Rukamakama commented Nov 12, 2024

junket commented Nov 26, 2024

kpumuk commented Sep 24, 2024 •

edited

Loading

kpumuk commented Sep 30, 2024 •

edited

Loading

qinmingyuan commented Oct 17, 2024 •

edited

Loading

qinmingyuan commented Oct 22, 2024 •

edited

Loading