How to Run (go)Harbor Docker Registry with Notary Behind Nginx Reverse Proxy

That’s a heck of very specific title for a post, don’t you think? At the same I think that would be the most desirable configuration for running a private Docker reporsitory on prem, don’t you think? And for some ridiculous reason, there is virtually no information in the Internet about how to set it up like this.

The default Harbor configuration basically assumes that there will be a dedicated server running it, since by default it occupies ports 80 and 443. And there is very little meaningful explanation about how to make it coexist with other services running on your server, like, for example an nginx reverse proxy.

Your typical reverse proxy configuration looks like this: Client->Nginx(https)->Harbor(http), where nginx terminates SSL and then unencrypted traffic goes to Harbor’s custom HTTP port. In order to achieve this config, all you need to do is

• Set Harbor’s http port to an available port (something different from 80, since port 80 is occupied by your nginx reverse proxy)
• Comment out the entire https section of harbor.yml
• Set external_url in harbor.yml to the URL your reverse proxy will be providing
• Configure your reverse proxy to terminate SSL and forward port to harbor’s http port.

This is all great, but here is the problem: if you try to install Harbor with Notary as ./install.sh --with-notary, this won’t work, since Harbor’s installer requires Notary to work with https enabled in harbor.yml. You see the problem?

So here is the solution:

1. Configure SSL termination on your nginx reverse proxy, for example as described here: https://andrey.mikhalchuk.com/2020/05/14/stop-paying-for-ssl-certificates.html
2. Still, set Harbor’s port to something available on your server, let’s say 8080
3. Configure Harbor’s https section to run on an alternative port, say 8083
4. Configure Harbor’s https to use the same certificates you generated with Let’s Encrypt for your reverse proxy.
5. Set extenal_url in harbor.yml to the URL your reverse proxy will be providing
6. Configure your nginx reverse proxy to forward to harbor’s https port, not http.
7. Restart nginx, if necessary
8. Run harbor’s ./install.sh --with-notary
9. Everything should work now as you wanted

If the instructions above still sound confusing, here are the example config files written for an imaginary harbor server harbor.example.com (replace it with the real name of your server):

harbor.yml (only the relevant sections is shown

hostname: harbor.example.com # harbor's documentation says that it's ignored if external_url is set, but ./install.sh won't work without it

http:          # this section is actually irrelevant, but without it 
  port: 8080   # harbor will try to use port 80 and fail on start

https:
  port: 8083
  certificate: /certbot/fullchain.pem
  private_key: /certbot/privkey.pem

external_url: https://harbor.example.com

vhosts/harbor.example.com.conf

server {
    listen 80;
    location /.well-known/ {
      root /www/certbot;
    }
    location / {
      return 301 https://$host$request_uri:843;
    }
    server_name harbor.example.com;
}

server {
      listen       443 ssl http2;
      server_name harbor.example.com;
      client_max_body_size 10G; # docker images are huge!
      location / {
        proxy_set_header        X-Forwarded-Host $host;
        proxy_set_header        X-Forwarded-Server $host;
        proxy_set_header    	X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header    	X-Real-IP $remote_addr;
	proxy_set_header Host $host;
        proxy_pass https://127.0.0.1:8083; # note the https part
      }
    ssl_certificate /certbot/fullchain.pem; # same as in harbor.yml
    ssl_certificate_key /certbot/privkey.pem; # same as in harbor.yml
}

And the way it works is because nginx first terminates the SSL and then reencrypts it It’s funny that it does it using the same keys, which means this would be not really necessary, if only harbor’s install script didn’t have that check for https. But, oh well …