Macadamian Blog

Creating a Private Docker Registry

Christian Nadeau

In order to use Rancher, we wanted to host our own Docker registry. Rancher provides a tutorial to do just that, however, we had a couple extra requirements that we go over here, to help you control the services that will route the registry.

Private Docker Registry

We wanted to be able to host our own docker registry in order to use it with Rancher. There was a very nice post by them on how to do it, but we wanted to have a bit more control over the services that will route the actual registry.

Here is the full set of requirements we had:

  • Uses an SSL certificate
  • Is password protected
  • Is hosted on Azure
  • Has a simple UI to browse the images

But first thing first: let’s stand up a very simple registry with a UI!

NOTE: The reference material for this article can be found here.

The Services Definition

Here is the template of the docker-compose.yml file for a basic local docker registry:

version: '2'
services:
 lb:
   image: dockercloud/haproxy:1.6.2
   links:
     - registry
     - registry-ui
   ports:
     - '80:80'
     - '443:443'
     - '5000:5000'
   restart: always
   volumes:
     - /var/run/docker.sock:/var/run/docker.sock
registry:
   build: ./registry
   restart: always
   expose:
     - 5000
   environment:
     TCP_PORTS: '5000'
     VIRTUAL_HOST: '*:5000, https://*:5000'
     FORCE_SSL: 'true'
     REGISTRY_STORAGE_DELETE_ENABLED: 'true'
registry-ui:
   image: konradkleine/docker-registry-frontend:v2
   restart: always
   environment:
     VIRTUAL_HOST: '*, https://*'
     ENV_DOCKER_REGISTRY_HOST: 'registry'
     ENV_DOCKER_REGISTRY_PORT: 5000
   links:
     - registry
   expose:
     - 80

Services

HAProxy

This service is the load balancer. The only thing we had to do, is to bind port 5000 in order to redirect the traffic to the registry.

Registry

This service is the docker registry. A lot of configuration was required:

  • Set the TCP_PORTS and VIRTUAL_HOST environment variable
    • This is required for HAProxy to redirect all traffic from port 5000 to this service
  • Set registry service specific environment variables:
    • – REGISTRY_STORAGE_DELETE_ENABLED=true: otherwise, the registry does not support deleting images

Registry UI (Docker Registry Frontend)

This service hosts a very simple docker UI name docker-registry-frontend by Konrad Kleine (thanks a lot!). In this service, not so much was required to be configured:

  • Set the VIRTUAL_HOST environment variable
    • This is required for HAProxy to redirect all traffic (not already taken care of ) to this service
  • Set the registry-ui service specific environment variables:
    • ENV_DOCKER_REGISTRY_HOST=registry : name of the service for which a link exists
    • ENV_DOCKER_REGISTRY_PORT=5000 : the port on which the registry listens to

How to start it

To start the registry locally, simply run this command:

docker-compose up -d

The registry is reachable at localhost:5000.

The registry UI is reachable http://localhost:80.

IMPORTANT NOTES – The registry is:

  1. Running locally
  2. Not using any authentication mechanism
  3. Storing docker images in the container only.
  4. If you want to persist it for some reason, add this volume to the registry service definition
  5. Not using SSL
volumes:
 ./local_registry_backup:/var/lib/registry

How to validate it works

Pull a known small image:

docker pull alpine:3.4

Tag that image to point to your local registry

NOTE: there is an optional username parameter that can be added if you want to name your image.

docker tag alpine:3.4 localhost:5000/<optional-username>/alpine:3.4

Push the image to your registry

docker push localhost:5000/<optional-username>/alpine:3.4

Validate it’s available in your registry UI by navigating to: http://localhost:80

Delete the image from your local docker images:

docker rmi localhost:5000/<optional-username>/alpine:3.4

You should not have any local docker images tagged with your registry now. To validate it:

docker images | grep localhost:5000

Fetch the image from your private registry:

docker fetch localhost:5000/<optional-username>/alpine:3.4

You’re good to go, the very basic registry is up and running!

Adding Basic Authentication

Now that we have a basic registry up and running locally, let’s configure the basic authentication.

The Services Definition

The docker-compose command allows you to stack docker-compose.yml files to override some services. Those are the overrides for the basic registry created above.

version: '2'
services:
 registry:
   environment:
     REGISTRY_AUTH: 'htpasswd'
     REGISTRY_AUTH_HTPASSWD_REALM: 'YOUR_DOCKER_REGISTRY_REALM'
     REGISTRY_AUTH_HTPASSWD_PATH: '/httpasswd_storage/htpasswd'
   volumes:
     - ~/htpasswd_backup:/httpasswd_storage

Service Overrides

The registry was overridden to set environment variables:

  • REGISTRY_AUTH=htpasswd : sets the authentication method to htpasswd (basic auth)
  • REGISTRY_AUTH_HTPASSWD_REALM: “YOUR REALM” : the Realm for your docker registry
  • REGISTRY_AUTH_HTPASSWD_PATH: ‘/httpasswd_storage/htpasswd’ : the full path to the htpasswd files containing your user:pass associations. This file will be shared between the host running your service and the service itself using the volumes definition

Generating the htpasswd file

This is how you can add a simple user to a local htpasswd file in ~/htpasswd_backup, which is the one configured in the previous example, using docker.

#Create the htpasswd_backup
mkdir -p ~/htpasswd_backup
docker run --rm --entrypoint htpasswd registry:2 -Bbn <username> "<password>" > ~/htpasswd_backup/htpasswd

How to start it

To start the registry locally, simply run this command:

docker-compose -f docker-compose.yml \
              -f docker-compose.auth.yml \
              up -d

The registry is reachable at localhost:5000.

The registry UI is reachable http://localhost:80, but you’ll be asked for a password.

IMPORTANT NOTES – At this stage the registry is:

  1. Running locally
  2. Authenticated using basic auth
  3. Storing docker images in the container only
  4. If you want to persist it for some reason, add this volume to the registry service definition,
  5. Not using SSL

How to validate it works

Try to pull the image you pushed in the basic registry:

docker pull localhost:5000/<optional-username>/alpine:3.4

You will receive an error:

Pulling repository localhost:5000/<optional-username>/alpine

Error: image <optional-username>/alpine:3.4 not found

This means the authentication works! Let’s authenticate:

docker login -u <optional-username> localhost:5000

You’ll be asked for your password, then you will be authenticated.

Try to pull the image again and it will succeed.

If you want to logout, run this command:

docker logout localhost:5000

You now have a registry with authentication!

Let’s Use Azure Storage

Now that we have a basic registry up and running locally using authentication, let’s configure the storage to use an Azure Storage Account.

The Services Definition

The docker-compose command will allow you to stack docker-compose.yml files to override some services. Those are the overrides for the basic authenticated registry created above.

version: '2'
services:
 registry:
   environment:
     # To make sure default filesystem storage is
     # deleted from default config
     REGISTRY_STORAGE: azure
     REGISTRY_STORAGE_AZURE_ACCOUNTNAME: 'STORAGE-ACCOUNT-NAME'
     REGISTRY_STORAGE_AZURE_ACCOUNTKEY: 'STORAGE-ACCESS-KEY'
     REGISTRY_STORAGE_AZURE_CONTAINER: 'CONTAINER-NAME'

Service Overrides

The registry was overridden to set environment variables:

  • REGISTRY_STORAGE=azure: The environment variables are overriding the basic configuration as mentioned here. To make sure the default filesystem is overridden properly, we set this environment variable to azure. Otherwise, you’ll end up with filesystem
  • REGISTRY_STORAGE_AZURE_ACCOUNTNAME=“STORAGE_ACCOUNT_NAME”: the Azure Storage Account name
  • REGISTRY_STORAGE_AZURE_ACCOUNTKEY: “STORAGE_ACCESS_KEY”: The access key for the storage account you are using

How to start it

To start the registry locally, simply run this command:

docker-compose -f docker-compose.yml \
              -f docker-compose.auth.yml \
              -f docker-compose.azure.yml \
              up -d

The registry is reachable at localhost:5000.

The registry UI is reachable http://localhost:80, you’ll be asked for a password.

IMPORTANT NOTES – At this stage the registry is:

  1. Running locally
  2. Authenticated using basic auth
  3. Storing all images in the Azure Storage Account you specified
  4. Not using SSL

How to validate it works

At first, in your local registry UI, you won’t see any images. Even pulling, once authenticated, won’t work because your Azure Storage Account does not contain it.

Try pushing the same image as above:

docker push localhost:5000/<optional-username>/alpine:3.4

Once it’s done, you’ll be able to see the image on your storage account from the Azure Portal or using the Azure Storage Explorer.

Let’s Secure The Registry

Now that we have a registry with authentication and storing docker images to Azure Storage, we need to get this server out on its own, meaning we need to secure it with SSL.
NOTE: We’ll base the haproxy and letsencrypt services on this previous article on using let’s encrypt and HAProxy with Docker.

The Services Definition

Important Assumption: You are running those docker commands from (or using docker-machine) a machine publicly accessible by the domain you want to get SSL certificates for.

version: '2'
services:
 haproxy:
   image: m21lab/haproxy:1.6.2
   links:
     - letsencrypt
   volumes_from:
     - letsencrypt
letsencrypt:
   image: m21lab/letsencrypt:1.0
   environment:
     DOMAINS: 'YOUR_DOMAIN'
     EMAIL: 'RECOVERYEMAIL@YOUR_DOMAIN'
     OPTIONS: '--staging'
registry:
   volumes_from:
     - letsencrypt:ro
   environment:
     REGISTRY_HTTP_SECRET: "your-http-secret"
     REGISTRY_HTTP_TLS_CERTIFICATE: /etc/letsencrypt/live/LETSENCRYPT_DOMAINS_ENV_VAR/fullchain.pem
     REGISTRY_HTTP_TLS_KEY: /etc/letsencrypt/live/LETSENCRYPT_DOMAINS_ENV_VAR/privkey.pem
registry-ui:
   environment:
     FORCE_SSL: 'true'
     ENV_REGISTRY_PROXY_FQDN: 'LETSENCRYPT_DOMAINS_ENV_VAR'
     ENV_DOCKER_REGISTRY_USE_SSL: 1

Service Overrides

HAProxy

Uses a different image which will be handling the SSL traffic.

Links to the new letsencrypt service to route the challenges http requests to it.

Uses volumes_from letsencrypt service to read the received SSL certificates and dynamically load them.

Let’s Encrypt
This service is used to generate valid CA signed SSL certificates for the domain hosting the registry. More information in this previous article.

Registry
This service is the docker registry. A lot of configuration was required:

  • Uses volumes_from letsencrypt service to read the received SSL certificates and dynamically load them
  • Set registry service specific environment variables:
    • REGISTRY_HTTP_SECRET=secret: A randomly generated secret used to sign state that may be stored. More information about it here.
    • REGISTRY_HTTP_TLS_CERTIFICATE=public key
    • REGISTRY_HTTP_TLS_KEY=private key:
      Those must be mapped to the letsencrypt service volume

Registry UI (Docker Registry Frontend)
This service hosts a very simple docker UI named docker-registry-frontend by Konrad Kleine (thanks a lot!). In this service, not so much was required to be configured.
Set the registry-ui environment variables:

  • FORCE_SSL=true: To make sure all the traffic redirected from the haproxy service is using SSL
  • ENV_REGISTRY_PROXY_FQDN=“DOMAIN”: The domain name you want to be displayed in the registry UI.
  • ENV_DOCKER_REGISTRY_USE_SSL=1: The registry must now be accessed using https

How to start it

To start the registry, from your domain accessible machine or using docker-machine to point your local docker environment to a domain accessible machine, simply run this command:

docker-compose -f docker-compose.yml \
              -f docker-compose.auth.yml \
              -f docker-compose.azure.yml \
              -f docker-compose.ssl.yml \
              up -d

The registry is reachable at DOMAIN:5000.

The registry UI is reachable https://DOMAIN, you’ll be asked for a password.

IMPORTANT NOTES – At this stage the registry is:

  1. Running on your domain
  2. Authenticated using basic auth
  3. Storing all images in the Azure Storage Account you specified
  4. Using SSL

How to validate it works

Since we already pushed an image to the local repository once we setup the Azure Storage Account, this image is still there. We can pull it using the domain instead of localhost to login/pull/logout:

docker login -u <optional-username> DOMAIN:5000
# enter your password
docker pull DOMAIN:5000/<optional-username>/alpine:3.4
docker logout DOMAIN:5000

We’re done! We hope this article helped you. You should now have the registry running on your domain, authenticated using basic auth, storing all images in your Azure storage account and using SSL.

Insights delivered to your inbox

Subscribe to the dev blog to get the latest insights in IoT, Alexa Skills development, and software development.

Author Overview

Christian Nadeau

Christian is a veteran software developer at Macadamian with specialties in .NET ( WPF, Silverlight, Window Phone 8 ), java (J2EE, JBoss), C++ (Qt, BB10). He holds a bachelor's degree in computer engineering from the University of Sherbrooke.
  • Jimmy

    Thanks for the awesome writeup. Do you know if it is possible to use an existing SSL cert for everything in lieu of letsencrypt?

    • Christian Nadeau

      Of course it’s possible 🙂

      Here is what I suggest:

      – Convert your cert to a PEM format (if not already)
      – Copy your cert over on the VM hosting your registry
      – In the lb service configuration in the docker-compose.yml
      – create a volume mapping your cert.pem to a file within your container
      – defined the CERT_FOLDER

      lb:
      volumes:
      – :/etc/ssl/private/cert.pem:ro
      environment:
      CERT_FOLDER: /etc/ssl/private/cert.pem

      Hope this helps!

  • Will Lopez

    Hey man thanks for this great post. I am new to docker containers and got stuck on the last part setting up the SSL. It looks like the letsencrypt container is not able to find the loadbalancer. I see this message in the logs of the letsencrypt:

    Loadbalancer not up yet, waiting 5 second
    nc: bad address ‘lb’

    Any ideas on to what could be the problem? thanks in advance.

    Here are my docker compose files: https://gist.github.com/willopez/068b43916d27fe5983ae87fce9f41a0d