The cool thing about GlusterFS is that it is a true cluster file share. It is installed on all nodes and makes all files available everywhere. So if any node crashes it does not affect the others.

But, this level of redundancy comes with a price. It is more difficult to maintain and it increases the needed storage space as the amount of nodes increase.

So I opted for the easy solution and switched to EFS.

You can read more about the hosting setup here. One problem with GlusterFS is that it needs to be installed on the docker host machine. It makes it much more difficult to scale the setup because we must take extra care when booting up a new host.

I would like my docker hosts to be 100% generic because that allows me to switch my setup to run inside ECS.

If we can replace GlusterFS with a generic NFS share, then we can remove this dependency. Lucky for me, Amazon provides their EFS system that allows us to create an NFS file share that we can mount directly into our docker containers. Removing all special configuration on the host machine.

EFS does have some drawbacks and should not be used for anything that is performance critical. The files in my NFS share consist of two types.

1. Media files, uploaded to Wordpress
2. PHP files for the installed plugins

The media files will be cached and served by a CDN so they should only be hit a few timed on the disk.

The PHP files are hit every time we need to generate a page view. Running application code filed from any network attached storage is a bad idea. A few notes here and here. The problem is that NFS is not built to be a low latency file system, making it unsuited to host application files.

Since the files do not change that often we benefit from a read cache. If we install cachefiled it should give us quite a boost. In the future, it would be better to bake the PHP files into the docker image, but that is an exercise for later.

Setting up EFS

EFS is made to be maintenance free. There are almost no options to choose from. It creates one filesystem with “infinite” space, and it can be mounted using standard NFS tools.

A wizard takes you through the whole process as shown below. It is suprisingly simple.

Selecting the VPC and security groups

When it is created you can mount it on your Linux machine with the standard mount command.

sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-xxx.efs.eu-west-1.amazonaws.com:/ efs

When it is mounted we can copy all files we need access to into the new mount point.

Docker and NFS

The standard local volume driver in docker supports NFS. That makes it easy to configure the volumes. The snippet below is from the docker swarm configuration file. In the bottom, we define the volumes with a connection to NFS, and setting them up on the services is standard.

version: "3.2"

services:
 php:
    image: 637345297332.dkr.ecr.eu-west-1.amazonaws.com/patch-php-fpm:latest
    build: php-fpm

    deploy:
      mode: global

    volumes:
      - wp_core:/var/www/datadriven-investment.com/:ro
      - datadriven_investment:/var/www/datadriven-investment.com/wp-content
  
volumes:
  wp_core:
    driver: local
    driver_opts:
      type: nfs
      o: addr=fs-xxx.efs.eu-west-1.amazonaws.com,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport
      device: fs-xxx.efs.eu-west-1.amazonaws.com:/patch_wp-core/_data/

  datadriven_investment:
    driver: local
    driver_opts:
      type: nfs
      o: addr=fs-xxx.efs.eu-west-1.amazonaws.com,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport
      device: fs-xxx.efs.eu-west-1.amazonaws.com:/patch_datadriven-investment-data/_data/

Pricing

With GlusterFS an Amazon EBS partition is needed for each host. With EBS the price is per provisioned Gb of storage per month. In my case, 2x 8Gb priced at $1,76 per month.

With EFS the pricing is based on the actual space used. My current space usage is 1,5Gb which cost $0,495 per month. A small saving, but it will be more expensive as my storage requirements grow.

Final remarks

It was suprisingly easy to set up EFS and connect it to Docker - thumbs up for that!

Some additional tips for EFS