Using EFS in AWS ECS Fargate

Amazon Web Services (AWS) announced last year that Elastic Container Service (ECS) and Fargate now support Elastic File System (EFS). A follow-up announcement on CloudFormation support was also made.

Preparation

To verify this feature, a mix of necessary AWS resources were created via CloudFormation and AWS console. Resources are as follow:

Network

• VPC
  • Internet Gateway (IGW), its VPC attachment
  • 2 Public Subnets in 2 different Availability Zones
  • Route Tables for the 2 subnets, with route to the IGW
• Security Groups:
  • For the Fargate task and EC2 instance
  • For the Elastic File System mounts
    Note: It is important for the EFS security group to have an ingress rule that allows NFS connection via port 2049.
• Identify and Access Management Roles
  • CodePipeline
  • CodeBuild
  • ECS Execution
  • ECS Task

  All of these contain basic necessary permissions.

• EFS

  • EFS instance
  • 2 Mount Targets for both subnets
• ECR
  • Elastic Container Registry repository
• ECS
  • ECS Cluster
  • ECS/Fargate log group
  • Task Definition
• CodeBuild
  • Log group
  • CodeBuild Project
• CodePipeline
  • S3 Buckets for deployment artifacts and source code
  • CodePipeline itself
• EC2
  • Amazon Linux 2 instance

General Flow

1. Upload zipped source codes to the S3 bucket source
2. Wait for the CodeBuild to finish successfully
3. Login to EC2 instance, mount EFS
4. Run Task repeatedly

Specifics

This guide was just generally followed. The most important addition is the CloudFormation support for Fargate to be able to connect to EFS. Below is a snippet of the Task Definition resource:

ECSTaskDefinition:
    Properties:
        ContainerDefinitions:
            .....
            MountPoints:
                - ContainerPath: /efs
                  SourceVolume: efs-test
            .....
        Volumes:
            - EFSVolumeConfiguration:
                FileSystemId:
                  Ref: EFSFileSystem
              Name: efs-test
    Type: AWS::ECS::TaskDefinition

The important additions are MountPoints (under the Container Definition) and the Volumes sections.

Execution

First, login to the EC2 instance and mount the EFS. Instructions are here. common.txt file was created under the mounted directory /mnt:

common.txt file was created to confirm whether the same file is accessible across multiple Fargate task instances.

Next, run the tasks. For this documentation, a simple Dockerfile was prepared. It just lists the files in the EFS and adds a new one:

FROM alpine:latest
RUN mkdir /efs
CMD echo 'Files before: ' ls /efs && count=ls /efs | wc -l && echo 'Number of files before: ' $count && touch /efs/$count.txt && echo 'Number of files after: ' $count && echo 'Files after touch: ' ls /efs && echo 'Accessing common.txt: ' cat /efs/common.txt

Go over to the ECS Cluster via console and run the task. Be sure to choose the following:

• Launch Type: Fargate
• Task Definition: Your task definition and its latest revision
• Platform Version: 1.4.0
• VPC: Your VPC
• Subnets: Either of the two subnets, or both
• Security Group: The security group for the Fargate Task

Sample results

Sample execution results are as follow. The two tasks were run in two different subnets to ensure independence (see Subnet Id in the execution details).

With these, it was shown that Fargate tasks running in different subnets can access the same EFS volume just like EC2 instances.

Sources

• https://aws.amazon.com/blogs/aws/amazon-ecs-supports-efs/
• https://aws.amazon.com/about-aws/whats-new/2020/08/amazon-ecs-announces-cloudformation-support-for-amazon-efs-volumes/
• https://docs.aws.amazon.com/AmazonECS/latest/developerguide/tutorial-efs-volumes.html
• https://docs.aws.amazon.com/efs/latest/ug/wt1-test.html