{"id":19485,"date":"2021-02-09T15:35:58","date_gmt":"2021-02-09T06:35:58","guid":{"rendered":"https:\/\/www.skyarch.net\/blog\/?p=19485"},"modified":"2021-02-10T09:04:03","modified_gmt":"2021-02-10T00:04:03","slug":"using-efs-in-aws-ecs-fargate","status":"publish","type":"post","link":"https:\/\/www.skyarch.net\/blog\/using-efs-in-aws-ecs-fargate\/","title":{"rendered":"Using EFS in AWS ECS Fargate"},"content":{"rendered":"<p>Amazon Web Services (AWS) <a href=\"https:\/\/aws.amazon.com\/blogs\/aws\/amazon-ecs-supports-efs\/\">announced last year<\/a> that Elastic Container Service (ECS) and Fargate now support Elastic File System (EFS). A <a href=\"https:\/\/aws.amazon.com\/about-aws\/whats-new\/2020\/08\/amazon-ecs-announces-cloudformation-support-for-amazon-efs-volumes\/\">follow-up announcement<\/a> on CloudFormation support was also made.<\/p>\n<h3>Preparation<\/h3>\n<p>To verify this feature, a mix of necessary AWS resources were created via CloudFormation and AWS console. Resources are as follow:<\/p>\n<h4>Network<\/h4>\n<ul>\n<li>VPC\n<ul>\n<li>Internet Gateway (IGW), its VPC attachment<\/li>\n<li>2 Public Subnets in 2 different Availability Zones<\/li>\n<li>Route Tables for the 2 subnets, with route to the IGW<\/li>\n<\/ul>\n<\/li>\n<li>Security Groups:\n<ul>\n<li>For the Fargate task and EC2 instance<\/li>\n<li>For the Elastic File System mounts<br \/>\n<strong>Note:<\/strong> It is important for the EFS security group to have an ingress rule that allows NFS connection via port 2049.<\/li>\n<\/ul>\n<\/li>\n<li>Identify and Access Management Roles\n<ul>\n<li>CodePipeline<\/li>\n<li>CodeBuild<\/li>\n<li>ECS Execution<\/li>\n<li>ECS Task<\/li>\n<\/ul>\n<p>All of these contain basic necessary permissions.<\/p>\n<\/li>\n<li>\n<p>EFS<\/p>\n<ul>\n<li>EFS instance<\/li>\n<li>2 Mount Targets for both subnets<\/li>\n<\/ul>\n<\/li>\n<li>ECR\n<ul>\n<li>Elastic Container Registry repository<\/li>\n<\/ul>\n<\/li>\n<li>ECS\n<ul>\n<li>ECS Cluster<\/li>\n<li>ECS\/Fargate log group<\/li>\n<li>Task Definition<\/li>\n<\/ul>\n<\/li>\n<li>CodeBuild\n<ul>\n<li>Log group<\/li>\n<li>CodeBuild Project<\/li>\n<\/ul>\n<\/li>\n<li>CodePipeline\n<ul>\n<li>S3 Buckets for deployment artifacts and source code<\/li>\n<li>CodePipeline itself<\/li>\n<\/ul>\n<\/li>\n<li>EC2\n<ul>\n<li>Amazon Linux 2 instance<\/li>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h4>General Flow<\/h4>\n<ol>\n<li>Upload zipped source codes to the S3 bucket source<\/li>\n<li>Wait for the CodeBuild to finish successfully<\/li>\n<li>Login to EC2 instance, mount EFS<\/li>\n<li>Run Task repeatedly<\/li>\n<\/ol>\n<h3>Specifics<\/h3>\n<p>This <a href=\"https:\/\/docs.aws.amazon.com\/AmazonECS\/latest\/developerguide\/tutorial-efs-volumes.html\">guide<\/a> was just generally followed. The most important addition is the CloudFormation support for Fargate to be able to connect to EFS. Below is a snippet of the Task Definition resource:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\nECSTaskDefinition:\n    Properties:\n        ContainerDefinitions:\n            .....\n            MountPoints:\n                - ContainerPath: \/efs\n                  SourceVolume: efs-test\n            .....\n        Volumes:\n            - EFSVolumeConfiguration:\n                FileSystemId:\n                  Ref: EFSFileSystem\n              Name: efs-test\n    Type: AWS::ECS::TaskDefinition\n<\/pre>\n<p>The important additions are <code>MountPoints<\/code> (under the Container Definition) and the <code>Volumes<\/code> sections.<\/p>\n<h3>Execution<\/h3>\n<p>First, login to the EC2 instance and mount the EFS. Instructions are <a href=\"https:\/\/docs.aws.amazon.com\/efs\/latest\/ug\/wt1-test.html\">here<\/a>. <code>common.txt<\/code> file was created under the mounted directory <code>\/mnt<\/code>:<\/p>\n<img decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2021\/02\/Screen-Shot-2021-02-09-at-10.40.43.png\" alt=\"EC2 Mount\" title=\"EC2 Mount\" \/>\n<p><code>common.txt<\/code> file was created to confirm whether the same file is accessible across multiple Fargate task instances.<\/p>\n<p>Next, run the tasks. For this documentation, a simple Dockerfile was prepared. It just lists the files in the EFS and adds a new one:<\/p>\n<p>FROM alpine:latest<br \/>\nRUN mkdir \/efs<br \/>\nCMD echo &#039;Files before: &#039; `ls \/efs` &amp;&amp; count=`ls \/efs | wc -l` &amp;&amp; echo &#039;Number of files before: &#039; $count &amp;&amp; touch \/efs\/$count.txt &amp;&amp; echo &#039;Number of files after: &#039; $count &amp;&amp; echo &#039;Files after touch: &#039; `ls \/efs` &amp;&amp; echo &#039;Accessing common.txt: &#039; `cat \/efs\/common.txt`<\/p>\n<p>Go over to the ECS Cluster via console and run the task. Be sure to choose the following:<\/p>\n<ul>\n<li><strong>Launch Type<\/strong>: Fargate<\/li>\n<li><strong>Task Definition<\/strong>: Your task definition and its latest revision<\/li>\n<li><strong>Platform Version<\/strong>: 1.4.0<\/li>\n<li><strong>VPC<\/strong>: Your VPC<\/li>\n<li><strong>Subnets<\/strong>: Either of the two subnets, or both<\/li>\n<li><strong>Security Group<\/strong>: The security group for the Fargate Task<\/li>\n<\/ul>\n<img decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2021\/02\/Screen-Shot-2021-02-09-at-15.12.57.png\" alt=\"Running the task\" title=\"Running the Task\" \/>\n<h4>Sample results<\/h4>\n<p>Sample execution results are as follow. The two tasks were run in two different subnets to ensure independence (see <code>Subnet Id<\/code> in the execution details).<\/p>\n<img decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2021\/02\/Screen-Shot-2021-02-09-at-15.53.09.png\" alt=\"Execution 1\" title=\"Execution 1\" \/>\n<img decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2021\/02\/Screen-Shot-2021-02-09-at-15.53.49.png\" alt=\"Execution 1\" title=\"Execution 1\" \/>\n<img decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2021\/02\/Screen-Shot-2021-02-09-at-15.55.09.png\" alt=\"Execution 2\" title=\"Execution 2\" \/>\n<img decoding=\"async\" src=\"https:\/\/www.skyarch.net\/blog\/wp-content\/uploads\/2021\/02\/Screen-Shot-2021-02-09-at-15.55.33.png\" alt=\"Execution 2\" title=\"Execution 2\" \/>\n<p>With these, it was shown that Fargate tasks running in different subnets can access the same EFS volume just like EC2 instances.<\/p>\n<h3>Sources<\/h3>\n<ul>\n<li>https:\/\/aws.amazon.com\/blogs\/aws\/amazon-ecs-supports-efs\/<\/li>\n<li>https:\/\/aws.amazon.com\/about-aws\/whats-new\/2020\/08\/amazon-ecs-announces-cloudformation-support-for-amazon-efs-volumes\/<\/li>\n<li>https:\/\/docs.aws.amazon.com\/AmazonECS\/latest\/developerguide\/tutorial-efs-volumes.html<\/li>\n<li>https:\/\/docs.aws.amazon.com\/efs\/latest\/ug\/wt1-test.html<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Amazon Web Services (AWS) announced last year that Elastic Container Service (ECS) and Fargate now support Ela&#8230;<\/p>\n","protected":false},"author":128,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_locale":"ja","_original_post":"https:\/\/www.skyarch.net\/blog\/?p=19485","footnotes":""},"categories":[20,131,515,19],"tags":[102,303,130,1018],"class_list":{"0":"post-19485","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-aws","7":"category-efs","8":"category-fargate","9":"category-19","10":"tag-aws","11":"tag-ecs","12":"tag-efs","13":"tag-fargate","14":"ja"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/19485","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/users\/128"}],"replies":[{"embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/comments?post=19485"}],"version-history":[{"count":19,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/19485\/revisions"}],"predecessor-version":[{"id":19515,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/posts\/19485\/revisions\/19515"}],"wp:attachment":[{"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/media?parent=19485"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/categories?post=19485"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.skyarch.net\/blog\/wp-json\/wp\/v2\/tags?post=19485"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}