gem-hunter
Language: Go
The Amazon ECS CLI enables users to run their applications on ECS/Fargate using the Docker Compose file format, quickly provision resources, push/pull images in ECR, and monitor running applications on ECS/Fargate.
✨ The AWS Copilot CLI is now in Generally Available! The CLI makes it easy to build, release and operate your container apps on Amazon ECS and AWS Fargate.
Learn more about the AWS Copilot CLIThe AWS Copilot CLI is a CLI focused on the full developer experience of building, deploying and operating your containerized apps. From helping manage all of your infrastructure, to setting up CD Pipelines, copilot is here to help. To learn more about AWS Copilot, check out the documentation.
The Amazon ECS Command Line Interface (CLI) is a command line tool for Amazon Elastic Container Service (Amazon ECS) that provides high-level commands to simplify creating, updating, and monitoring clusters and tasks from a local development environment. The Amazon ECS CLI supports Docker Compose, a popular open-source tool for defining and running multi-container applications. Use the CLI as part of your everyday development and testing cycle as an alternative to the AWS Management Console or the AWS CLI.
For more information about Amazon ECS, see the Amazon ECS Developer Guide.
The AWS Command Line Interface (AWS CLI) is a unified client for AWS services that provides commands for all public API operations. These commands are lower level than those provided by the Amazon ECS CLI. For more information about supported services and to download the AWS CLI, see the AWS Command Line Interface product detail page.
If you think you’ve found a potential security issue, please do not post it in the Issues. Instead, please follow the instructions here or email AWS security directly at aws-security@amazon.com.
Download the binary archive for your platform, and install the binary on your $PATH.
You can use the provided md5 hash to verify the integrity of your download.
For information about installing and using the Amazon ECS CLI, see the ECS Command Line Interface.
As of v0.6.2 the ECS CLI supports the cn-north-1 region in China. The following links are the exact same binaries, but they are localized within China to provide a faster download experience.
Using the URLs above, replace latest with the desired tag, for example v1.0.0. After
downloading, remember to rename the binary file to ecs-cli.
NOTE: Windows is only supported starting with version v1.0.0.
NOTE: ARM64 is only supported starting with version v1.20.0.
If you wish to verify your ECS CLI download, you can use the PGP Signatures.
Install gpg using the package manager on your flavor of linux.
One easy way is to use Homebrew, a package manager for OS X. Install Homebrew using the instructions on its site.
brew install gnupg brew install amazon-ecs-cli
Go to the GnuPG download page and download the simple installer for Windows. Use the installer to install the GPG tool.
You can find the Public Key in our GitHub Repo, in the file amazon-ecs-public-key.gpg.
gpg --import amazon-ecs-public-key.gpg
Key Metadata:
ECS CLI signatures are ascii armored detached PGP signatures stored in files with the extension ".asc". The signatures file will have the same name as its corresponding executable with ".asc" appended. In the
curl -o ecs-cli.asc https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-darwin-amd64-latest.asc
curl -o ecs-cli.asc https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-linux-amd64-latest.asc
curl -o ecs-cli.asc https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-linux-arm64-latest.asc
PS C:\> Invoke-WebRequest -OutFile ecs-cli.asc https://amazon-ecs-cli.s3.amazonaws.com/ecs-cli-windows-amd64-latest.exe.asc
Assuming you installed the ECS CLI in the recommended location for your platform:
gpg --verify ecs-cli.asc /usr/local/bin/ecs-cli
gpg --verify ecs-cli.asc C:\Program Files\Amazon\ECSCLI\ecs-cli.exe
Expected output:
gpg: Signature made Tue Apr 3 13:29:30 2018 PDT
gpg: using RSA key DE3CBD61ADAF8B8E
gpg: Good signature from "Amazon ECS <ecs-security@amazon.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: F34C 3DDA E729 26B0 79BE AEC6 BCE9 D9A4 2D51 784F
Subkey fingerprint: EB3D F841 E2C9 212A 2BD4 2232 DE3C BD61 ADAF 8B8E
The warning in the output is expected and is not problematic; it occurs because there is not a chain of trust between your personal PGP key (if you have one) and the Amazon ECS PGP key. For more information, learn about the Web of trust.
The Amazon ECS CLI requires some basic configuration information before you can use it, such as your
AWS credentials, the AWS region in which to create your cluster, and the name of the Amazon ECS
cluster to use. Configuration information is stored in the ~/.ecs directory on macOS and Linux
systems and in C:\Users\<username>\AppData\local\ecs on Windows systems.
The Amazon ECS CLI supports configuring multiple sets of AWS credentials as named profiles using the
ecs-cli configure profile command. These profiles can then be referenced when you run Amazon ECS
CLI commands using the --ecs-profile flag; if a custom profile is not specified, the default
profile will be used.
Set up a CLI profile with the following command, substituting profile_name with your desired
profile name, and $AWS_ACCESS_KEY_ID, $AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables with your
AWS credentials.
ecs-cli configure profile --profile-name profile_name --access-key $AWS_ACCESS_KEY_ID --secret-key $AWS_SECRET_ACCESS_KEY --session-token AWS_SESSION_TOKEN
A cluster configuration is the set of fields that describes an Amazon ECS cluster, including the
name of the cluster and the region. These configurations can then be referenced when you run Amazon
ECS CLI commands using the --cluster-config flag; otherwise, the default configuration is used.
Create a cluster configuration with the following command, substituting region_name with your
desired AWS region, cluster_name with the name of an existing Amazon ECS cluster or a new cluster
to use, and configuration_name with the name you'd like to give this configuration.
ecs-cli configure --cluster cluster_name --region region_name --config-name configuration_name
You can also optionally add --default-launch-type to your cluster configuration. This value will
be used as the launch type for tasks run in this cluster (see: Launching an AWS Fargate
Task) , and will also be used to determine which resources to
create when you bring up a cluster (see: Creating a Fargate Cluster).
Valid values for this field are EC2 or FARGATE. If not specified, ECS will default to EC2 launch
type.
The first Cluster Configuration or ECS Profile that you configure will be set as the default. The
default ECS Profile can be changed using the ecs-cli configure profile default command; the
default cluster configuration can be changed using the ecs-cli configure default command. Note
that unlike in the AWS CLI, the default ECS Profile does not need to be named "default".
The --aws-profile flag and $AWS_PROFILE environment variable allow you to reference any named profile in ~/.aws/credentials.
Here is an example on how to assume a role: amazon-ecs-cli/blob/master/ecs-cli/modules/config/aws_credentials_example.ini
If you are trying to use Multi-Factor Authentication, please see this comment and the associated issue: #284 (comment).
For more information, see ECS CLI Configuration.
ECS now offers two different launch types for tasks and services: EC2 and FARGATE. With the FARGATE launch type, customers no longer have to manage their own container-instances.
In the ECS-CLI, you can specify either launch type when you bring up a cluster using the
--launch-type flag (see: Creating an ECS Cluster). You can also
configure your cluster to use a particular launch type with the --default-launch-type flag (see:
Cluster Configurations).
You can also specify which launch type to use for a task or service in compose up or compose service up, regardless of which launch type is configured for your cluster (see: Starting/Running
Tasks).
After installing the Amazon ECS CLI and configuring your credentials, you are ready to create an ECS cluster. The basic command for creating a cluster is:
ecs-cli up
(To see all available options, run ecs-cli up --help)
For example, to create an ECS cluster with two Amazon EC2 instances using the EC2 launch type, use the following command:
$ ecs-cli up --keypair my-key --capability-iam --size 2
It takes a few minutes to create the resources requested by ecs-cli up. To see when the cluster
is ready to run tasks, use the AWS CLI to confirm that the ECS instances are registered:
$ aws ecs list-container-instances --cluster your-cluster-name
{
"containerInstanceArns": [
"arn:aws:ecs:us-east-1:980116778723:container-instance/6a302e06-0aa6-4bbc-9428-59b17089b887",
"arn:aws:ecs:us-east-1:980116778723:container-instance/7db3c588-0ef4-49fa-be32-b1e1464f6eb5",
]
}
In addition to EC2 Instances, other resources created by default include:
You can provide your own resources (such as subnets, VPC, or security groups) via their flag options.
Note: The default security group created by ecs-cli up allows inbound traffic on port 80 by
default. To allow inbound traffic from a different port, specify the port you wish to open with the
--port option. To add more ports to the default security group, go to EC2 Security Groups in
the AWS Management Console and search for the security group containing “ecs-cli”. Add a rule as
described in the Adding Rules to a Security Group
topic.
Alternatively, you may specify one or more existing security group IDs with the --security-group option.
You can also create an empty ECS cluster by using the --empty or --e flag:
ecs-cli up --cluster myCluster --empty
This is equivalent to the create-cluster command, and will not create a CloudFormation stack associated with your cluster.
You can specify the AMI to use with your EC2 instances using the --image-id flag. Alternatively, if you do not specify an image ID, the ECS CLI will use the recommended Amazon Linux 2 ECS Optimized AMI. By default, the x86 variant of this AMI is used. However, if you specify an instance in the A1 family using --instance-type, then the arm64 version of the ECS Optimized AMI will be used. Note: arm64 ECS Optimized AMIs are only supported in some regions; please see Amazon ECS-Optimized Amazon Linux 2 AMI.
For the EC2 launch type, the ECS CLI always creates EC2 instances that include the following User Data:
#!/bin/bash
echo ECS_CLUSTER={ clusterName } >> /etc/ecs/ecs.config
This user data directs the EC2 instance to join your ECS Cluster. You can optionally include extra user data with --extra-user-data; this flag takes a file name as its argument.
The flag can be used multiple times to specify multiple files. Extra user data can be shell scripts or cloud-init directives- see the EC2 documentation for more information.
The ECS CLI takes all the User Data, and packs it into a MIME Multipart archive which can be used by cloud-init on the EC2 instance. The ECS CLI even allows existing MIME Multipart archives to be passed in with --extra-user-data.
The CLI will unpack the existing archive, and then repack it into the final archive (preserving all header and content type information). Here is an example of specifying extra user data:
ecs-cli up \
--capability-iam \
--extra-user-data my-shellscript \
--extra-user-data my-cloud-boot-hook \
--extra-user-data my-mime-multipart-archive \
--launch-type EC2
$ ecs-cli up --launch-type FARGATE
This will create an ECS Cluster without container instances. By default, this will create the following resources:
The subnet and VPC ids will be printed to the terminal once the creation is complete. You can then use the subnet IDs in your ECS Params file to launch Fargate tasks.
For more information on using AWS Fargate, see the ECS CLI Fargate tutorial.
After the cluster is created, you can run tasks – groups of containers – on the ECS cluster. First, author a Docker Compose configuration file. You can run the configuration file locally using Docker Compose. Information about specific compose versions and fields supported by the ecs-cli can be found here.
Here is an example Docker Compose configuration file that creates a web page:
version: '2'
services:
web:
image: amazon/amazon-ecs-sample
ports:
- "80:80"
To run the configuration file on Amazon ECS, use ecs-cli compose up. This creates an ECS task
definition and starts an ECS task. You can see the task that is running with ecs-cli compose ps,
for example:
$ ecs-cli compose ps
Name State Ports TaskDefinition
fd8d5a69-87c5-46a4-80b6-51918092e600/web RUNNING 54.209.244.64:80->80/tcp web:1
Navigate your web browser to the task’s IP address to see the sample app running in the ECS cluster.
You can also run tasks as services. The ECS service scheduler ensures that the specified number of tasks are constantly running and reschedules tasks when a task fails (for example, if the underlying container instance fails for some reason).
$ ecs-cli compose --project-name wordpress-test service create
INFO[0000] Using Task definition TaskDefinition=wordpress-test:1
INFO[0000] Created an ECS Service serviceName=wordpress-test taskDefinition=wordpress-test:1
You can then start the tasks in your service with the following command:
$ ecs-cli compose --project-name wordpress-test service start
It may take a minute for the tasks to start. You can monitor the progress using the following command:
$ ecs-cli compose --project-name wordpress-test service ps
Name State Ports TaskDefinition
34333aa6-e976-4096-991a-0ec4cd5af5bd/wordpress RUNNING 54.186.138.217:80->80/tcp wordpress-test:1
34333aa6-e976-4096-991a-0ec4cd5af5bd/mysql RUNNING wordpress-test:1
See the $ ecs-cli compose service documentation page for more information about available service options, including load balancing.
Since there are certain fields in an ECS task definition that do not correspond to fields in a
Docker Composefile, you can specify those values using the --ecs-params flag. Currently, the file
supports the follow schema:
version: 1
task_definition:
ecs_network_mode: string // Supported string values: none, bridge, host, or awsvpc
task_role_arn: string
task_execution_role: string // Needed to use Cloudwatch Logs or ECR with your ECS tasks
task_size: // Required for running tasks with Fargate launch type
cpu_limit: string
mem_limit: string // Values specified without units default to MiB
pid_mode: string // Supported string values: task or host
ipc_mode: string // Supported string values: task, host, or none
services:
<service_name>:
essential: boolean
depends_on:
- container_name: string // <service_name> of any other service in services
condition: string // Valid values: START | COMPLETE | SUCCESS | HEALTHY
repository_credentials:
credentials_parameter: string
cpu_shares: integer
firelens_configuration:
type: string // Supported string values: fluentd or fluentbit
options: list of strings
mem_limit: string // Values specified without units default to bytes, as in docker run
mem_reservation: string
gpu: string
init_process_enabled: boolean
healthcheck:
test: string or list of strings
interval: string
timeout: string
retries: integer
start_period: string
logging:
secret_options:
- value_from: string
name: string
secrets:
- value_from: string
name: string
docker_volumes:
- name: string
scope: string // Valid values: "shared" | "task"
autoprovision: boolean // only valid if scope = "shared"
driver: string
driver_opts:
string: string
labels:
string: string
efs_volumes:
- name: string
filesystem_id: string
root_directory: string
transit_encryption: string // Valid values: "ENABLED" | "DISABLED" (default). Required if
// IAM is enabled or an access point ID is
// specified
transit_encryption_port: int64 // required if transit_encryption is enabled
access_point: string
iam: string // Valid values: "ENABLED" | "DISABLED" (default). Enable IAM
// authentication for FS access.
placement_constraints:
- type: string // Valid values: "memberOf"
expression: string
run_params:
network_configuration:
awsvpc_configuration:
subnets: array of strings // These should be in the same VPC and Availability Zone as your instance
security_groups: list of strings // These should be in the same VPC as your instance
assign_public_ip: string // supported values: ENABLED or DISABLED
task_placement:
strategy:
- type: string // Valid values: "spread"|"binpack"|"random"
field: string // Not valid if type is "random"
constraints:
- type: string // Valid values: "memberOf"|"distinctInstance"
expression: string // Not valid if type is "distinctInstance"
service_discovery:
container_name: string
container_port: integer
private_dns_namespace:
id: string
name: string
vpc: string
description: string
public_dns_namespace:
id: string
name: string
service_discovery_service:
name: string
description: string
dns_config:
type: string
ttl: integer
healthcheck_custom_config:
failure_threshold: integer
Version Schema version being used for the ecs-params.yml file. Currently, we only support version 1.
Task Definition
Fields listed under task_definition correspond to fields that will be included in your ECS Task Definition.
ecs_network_mode corresponds to NetworkMode on an ECS Task Definition (Not to be confused with the network_mode field in Docker Compose). Supported values are none, bridge, host, or awsvpc. If not specified, this will default to bridge mode. If you wish to run tasks with Network Configuration, this field must be set to awsvpc.
task_role_arn should be the ARN of an IAM role. NOTE: If this role does not have the proper permissions/trust relationships on it, the up command will fail.
services correspond to the services listed in your docker compose file, with service_name matching the name of the container you wish to run. Its fields will be merged into an ECS Container Definition.
essential field is not specified, the value defaults to true.depends_on field maps to dependsOn parameter in task definition. It allows you to specify a list of ContainerDependency, which can be used for conditional startup of dependent containers or ensuring order of startup between containers. Refer example.cpu_shares, mem_limit, and mem_reservation fields are optional and must be specified in the ECS params file rather than the compose file.cpu_shares, mem_limit, and mem_reservation fields can be specified in either the compose or ECS params file. If they are specified in the ECS params file, the values will override values present in the compose file.repository_credentials allows you to specify an AWS Secrets Manager secret ARN for the name of the secret containing your private repository credentials as a credential_parameter.init_process_enabled is a Linux-specific option that can be be set to run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run. This parameter requires version 1.25 of the Docker Remote API or greater on your container instance.firelens_configuration contains configuration parameters for Firelens.
type Valid options are fluentbit or fluentdoptions Please see the AWS docs for Firelensgpu is the number of physical GPUs the Amazon ECS container agent will reserve for the container. Maps to the GPU resource requirement field in the task definition. For example: "1", "4", "8", "16".healthcheck This parameter maps to healthcheck in the Docker compose file reference. This field can either be used here in the ECS Params file, or it can be used in Compose File version 3 with the ECS CLI.
test can also be specified as command and must be either a string or a list or strings. If test is specified as a list of strings, the first item must be either NONE, CMD, or CMD-SHELL. If test or command is specified as a string, CMD-SHELL will be prepended and ECS will run the command in the container's default shell.interval, timeout, and start_period are specified as durations in a string format. For example: 2.5s, 10s, 1m30s, 2h23m, or 5h34m56s.secrets allows you to specify secrets which will be retrieved from SSM Parameter Store. See the ECS Docs for more information, including how reference AWS Secrets Managers secrets from SSM Parameter Store.
value_from is the SSM (or Secrets Manager) Parameter ARN or name (if the parameter is in the same region as your ECS Task).name is the name of the environment variable in which the secret will be stored.secret_options under logging. For more information, See the logging secrets section of the ECS docs.
value_from is the SSM (or Secrets Manager) Parameter ARN or name (if the parameter is in the same region as your ECS Task).name is the name of the logging option in which the secret will be stored.docker_volumes allows you to create docker volumes. The name key is required, and scope, autoprovision, driver, driver_opts and labels correspond with the fields under dockerVolumeConfiguration in an ECS Task Definition. Volumes defined with the docker_volumes key can be referenced in your compose file by name, even if they were not also specified in the compose file.
efs_volumes allows you to mount EFS volumes to your container. The name and EFS filesystem ID are required. EFS volumes can be referenced by name in your compose file like docker_volumes.
task_execution_role should be the ARN of an IAM role. NOTE: This field is required to enable ECS Tasks to be configured with Cloudwatch Logs, or to pull images from ECR for your tasks.
task_size Contains two fields, CPU and Memory. These fields are required for launching tasks with Fargate launch type. See the documentation on ECS Task Definition Parameters for more information.
placement_constraints allows you to specify a list of constraints on task placement within the task definition. Not supported with the FARGATE launch type.
pid_mode allows you to control the process namespace in which your containers run. Valid values are task or host. See the ECS documentation for more information.
ipc_mode allows you to control the IPC resource namespace in which your containers run. Valid values are task, host, or none. See the ECS documentation for more information.
Run Params
Fields listed under run_params are for values needed as options to API calls not related to a Task Definition, such as compose up (RunTask) and compose service up (CreateService).
Currently, the only parameter supported under run_params is network_configuration. This is required to run tasks with Task Networking, as well as with Fargate launch type.
network_configuration is required if you specify ecs_network_mode as awsvpc. It takes one nested parameter, awsvpc_configuration, which has three subfields:
subnets: list of subnet ids used to launch tasks. NOTE These should be in the same VPC and availability zone as the instances on which you wish to launch your tasks.security_groups: list of securtiy-group ids used to launch tasks. NOTE These should be in the same VPC as the instances on which you wish to launch your tasks.assign_public_ip: supported values for this field are either "ENABLED" or "DISABLED". This field is only used for tasks launched with Fargate launch type. If this field is present in tasks with network configuration launched with EC2 launch type, the request will fail.task_placement is an optional field with EC2 launch-type only (it is not valid for FARGATE). It has two subfields:
strategy: A list of objects, with two keys. Valid keys are type