correct aws cli syntax to find find a VPC security group in a non default VPC
amazon-web-services,amazon-ec2,amazon-vpc,aws-cli
The documentation says: --group-names (list) [EC2-Classic, default VPC] One or more security group names. So, it would seem that --group-names cannot be used on a non-default VPC. However, there are alternative methods: aws ec2 describe-security-groups --group-ids sg-xxxxxxxx aws ec2 describe-security-groups --filters Name=group-name,Values=MyVpcSecGroup To filter based on a specific VPC and...
Throttling S3 commands with aws cli
amazon-web-services,amazon-s3,aws-cli
I ended up using Trickle and capping download & upload speeds at 20,000 kb/s. This let me use my existing script without much modification (all I had to do was add the trickle call to the beginning of the command). Also, it looks like bandwidth throttling has been added as...
AWS CLI using Instance Profile Credentials
amazon-web-services,amazon-ec2,aws-cli
You are talking about IAM Roles. These are attached to the EC2 instance and the keys are rolled/rotated every four hours. You do not need to pull those from the instance metadata and supply it to the aws-cli or a SDK, they will pull it automatically. If you run aws-cli...
What is the correct syntax for filtering by tag in describe-vpcs?
amazon-web-services,amazon-ec2,amazon-vpc,aws-cli
You got pretty close to solving it -- the only problem is that you are not specifying a valid filter for describe-vpcs. Here's the filter that would be relevant to your use case: tag:key=*value* - The key/value combination of a tag assigned to the resource. So when it is asking...
Using troposphere for cloud formation, how do I add “propagate at launch” to tags
python,amazon-cloudformation,aws-cli,troposphere
---- UPDATE --- The feature has been added to the master branch, I just leave my previous answer for reference and in case you don't have access to the latest version of troposphere (ie if you don't clone the repository). You can still use the short function in your code...
AWS API Tools vs AWS CLI: ec2-import-instance
python,amazon-web-services,amazon-ec2,aws-cli,ec2-api-tools
It will be supported in a later release. As of now, there is no support for it. For more info...
What does the parameter 'statement-id' of 'add-permission' mean?
amazon-web-services,amazon-s3,aws-cli,aws-lambda
It's an arbitrary/opaque identifier that you essentially make up. In some situations, it's optional. In IAM policies, this is shown as the Sid. An example might be simply 1 or MyStringIMadeUp12345. It's typically used to describe or track the policy and has no meaning to AWS. Some services impose uniqueness...
What does the JSON for batch invalidation of a Cloudfront distribution look like?
amazon-web-services,amazon-cloudfront,aws-cli
You can generate sample JSON using the following command. $ aws cloudfront create-invalidation --generate-cli-skeleton | vi - Output: { "DistributionId": "", "InvalidationBatch": { "Paths": { "Quantity": 0, "Items": [ "" ] }, "CallerReference": "" } } ...
Launching Instances in Default VPC Produces Error
amazon-web-services,amazon-ec2,aws-cli
Security Groups are linked to specific VPCs. Check whether your Security Group is associated to a VPC that is different to the one in which you are launching the Amazon EC2 Instance (in this case, different to the Default VPC). If so, create a new Security Group in the Default...
Why does the AWS CLI pip package installation install a six package it can't use?
pip,aws-cli,aws-command-line,six
That was actually a bug in the awsebcli package's setup script that has now been fixed.
how to find the attached ebs volume for an instance
With the aws-cli, you could do this: aws ec2 describe-instances --instance-ids xxxxxxxx where xxxxxxx, is the instance id you are after. It will give you a lot more information that just the ebs volumne id, but it is in there for you to parse out....
ECS not an option in AWS CLI?
ubuntu,putty,aws-cli,amazon-ecs,amazon-ec2-windows
ECS is a relatively new service, and the version that apt-get gives us doesn't have that option yet. After apt-get installing awscli on ubuntu, the version is 1.2.9, whereas the one I downloaded from the site onto my machine is 1.7.36..... The aws ecs command works from my command prompt...
How to poll AWS CLI in shell script?
shell,amazon-web-services,aws-cli
AWS CLI has support for wait commands, those will block and wait for the condition you specify, such as waiting for an instance to be ready. The Advanced Usage of the AWS CLI talk from Re:Invent 2014 shows how to use waiters (18:55), queries, profiles and other tips for using...
Is there a way to shutdown and start an AWS redshift cluster with the cli?
bash,amazon-web-services,amazon-redshift,aws-cli
It sounds like you are looking for: delete-cluster, that explicitly specifies a final snapshot restore-from-cluster-snapshot, restoring the snapshot taken above From the aws-cli aws redshift delete-cluster documentation: If you want to shut down the cluster and retain it for future use, set SkipFinalClusterSnapshot to "false" and specify a name for...
How to use awscli inside python script?
python,amazon-web-services,amazon-ec2,aws-cli
The CLI would be more suited for the shell prompt, for a better python API, check the boto library. This example shows how to launch an instance: http://boto.readthedocs.org/en/latest/ec2_tut.html
How to use AWS S3 CLI to dump files to stdout in BASH?
dump the contents of all of the file objects to stdout. You can accomplish this if you pass - for destination of aws s3 cp command. For example, $ aws s3 cp s3://mybucket/stream.txt -. What you're trying to do is something like this? :: #!/bin/bash BUCKET=YOUR-BUCKET-NAME for key in...
Fetching CloudWatch metrics using the AWS Java SDK?
java,amazon-web-services,amazon-ec2,amazon-cloudwatch,aws-cli
I was missing the endpoint setting on the client. Working now. Changed this: private static AmazonCloudWatchClient client(final String awsAccessKey, final String awsSecretKey) { return new AmazonCloudWatchClient(new BasicAWSCredentials(awsAccessKey, awsSecretKey)); } to this: private static AmazonCloudWatchClient client(final String awsAccessKey, final String awsSecretKey) { final AmazonCloudWatchClient client = new AmazonCloudWatchClient(new BasicAWSCredentials(awsAccessKey, awsSecretKey));...
How do I add an array of RecordSets into Cloud Formation using troposphere?
python,amazon-web-services,amazon-cloudformation,aws-cli,troposphere
AliasTarget takes an AliasTarget helper class so you want to do it this way: from troposphere import GetAtt, Join, Template from troposphere.route53 import AliasTarget, RecordSetType, RecordSetGroup, RecordSet t = Template() myDNSRecord = t.add_resource(RecordSetGroup( "devdevopsdemoELBDNSARecord0", HostedZoneName=Join("", ["example.net", "."]), Comment="DNS Entry to point to the ELB for devopsdemo", RecordSets=[ RecordSet( Name="devopsdemo.dev.example.net.", Type="A",...
Getting “Name or service not known” error on every CLI opsworks command
python,amazon-web-services,aws-opsworks,aws-cli
You'll need to use the us-east-1 region when calling Opsworks commands. You can change this value via the --region argument or by setting a different default region when you run aws configure. From the FAQ: You can configure OpsWorks instances to be launched in any AWS region except GovCloud. OpsWorks...
How to use aws cli --generate-cli-skeleton
amazon-web-services,command-line-interface,aws-cli
This was added in version 1.5.5 (https://github.com/aws/aws-cli/blob/develop/CHANGELOG.rst#155), make sure your are using version 1.5.5 or higher. You can get the version via aws --version.
Why does AWS CLI need access/secret key?
amazon-web-services,aws-cli,aws-iam-roles
You don't need it. As the readme says: The quickest way to get started is to run the aws configure command: That doesn't mean it's the only way. For IAM roles, you don't need to do anything, it'll work correctly out of the box: The final option for credentials is...
AWS CLI and ECS query - HTTPSConnectionPool 443 error; Max retries exceeded; nodename nor servname provided, or not known
amazon-web-services,command-line-interface,aws-cli
The Amazon EC2 Container Service is currently only available in us-east-1. From: http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_GetStarted.html Customers accessing Amazon ECS during the preview should use the US East (N. Virginia) region. ...
Why do Amazon EMR clusters started with CLI not show up in the web console?
amazon-web-services,amazon-emr,aws-cli
Two things to check : Are you looking in the correct region? Maybe your CLI is starting the cluster in another region than the one you're looking at in the web console. If you are using different users between web console and CLI, are you using the ---visible-to-all-users option in...
AWS CLI : Cannot register a EC2 EBS-Backed Instance on Opsworks
amazon-ec2,aws-opsworks,aws-cli
Out of curiosity, did you follow the documentation on creating AMIs from OpsWorks instances when you made yours? There are certain things you need to do on the instance before you image it. I imagine that it could interfere with the opsworks init/bootstrap process. It seems the error you are...
Deploy AWS Elastic beanstalk to an environment in different region
git,amazon-web-services,elastic-beanstalk,aws-cli
To do it once: eb deploy {environment-name} --region {region-name} To always deploy to it: eb init --region {region-name} eb use {environment-name} Then use: eb deploy ...
Is there a way to avoid storing the AWS_SECRET_KEY on the .ebextensions?
I ended up removing the collecstatic step from the config file, and simply take care of uploading statics on the build side. After that, all credentials can be removed and all other boto commands will grab the credentials from the security role on the EC2 instance....
AWS CLI for EBS snapshots
amazon-web-services,amazon-ec2,aws-cli
Use us-west-2 instead of us-west-2b in your Endpoint URL. Most probably, this would be the value of REGION in your configuration file that was created by running aws configure us-west-2 is a region us-west-2b is an availability Zone in that region. You should be connecting to the Region and not...
How can multiple files be specified with “-files” in the CLI of Amazon for EMR?
hadoop,amazon-web-services,amazon-emr,aws-cli
For passing multiple files in a streaming step, you need to use file:// to pass the steps as a json file. AWS CLI shorthand syntax uses comma as delimeter to separate a list of args. So when we try to pass in parameters like: "-files","s3://betaestimationtest/mapper.py,s3://betaestimationtest/reducer.py", then the shorthand syntax parser...
Unable to Run AWS CLI: “ImportError: cannot import name shlex_quote”
Try reinstalling six via pip uninstall/pip install. It worked for me.
Launching EC Instance :: A client error (UnauthorizedOperation) occurred when calling the RunInstances operation
Do you know what this means? --iam-instance-profile Arn=arn:aws:iam::132895064974:instance-profile/ec2Admin If you are launching an EC2 instance with an IAM role, then you need to have IAM administrative privilege. It is discussed in detail here You can also try to decode the error message and see what it says: aws sts decode-authorization-message...
install redis on aws micro instance
amazon-web-services,redis,aws-cli
Rather than spin up an EC2 instance and install/manage redis there, you could create an Elasticache instance running redis and let AWS manage it all for you. If you really do want to run your own redis server then you'll want to launch an EC2 instance and then manually install...
Amazon AWS and tomcat non-stop(graceful) deployment
tomcat,amazon-web-services,deployment,suspend,aws-cli
What you're looking for is ELB connection draining. From the docs: "Connection draining causes the load balancer to stop sending new requests to the back-end instances when the instances are deregistering or when instances become unhealthy, while ensuring that in-flight requests continue to be served."
Cloudformation template validate
amazon-web-services,amazon-cloudformation,aws-cli
I knew how to fix it now. You need give the file full path with file:/// aws cloudformation validate-template --template-body file:///home/local/test/S3_Bucket.template ...
ElasticBeanstalk with Docker: how to use create-environment from aws cli
docker,elastic-beanstalk,aws-cli
I've not given this a shot but I assume there's a --solution-stack-name option where you can pass values such as 64bit Amazon Linux 2015.03 v1.4.1 running Docker 1.6.0. You can alternatively specify the solution stack on json file and specify the json file via --option-settings file://your_options.json where you can include...
aws command line interface - aws ec2 wait - Max attempts exceeded
amazon-web-services,amazon-ec2,aws-cli
So, finally, I used follow way to solve it: Create snapshot Use loop to check exit status of command aws ec2 wait snapshot-completed If exit status is not 0 then print current state, progress and run waiter again. # Create snapshot SNAPSHOT_DESCRIPTION="Snapshot of Primary frontend instance $(date +%Y-%m-%d)" SNAPSHOT_ID=$(aws ec2...
Filter S3 list-objects results to find a key matching a pattern
amazon-s3,pattern-matching,aws-cli,jmespath
The --query argument uses JMESPath expressions. JMESPath has an internal function contains that allows you to search for a string pattern. This should give the desired results: aws s3api list-objects --bucket myBucketName --query "Contents[?contains(Key, `mySearchPattern`)]" If you want to search for keys starting with certain characters, you can also use...
How to capture stderr to a variable if there is an error executing the script?
bash,amazon-web-services,stderr,aws-cli
myCmd=("aws --profile myProfile --region eu-west-1 ec2 authorize-security-group-ingress --group-id sg-999aa999 --protocol tcp --port 80 --cidr 0.0.0.0/0 ") if "${myCmd[@]}" > myJson.file 2> error.file; then echo ok else err="$(cat error.file)" # do domething with $err fi ...
Autoscaling: Instance starts and terminates abruptly [AWS]
amazon-web-services,autoscaling,aws-cli
When the instance comes online it must pass ELB health checks or it will be terminated by autoscaling. Does your launch configuration include userdata for instances to bootstrap themselves, or does the AMI your launching start up the web service automatically? If not, you need to configure this in order...