Amazon Cloudformation

It is a part of amazon that permit to create templates and reuse many times.

I have seen this interesting video and take some notes.
https://www.brighttalk.com/webcast/9019/105175
It is possible also download the slides

Some key points:

  • you use template and avoid to repete operations
  • integrate with your development tools
  • template in json format
  • you can place your template in s3 bucket or other locations
  • slides 52 - 53 reference can change at runtime
  • parameter from command lime or from web console, is possible to evaluate the parameters
  • slide 59 "conditional values" for example for difference region you can use specify different ami
  • slide 63 it is possible run a script inside the json code, run command, create files , control linux service
  • it si possible integrate with puppet or chef
  • slides 74 examples wordpress in multiregion environments in template and settings, HPC is super advance case use

Get Started

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/GettingStarted.Walkthrough.html
and this page http://www.devopscloud.com/05.01.html

Resource Type

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html

Some examples

if possible find complicate examples of things built with cloudformation, but to start you need simple and working example

Nested templates

https://blogs.aws.amazon.com/application-management/post/Tx1T9JYQOS8AB9I/Use-Nested-Stacks-to-Create-Reusable-Templates-and-Support-Role-Specialization

Best practice

http://harish11g.blogspot.ch/2014/08/amazon-cloudformation-templates-automation-Amazon-CFT-AWS-top-best-practices-tips.html

Complex Template Generation

troposphere - library to create AWS CloudFormation descriptions

from the readme page https://github.com/cloudtools/troposphere
The troposphere library allows for easier creation of the AWS CloudFormation JSON by writing Python code to describe the AWS resources. troposphere also includes some basic support for OpenStack resources via Heat.

To facilitate catching CloudFormation or JSON errors early the library has property and type checking built into the classes.

Terraform , multiproviders not only cloudformation

terraform

Monsanto with scala

in this post of Monsanto http://engineering.monsanto.com/2015/07/10/cloudformation-template-generator/ they explain why didn't choose Terraform to generate template https://www.terraform.io/ and instead used a scala tool available here https://github.com/MonsantoCo/cloudformation-template-generator

Advanced Parameters usage

https://blogs.aws.amazon.com/application-management/post/Tx3DV2UYG9SC38G/Using-the-New-CloudFormation-Parameter-Types

Functions Usage

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html

Some Examples

S3 examples

create an s3 bucket to host website with input parameters

{
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Description" : "This is a Description",

"Parameters" : {
    "BucketName" : {
        "Description" : "Please enter the Bucket Name",
        "Type": "String",
        "MinLength" : "5",
        "MaxLength" : "20"
    }
},

"Resources" : {
    "S3Bucket": {
        "Type" : "AWS::S3::Bucket",
        "Properties" : {
            "AccessControl" : "PublicRead",
               "BucketName" : { "Ref" : "BucketName" },
               "WebsiteConfiguration" : {
                   "ErrorDocument" : "error.html",
                "IndexDocument" : "index,html"
               }
           },
           "DeletionPolicy" : "Retain"
    }
},

"Outputs" : {
    "WebsiteURL" : {
        "Value" : { "Fn::GetAtt" : [ "S3Bucket", "WebsiteURL"]},
        "Description" : "Url on website on s3"
    },
    "S3BucketSecureURL" : {
        "Value" : {"Fn::Join" : [
        "",  [ "https://" , { "Fn::GetAtt": ["S3Bucket", "DomainName"] } ]
        ] },
        "Description" : "Name of s3 bucket to hold website content" 
    }
}

}

Elastick Beanstalk examples

important reference for the options http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options.html#command-options-general

Solutions Stake available http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.platforms.html

create an elastick beanstalk application (a container only)

{
  "AWSTemplateFormatVersion" : "2010-09-09",

  "Description" : "myfirstcloudformationtemplate",

  "Resources" : {

    "SampleApplication" : {
      "Type" : "AWS::ElasticBeanstalk::Application",
      "Properties" : {
        "Description" : "AWS Elastic Beanstalk Test PHP Sample Application",
        "ApplicationName" : "myfirstbeanstalk",
        "ApplicationVersions" : [{
          "VersionLabel" : "Initial Version",
          "Description" : "Version 1.0"
        }]
      }
    }
  }
}

create an elastick beanstalk application container and environment

{
  "AWSTemplateFormatVersion" : "2010-09-09",

  "Description" : "myfirstcloudformationtemplate",

  "Resources" : {

    "SampleApplication" : {
      "Type" : "AWS::ElasticBeanstalk::Application",
      "Properties" : {
        "Description" : "AWS Elastic Beanstalk Test PHP Sample Application",
        "ApplicationName" : "mysecondbeanstalk",
        "ApplicationVersions" : [{
          "VersionLabel" : "Initial Version",
          "Description" : "Version 1.0",
          "SourceBundle" : {
            "S3Bucket" : { "Fn::Join" : ["-", ["elasticbeanstalk-samples", { "Ref" : "AWS::Region" }]]},
            "S3Key" : "python-sample.zip"
          }
        }]
      }
    },

    "SampleEnvironment" : {
      "DependsOn" : ["SampleApplication"],
      "Type" : "AWS::ElasticBeanstalk::Environment",
      "Properties" : {
        "ApplicationName" : { "Ref" : "SampleApplication" },
        "CNAMEPrefix" : "myapppython-url",  
        "Description" :  "AWS Elastic Beanstalk Environment running Python Sample Application",
        "SolutionStackName" : "64bit Amazon Linux running Python",
        "EnvironmentName" :  "myappnpython-name",
        "Tier" : { "Name" : "WebServer", "Type" : "Standard", "Version" : "1.0" },
        "VersionLabel" : "Initial Version",
        "OptionSettings" : [
          {"Namespace" : "aws:autoscaling:launchconfiguration", "OptionName" : "EC2KeyName", "Value" : "mypersonalkeyname"}]
      }
    }
  }
}

certificate

to have the certificate you need to have this options

 {"Namespace" : "aws:elb:loadbalancer","OptionName" : "SSLCertificateId", "Value" : "arn:aws:iam::9876543210:server-certificate/mycertificatename"},

to obtain the correct arn for the certificate you need to run this command
aws iam get-server-certificate --server-certificate-name myname

to run this command you must have eight privileges

Elastick Beanstalk - CloudFormation and Tags

  • Tags are useful to give permissions to the resources and also to split the cost in the bills.
  • When you create a Elastick Benstalk from webconsole and from command line you can add tags , this tags are propagated to the ec2 instances generated by beanstalk.
  • When you use CloudFormation you can add tags to the template by webconsole or command line but these tags where not propagated to the content inside the template.
  • if you create an RDS from CloudFormation you can specify the tags like another options of the RDS, is the same for EC2 , but you can't do this for beanstalk.

Iam users

I have created the user from command line as is written here http://gborgese.wikidot.com/amazon-iam-management , with command line is not possible specify the password, but is possible specify the username
with cloudformation is possible specify the password but not the username , it will be generated automatically, probably for security reasons
as is written either here http://stackoverflow.com/questions/16185109/how-to-set-user-name-and-group-name-in-iam-using-cloudformation

{  
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Description" : "create a new User",

    "Resources" : {

        "createusers" : {
            "Type": "AWS::IAM::User",
               "Properties": {
                  "Groups": ["mygroupofspecialuser"],
                  "LoginProfile": { "Password" : "mysecretpwd-chooseyours" }
               }
        }
    }
}

the syntax is here http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iam-user.html but the field Path is not the username

RDS

simple examples
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-rds.html

spec
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-database-instance.html#cfn-rds-dbinstance-sourcedbinstanceidentifier

EC2 machine with cloud init

nice guide here http://answersforaws.com/episodes/4-user-data-cloud-init-cloudformation/
my example here

{
  "AWSTemplateFormatVersion" : "2010-09-09",

  "Description" : "the machine for monitor ",

  "Resources" : {
       "TabProxyMonitorIPAddress" : {
          "Type" : "AWS::EC2::EIP",
          "Properties" : {
            "Domain" : "vpc",
            "InstanceId" : { "Ref" : "TabProxyMonitor" }
          }
        },

        "TabProxyMonitor" : {
          "Type" : "AWS::EC2::Instance", 
          "Properties" : {
            "ImageId" : "ami-3d50120d",
            "KeyName" : "development",
            "InstanceType" : "t2.micro",
            "SubnetId" : "subnet-f0b97n5",
            "InstanceInitiatedShutdownBehavior" : "stop",
            "SecurityGroupIds" : [{ "Ref" : "TabProxyMonitorSecurityGroup" }],         
             "Tags" : [ { "Key" : "Name", "Value" : "TabProxyMonitor"},
                              { "Key" : "TEAM", "Value" : "developer"},
                             { "Key" : "OWNER", "Value" : "Giuiseppe"}  ],
             "UserData": {
                      "Fn::Base64": {
                        "Fn::Join": [
                          "\n",
                          [
                            "#!/bin/bash",
                            "apt-get update",
                            "apt-get upgrade -y",
                            "apt-get install apache2 libapache2-mod-proxy-html -y",
                            "a2enmod proxy",
                            "a2enmod proxy_connect",
                            "a2enmod proxy_http",
                            "a2enmod proxy_html",
                            "a2enmod ssl",
                            "a2enmod proxy_balancer",
                               "echo \"<VirtualHost *:80>\" > /etc/apache2/sites-available/000-default.conf",
                             "echo \"        DocumentRoot /var/www/html\" >> /etc/apache2/sites-available/000-default.conf",
                             "echo \"        ProxyPass               /tabamazon/admin/systeminfo       https://tab.mydom.com/admin/systeminfo\" >> /etc/apache2/sites-available/000-default.conf",
                               " echo \"       ProxyPassReverse        /tabamazon/admin/systeminfo       https://tab.mydom.com/admin/systeminfo\" >> /etc/apache2/sites-available/000-default.conf ",
                            " echo \"SSLProxyEngine on\" >> /etc/apache2/sites-available/000-default.conf",
                               "echo \"</VirtualHost>\" >> /etc/apache2/sites-available/000-default.conf",
                              "service apache2 restart"
                          ]
                        ]
                      }
               },
          "DisableApiTermination" : "True"
          }
        },

        "TabProxyMonitorSecurityGroup" : {
          "Type" : "AWS::EC2::SecurityGroup",
          "Properties" : {
            "GroupDescription" : "Enable access to the TabProxyMonitor host from all the world",
            "Tags" : [ { "Key" : "Name", "Value" : "TabProxyMonitor-sg"}],
             "VpcId" : "vpc-71648314",
            "SecurityGroupIngress" : [ { "IpProtocol" : "tcp", "FromPort" : "0",  "ToPort" : "65535",  "CidrIp" : "10.0.0.0/16"},
                                           { "IpProtocol" : "tcp", "FromPort" : "0",  "ToPort" : "65535",  "CidrIp" : "192.168.1.0/24"},
                                           { "IpProtocol" : "tcp", "FromPort" : "80",  "ToPort" : "80",  "CidrIp" : "0.0.0.0/0"},
                                           { "IpProtocol" : "icmp", "FromPort" : "-1",  "ToPort" : "-1",  "CidrIp" : "0.0.0.0/0"}]
          }
        }
    },

    "Outputs": {
        "Instance" : {
          "Description" : "IP Address of the logstash host",
          "Value" :  { "Ref" : "TabProxyMonitorIPAddress" }

        }
    }
 }

extract the private ip of an ec2 machine

"Outputs" : {
"Instance" : {
"Description" : "IP Address of the logstash host",
"Value" : { "Fn::GetAtt" : ["OpenDJInstance","PrivateIp"] }
},

Create a vpc with cloudformation

If you want create the full vpc with all networks with cloudformation there some things to keep in mind.

  1. if you want to do a vpc peering you need the approbation from the other side so if you put this in your template probably will fail
  2. the vpn connection require a manual step so you can't do this in the template

To resolve both 1 and 2 issues you can do these actions manually before

  1. create an empty vpc and choose the cidr
  2. create the peering inside the just create vpc
  3. create the vpn connection
  4. use the cloudformation template and pass as values the vpcid , cidr, vpcpeerid, vpnip

It is also important understand the relations between the network components, who refers who

AWS::EC2::RouteTable ==> AWS::EC2::Route ==> AWS::EC2::SubnetRouteTableAssociation ==> AWS::EC2::Subnet <== AWS::EC2::SubnetNetworkAclAssociation ==> AWS::EC2::NetworkAcl <== AWS::EC2::NetworkAclEntry

Codecommit

A repo with a prefix

AWSTemplateFormatVersion: "2010-09-09"
Parameters:
  RepoPrefix:
    Type: String
    Description: prefix of every repo created you need to pass the same parameter to the jenkins machine, examples prod OR dev OR test
Resources:
  BackupJenkinsJobs:
    Type: AWS::CodeCommit::Repository
    Properties:
      RepositoryName: !Join [ "_",  [ Ref: RepoPrefix , backup_jenkins_jobs ] ]
      RepositoryDescription: this is to have a in the repo the dir /var/lib/jenkins/jobs/

Parameters

You can read from this post https://aws.amazon.com/blogs/devops/using-the-new-cloudformation-parameter-types/ you have a way to choose by a dropdown list all the keys available for that region.
It query the infrastructure and fill up a list

"Parameters" : {
    "KeyName": {
      "Description": "Name of an existing EC2 KeyPair to enable SSH access to the instance",
      "Type": "AWS::EC2::KeyPair::KeyName",
      "ConstraintDescription": "must be the name of an existing EC2 KeyPair."
    }
  },
Salvo diversa indicazione, il contenuto di questa pagina è sotto licenza Creative Commons Attribution-ShareAlike 3.0 License