Amazon CodeCommit - CodeDeploy - CodePipeline - CodeBuild - DataPipeline

CodeBuild

DataPipeline

Run commands from Data Pipeline
https://aws.amazon.com/premiumsupport/knowledge-center/stop-start-ec2-instances/

from the linuxacademy course

Code Commit

Configuration

Config the ssh

  • install the aws cli configure and choose a region in this case we have choosent the default one us-east-1
  • generate a ssh key , in this example we choose as name codecommit
  • go in the your IAM user and add the the public key in the "Security Credentials" => "SSH keys for AWS CodeCommit" and copy the Access Key ID
  • in the tab "Permissions" add the policy "AWSCodeCommitFullAccess"
  • you need to have this conf inside your config file and have the right permissions on it
[ec2-user@ip-172-31-33-137 ~]$ cat .ssh/config
Host git-codecommit.*.amazonaws.com
        User AAAAAAA2NHT5VZYKEQ
        IdentityFile ~/.ssh/codecommit

chmod 600 .ssh/config
  • test if it is working this is an example of good success
[ec2-user@ip-172-31-33-137 ~]$ ssh git-codecommit.us-east-1.amazonaws.com
Warning: Permanently added the RSA host key for IP address '54.239.20.155' to the list of known hosts.
You have successfully authenticated over SSH. You can use Git to interact with AWS CodeCommit. Interactive shells are not supported.Connection to git-codecommit.us-east-1.amazonaws.com closed by remote host.
Connection to git-codecommit.us-east-1.amazonaws.com closed.

Configure git

git config --global credential.helper '!aws --profile codecommit credial-helper $@'
git config --global credential.UseHttpPath true

check if the conf is correct with the command

git config --global --edit

Commands CLI

create

 aws codecommit create-repository --repository-name CLIREPO --repository-description "a repos created by command line"
{
    "repositoryMetadata": {
        "repositoryName": "CLIREPO",
        "cloneUrlSsh": "ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/CLIREPO",
        "lastModifiedDate": 1472824875.592,
        "repositoryDescription": "a repos created by command line",
        "cloneUrlHttp": "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/CLIREPO",
        "creationDate": 1472824875.592,
        "repositoryId": "2f046f4b-82aa-4b81-8030-202872fcf793",
        "Arn": "arn:aws:codecommit:us-east-1:492852102645:CLIREPO",
        "accountId": "492852102645"
    }
}

list
aws codecommit list-repositories
{
    "repositories": [
        {
            "repositoryName": "CLIREPO",
            "repositoryId": "2f046f4b-82aa-4b81-8030-202872fcf793"
        },
        {
            "repositoryName": "testgiuseppe",
            "repositoryId": "863e8a88-5991-45b8-858d-1ca997415fbe"
        }
    ]
}
[ec2-user@ip-172-31-33-137 ~]$ aws codecommit get-repository --repository-name CLIREPO
{
    "repositoryMetadata": {
        "repositoryName": "CLIREPO",
        "cloneUrlSsh": "ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/CLIREPO",
        "lastModifiedDate": 1472824875.592,
        "repositoryDescription": "a repos created by command line",
        "cloneUrlHttp": "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/CLIREPO",
        "creationDate": 1472824875.592,
        "repositoryId": "2f046f4b-82aa-4b81-8030-202872fcf793",
        "Arn": "arn:aws:codecommit:us-east-1:492852102645:CLIREPO",
        "accountId": "492852102645"
    }
}

 aws codecommit batch-get-repositories --repository-names CLIREPO testgiuseppe
{
    "repositories": [
        {
            "repositoryName": "CLIREPO",
            "cloneUrlSsh": "ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/CLIREPO",
            "lastModifiedDate": 1472824875.592,
            "repositoryDescription": "a repos created by command line",
            "cloneUrlHttp": "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/CLIREPO",
            "creationDate": 1472824875.592,
            "repositoryId": "2f046f4b-82aa-4b81-8030-202872fcf793",
            "Arn": "arn:aws:codecommit:us-east-1:492852102645:CLIREPO",
            "accountId": "492852102645"
        },
        {
            "repositoryName": "testgiuseppe",
            "cloneUrlSsh": "ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/testgiuseppe",
            "lastModifiedDate": 1472824523.284,
            "repositoryDescription": "this is only a first repos test",
            "cloneUrlHttp": "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/testgiuseppe",
            "creationDate": 1472824523.284,
            "repositoryId": "863e8a88-5991-45b8-858d-1ca997415fbe",
            "Arn": "arn:aws:codecommit:us-east-1:492852102645:testgiuseppe",
            "accountId": "492852102645"
        }
    ],
    "repositoriesNotFound": []
}

Git commands

git status -sb

git push remote-name branch-name

# to know differences  
git diff --stat origin/master

#from the master run the merge
git checkout master
git merge new_feature 

#show all tags
git tag

#create tag
git tag tag-name commit-id

#delete a tag
git tag -d tag-name

#dettailed info about tags
git show tag-name

#info about all the commit
git log

# create a tag and push remote
git tag v5.0 e603acb
git push origin v5.0

# show all the remotes tags
git ls-remote --tags

# delete the tag from remote 
git push --delete origin v5.0

Triggers

Which events can invoke a trigger

  • all repository events
  • a push to an existing branch
  • the creation/deletion of a branch or a tag

Triggers can invoke actions on the following aws services:

  • aws SNS (this means also possibility to integrate with AWS SQS ) or send emails to other developers
  • aws Lambda

Code Deploy

Editing application

  • you can edit an application name only from cli or api not from web console
  • instead if you want delete you can do also from web console

Custom Deployment Configuration

  • There are 3 possibility to deploy new software, once in a time, half in a time, one machine a time. If you want another options you can do by api or by cli but not from the web console

here at example by host , you can also choose a custom percent %

aw deploy create-deployment-config --deployment-config-name MyCustomConfig --minimum-healthy-hosts type=HOST_COUNT,value=1

after this command you will have 4 ways to deploy instead of 3. You can use the new options from the web console , of course you can't delete one of the 3 default type.

AppSepc File

  • permessions apply only to linux and it is optional
permissions:
 - object: /wonderwidgets/html 
   pattern: "*.html"
   except: "indext.thml"
   mode: 400
   type:
     - file
  • there are many moments where you can run scripts
  • in the hook section you can access to some environment variables from inside the script
AfterInstall:
   - location: myfile.sh
     timeout: 180
  • if you don't specify the exact number of spaces the deploy will fails, you can use an yaml validator like this http://www.yamllint.com/
  • the yaml file needs to call appspec.yml and must be in the root directory of our source file

Push revision on S3

  • we have used codecommit for our sources but at the moment June 2016 it is not possible use this like bundle repos, so we need to upload to an s3 bucket

this command is automatically zip the code

aws deploy push --application-name TestApplication --description "my test deploymnet" --ignore-hidden-files --s3-location s3://wonderwidgets-deployment-test/wonderwidgets.zip --source .

it is important ignore hidden files because we are commiting from a git repos.

Code Pipeline

  1. Continuos Delivery: push the software in automatic way from development in production
  2. Continuos Integration: merging working copies to a share mainline

Pipeline: a workflow framework
Stages: are the actions. The deafult are source, build, beta, stages are connected to transictions

Introduction

The code pipeline can take a source from

  • Amazon s3
  • Github
  • Amazon Code Commit (only if you are in Virginia because code commit at the moment it is only available in Virginia)

Use a deploy provider already configured to put in place the new code in machines, there are 3 deploy providers

  • Jenkins
  • Solano
  • Amazon Code Deploy

The pipeline orchestrate all the steps

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License