Using Lambda Layers to share lambda code from different functions
https://medium.com/the-cloud-architect/getting-started-with-aws-lambda-layers-for-python-6e10b1f9a5d
Develop a lambda in your laptop using https://github.com/lambci/docker-lambda
Reference for python api http://boto3.readthedocs.io/en/latest/reference/core/session.html
if you call in this way , you don't have the problem of the ssl certificate verification
client = boto3.client('autoscaling',verify=False,region_name='eu-west-1')
but an annoing message is printed every time you do a call
Some notes
- An interesting page with all the mistakes (and how to fix it) that you can on with Lambda http://www.iheavy.com/2016/02/14/getting-errors-building-amazon-lambda-python-functions-help-howto/
- nice article http://blog.contino.io/5-killer-use-cases-for-aws-lambda with some use cases , there are also some others services based on lambda to simplify the usability I think https://serverless.com/ http://apex.run/ http://gosparta.io/
Using external library
- in this page there is the guide for import an external library in labda http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html
- you need to put the library downloaded like is showed in the how to in the same directory at the same level of your function
ls
lambda_function.py requests/ requests-2.11.1.dist-info/
- after you need to zip the content of the directory in a file zip and upload
With pip3
pip3 install --target=. requests
Collecting requests
Using cached https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-
........
Installing collected packages: chardet, urllib3, idna, certifi, requests
Successfully installed certifi-2018.11.29 chardet-3.0.4 idna-2.8 requests-2.21.0 urllib3-1.24.1
compiling-dir> ll
total 8
drwxr-xr-x 3 giuseppe staff 96B Jan 18 17:27 bin
drwxr-xr-x 7 giuseppe staff 224B Jan 18 17:27 certifi
drwxr-xr-x 10 giuseppe staff 320B Jan 18 17:27 certifi-2018.11.29.dist-info
drwxr-xr-x 43 giuseppe staff 1.3K Jan 18 17:27 chardet
drwxr-xr-x 10 giuseppe staff 320B Jan 18 17:27 chardet-3.0.4.dist-info
drwxr-xr-x 11 giuseppe staff 352B Jan 18 17:27 idna
drwxr-xr-x 8 giuseppe staff 256B Jan 18 17:27 idna-2.8.dist-info
-rw-r--r-- 1 giuseppe staff 2.1K Jan 18 17:03 package.py
drwxr-xr-x 21 giuseppe staff 672B Jan 18 17:27 requests
drwxr-xr-x 8 giuseppe staff 256B Jan 18 17:27 requests-2.21.0.dist-info
drwxr-xr-x 16 giuseppe staff 512B Jan 18 17:27 urllib3
drwxr-xr-x 8 giuseppe staff 256B Jan 18 17:27 urllib3-1.24.1.dist-info
now that your code and all the library are in the same dir it is possible do a .zip
I have also resolved the error with homebrew creating a file in ~/.pydistutils.cfg
[install]
prefix=
Using the Environment Variables in a function
this is a simple example with Python
import os
def lambda_handler(event, context):
return os.environ['PREFIX']
IAM permissions to run/exec/invoke only the functions
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"lambda:GetEventSourceMapping",
"lambda:GetFunction",
"lambda:GetFunctionConfiguration",
"lambda:GetPolicy",
"lambda:InvokeFunction",
"lambda:InvokeAsync",
"lambda:ListEventSourceMappings",
"lambda:ListFunctions"
],
"Resource": "*"
}
]
}
Linuxacademy course
Following the linuxacademy course I created the first lambda function with node.js
An useful resource is the sdk for node js http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html
Steps to create:
- Have an s3 bucket in the same region of the lambda function
- from inside lambda create a trigger for any creation object in that bucket
- use the example of node.js get s3-get-object
- after the upload is possible to see the logs of the action and also the cloudwatch with the output.
the python environment for lambda is python2.7
some useful code
print("Memory limit: ", context.memory_limit_in_mb
print("Request ID: ", context.aws_request_id )
Grunt , testing NODE.JS lambda code locally
The repos for this project is https://github.com/Tim-B/grunt-aws-lambda
Emulambda , testing PYTHON lambda code locally
Using the repository https://github.com/fugue/emulambda you can have a way to test the code locally
installation
cd ~
git clone https://github.com/fugue/emulambda
sudo /usr/local/bin/pip install -e emulambda
create a simple test that print a line
- create a basic python function
cat firstfunction.py
import boto3
def first_handler(event, context):
print ("Test Inside the handler")
return "Ok return"
- and an empty json file
cat event.json
{}
- run the code with -v you will have more info
$ emulambda firstfunction.first_handler event.json
Test Inside the handler
Ok return
$ emulambda -v firstfunction.first_handler event.json
Test Inside the handler
Executed firstfunction.first_handler
Estimated...
...execution clock time: 0ms (100ms billing bucket)
...execution peak RSS memory: 36B (36 bytes)
----------------------RESULT----------------------
Ok return
Examples
Disable and enable the autoscaling Launch Event
import boto3
import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def lambda_handler(event, context):
howmanydisable = 0
regionstoscan = ['us-east-1','eu-west-1','us-west-2']
for region in regionstoscan:
logger.info("region: " + region + "___________")
client = boto3.client('autoscaling',region_name=region)
response = client.describe_auto_scaling_groups()
for group in response['AutoScalingGroups']:
groupname=group['AutoScalingGroupName']
logger.info(groupname)
#response = client.suspend_processes(AutoScalingGroupName=groupname, ScalingProcesses=['Launch',])
#response = client.resume_processes(AutoScalingGroupName=groupname, ScalingProcesses=['Launch',])
disableLaunch = True
for tag in group['Tags']:
if (tag['Key'] == 'EnableDuringNight') & (tag['Value'] == 'true'):
disableLaunch = False
if disableLaunch:
howmanydisable = howmanydisable + 1
response = client.resume_processes(AutoScalingGroupName=groupname, ScalingProcesses=['Launch',])
logger.info(groupname + "will have a disable launch acction")
print '--------------------------'
return "Total reenable: " + str(howmanydisable)
call a function with parameter from command line
aws lambda invoke --invocation-type RequestResponse --function-name myfunctioname --region eu-west-2 --log-type Tail --payload {"par1":"myfirstparameter", "par2": "mysecondparameter"} outputfile.txt
inside the func
import time
import requests
import simplejson as json
from requests.auth import HTTPDigestAuth
def lambda_handler(event, context):
parone = event['par1']
partwo = event['par2']
Codecommit trigger lambda
this is the event
{
"Records": [
{
"eventId": "5a824061-17ca-46a9-bbf9-114edeadbeef",
"eventVersion": "1.0",
"eventTime": "2016-01-01T23:59:59.000+0000",
"eventTriggerName": "my-trigger",
"eventPartNumber": 1,
"codecommit": {
"references": [
{
"commit": "5c4ef1049f1d27deadbeeff313e0730018be182b",
"ref": "refs/heads/master"
}
]
},
"eventName": "TriggerEventTest",
"eventTriggerConfigId": "5a824061-17ca-46a9-bbf9-114edeadbeef",
"eventSourceARN": "arn:aws:codecommit:us-east-1:0123456789:my-repo",
"userIdentityARN": "arn:aws:iam::0123456789:root",
"eventSource": "aws:codecommit",
"awsRegion": "us-east-1",
"eventTotalParts": 1
}
]
}
IAM permisssions so your lambda function can write on cloudwatch
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": "arn:aws:logs:us-east-1:YOUR_ACCOUNT_ID:*"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:us-east-1:YOUR_ACCOUNT_ID:log-group:/aws/lambda/YOUR_LAMBDA_NAME:*"
]
}
]
}
total volumes size
import boto3
def lambda_handler(event, context):
client = boto3.client('ec2',"eu-west-1")
response = client.describe_volumes()
vol = response['Volumes']
total = 0
for v in vol :
total = total + v['Size']
return total
Run Lambda function regularly like cron
Keep a ec2 machine always up
- no need external libraries
- increase the timeout
- 2 external variables defined
import boto3
import time
import logging
import os
logger = logging.getLogger()
logger.setLevel(logging.INFO)
region = os.environ['REGION']
ec2id = os.environ['EC2ID']
def lambda_handler(event, context):
client = boto3.client('ec2',region_name=region)
response = client.describe_instances(InstanceIds=[ec2id])
status = response['Reservations'][0]['Instances'][0]['State']['Name']
responsestart = "no action performed"
logger.info("Status is: " + status)
if status == 'running':
logger.info("running system nothing to do")
if status == 'pending':
logger.info("pending system nothing to do")
if status == 'stopped':
logger.info("stop system, run a start action")
responsestart = client.start_instances(InstanceIds=[ec2id])
if status == 'stopping':
logger.info("wait a minute and run a start action")
time.sleep(45)
responsestart = client.start_instances(InstanceIds=[ec2id])
logger.info(responsestart)
Policy to apply to the role
- ec2 full access
- allow cloudwatch login
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
}