It would have been great if you would have also discussed about versioning the lambda functions for different stages. So that we can publish different code to different stages.
great article, it works awesome. one question how to put variable values in the lambda code (sam template). like if the same .yml needs deployment to different environments then value of things like environment variables of lambda would have different values for different environment deployments. how do we have this kind of thing achieved ? we can have different branches deployed to different environments but few values would need changing while deployment.
Hello Prasad - I have security scenario for this architecture as how to secure the requests coming from the internet traffic preventing the backend getting exploited. Three scenarios here:- 1) if http request coming to api gateway and how can you secure at the gateway level. 2) if http request from the salesforce authenticating which gives saml, how do we secure and integrate with API gateway. 3) if http request coming from OAM auth and how do we secure and integrate it with API gateway.
Very good video! Can you also save the manual configuration you did for setting up IAM and CodeBuild in the AWS Console in a SAM or CloudFormation template?
@Prasad, Really appreciate all your videos and they are very helpful! One question regarding the python dependencies. If I have pip installed additional dependencies such as `googlemaps` in a local virtual environment, how should I include this additional dependency in the deployment process with CodePipeline and CloudFormation?
I know I’m late to respond but you would package the dependencies as Lamda Layer and then deploy the Lambda Layer as an additional resource in your template
Thanks Prasad for this video! I have a question though. If my python code has additional dependencies and it requires a virtual env for python to be deployed, how should I make use of code build here or if you can suggest a better way? Or will Sam template take care of it somehow? I was wondering if I have to use Sam build first?
Thank you for the tutorial. I have a doubt actually... In the online policy json file there's this line "Resource": [ "arn:aws:s3:::codepipeline*" ] Here, what is meant by codepipeline? Is it the S3 bucket name to store the artifacts after build step.... Do I need to change it with the name of my S3 bucket?
Hi Prasad I have 50 number of lambda functions in one git repo with python lambda functions and node js lambda functions , how can we achive codepipeline in aws in this scenario , please let me know .
Expecting appspec file at location /home/ec2-user/appspec.yml but it is not found there. Please either run the CLI from within a directory containing the appspec.yml file or specify a bundle location containing an appspec.yml file in its root directory
I have my dotnet core application artifacts that should be packaged and deployed to lambda. But when I try to upload the artifacts as zip or from S3 and save it , then it removes the zip file or S3 link. How do we know that artifacts are deployed to lambda? I have used CloudFormation to create the lambda.
Hey Prasad !! First of all superb explanation of AWS CI-CD. Can you please provide the git link for all the code that you used for my POC purpose, specially these yaml files. Plz share asap have a deadline to meet ;-)
You can add multiple Lambda functions in your SAM template. Please have a look at my video on Building Lambda based REST API where I used multiple Lambdas within the same template. th-cam.com/video/50rBFasH3OE/w-d-xo.html
API Gateway is not mandatory. If you have other ways of triggering your Lambda directly, You can skip API Gateway which is for sending HTTP requests to Lambda from your application.
@@PrasadDomala - Thank you for the video, it is very helpful. Can you tell a couple other possible ways of triggering lambda without API gateway. I am asking this coz, I have an implementation with integrating lambda only and not quite sure on how to configure triggering.
i cant understand this vedio becoz it is high level, still i understood samTemplate.yaml is for creating lambda using cloudFormation, buildspec.yml file for codepipeline where we can get outputTemplate.yaml file, can anyone explain me little more, i am unaware of cloud front so i am not understanding, cloud front is like terraform right for infrastructure as a code, if you help thanks in advance, what is the flow going here
I got an error in the Executechangeset block of the pipeline. Later I check my cloud formation template and there I found this error "The role defined for the function cannot be assumed by Lambda. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException;" . I am working on it. Do you have any idea @Prasad_Domala?
If your Lambda is unable to assume the execution role, there are high chances that the trust policy is missing on the execution role. You should provide 'sts:AssumeRole' permission for 'lambda.amazonaws.com' service under trust relationships.
Hi Prasad/All, I am getting "Missing authentication token" error everytime m hitting the apigateway to invoke lambda function. Authorization is set to NONE everywhere. Could you please help!!
Hello sir thank you so much. One thing I just wanted ask you how we can add model (schema) to our api gateway through serveless not from the console. Thanks
How do you manage different environments with SAM-CLI? because there's serverless framework from serverless.com which already allows you to manage stages of your app like dev, staging, pre-production and production. I want to be a able to do that with sam-cli as well.
SAM uses CloudFormation in the backend. The approach I would follow for multiple environments is hooking up your repo with some CI/CD tool like terraform and manage the environments using Terraform variables. You can even use Git branching strategy to isolate your resources specific environments. For example, commit to your develop branch should create your develop resources and prod branch should create your production resources and so on. I might demo this process in my future video soon.
@@PrasadDomala Fantastic! A demo is just what I needed, there's an issue in sam-cli repo about this. I would imagine that different environments are just CF Stacks with prefix corresponding to their environment, like `-dev` or `-staging` or `-preproduction`, but it seemed quite complicated. If you may, can you also discuss the concept behind that? how would that look like if you do it using a bash script. github.com/awslabs/aws-sam-cli/issues/814
Hi Prasad, Your video is awesome, so I started to follow it. But in between, where you are at the CodeDeploy step, you are able to mention only the template file name as "outputSamTemplate" but in my case, its asking me to enter like this:- :: Please help.
@Ishara Karunathilaka Yes, actually doing some hit and trial, I was able to achieve the exact same name you have mentioned in reply. Thank you so much...👍🏼
Great video, but I have the same issue. Here's the entry in my buildspec.yml. - aws cloudformation package --template-file samTemplate.yaml --s3-bucket my-lambda-pipeline --output-template-file outputSamTemplate.yaml Any suggestions?
Nice tutorial Prasad, At build stage I am getting below error. after googleing I got to know buildspec.yml having problem but I kept all same except the S3 bucket name. Which I have created and give access public, Error: Phase context status code: YAML_FILE_ERROR Message: YAML file does not exist
We followed the same procedure as above video,but we are facing issue at endpoint access url at api gateway.Its showing error as : {"message":"Missing Authentication Token"} and {"message": "Internal server error"}.Can you suggest any recommended step to be involved to override this error.
Make sure there are no authorisers on API Gateway and the backend Lambda is working fine. Also please test the API Gateway resource from AWS Console and check if you are getting expected response from backend Lambda.
@@PrasadDomala As @kavya panchumarthy i got the same error "{"message": "Internal server error"}" and as you said when I checked my AWS console I am getting the same error. And when I created an API manually from existed Lambda code it is working. What could be your suggestion?
@@PrasadDomala And when I disabled the option of "Use Lambda Proxy integration" at Integration Request stage of the API and when I hit the API, I am getting this error {"message": "Missing Authentication Token"}. Any idea?
Hi prasad, When I am using your lambda code it is working Working working: import boto3 import json ec2 = boto3.client('ec2') def lambda_handler(event, context): response = ec2.describe_availability_zones() return {"statusCode": 200, "body": json.dumps(response)} But when I am using this code or any code with lambda layers i am getting this error: {"message": "Internal server error"} Code that don't work: import json import urllib3 def lambda_handler(event, context): http=urllib3.PoolManager(); d=http.request( 'GET', 'pricing.us-east-1.amazonaws.com/offers/v1.0/aws/index.json') data_json=json.loads(d.data.decode('utf-8')) print(data_json) service_list= data_json['offers'] response = {"message":"Hello welcome to Lambda demo","List of AWS Services": json.dumps(list(service_list.keys())) } return {"statusCode": 200, "body": response} I followed every step you said. But getting the above error when I am changing the code at API gateway and code is working fine at lambda level. Any idea?
![[#2024MAMA] ROSÉ (로제), Bruno Mars - APT. | Mnet 241122 방송](http://i.ytimg.com/vi/dgGqD28J6aQ/mqdefault.jpg)
![[MV] A leap of faith...ความเชื่อใจครั้งสุดท้าย (From "Petrichor the series")](http://i.ytimg.com/vi/U1piZH2CNXA/mqdefault.jpg)
![BOWKYLION - วิงวอน (ex-change) [Official MV]](http://i.ytimg.com/vi/cLdnZWDaO-w/mqdefault.jpg)
It would have been great if you would have also discussed about versioning the lambda functions for different stages. So that we can publish different code to different stages.
@Prasad. 5:23 you mention that you would put a link in the description for the role policies. Could you please do that? That would be fantastic.
did you get it ?
Great video. Thanks for the detailed explanation
Excellent, crisp, to the point video. Thank you.
Awesome video ...
Thanks for sharing your knowledge and spending your valuable time ..
Excellent... yes, please need more videos
where is the link to the code ?
i don't find the policy in the description
great article, it works awesome. one question how to put variable values in the lambda code (sam template). like if the same .yml needs deployment to different environments then value of things like environment variables of lambda would have different values for different environment deployments. how do we have this kind of thing achieved ? we can have different branches deployed to different environments but few values would need changing while deployment.
Hello Prasad - I have security scenario for this architecture as how to secure the requests coming from the internet traffic preventing the backend getting exploited.
Three scenarios here:-
1) if http request coming to api gateway and how can you secure at the gateway level.
2) if http request from the salesforce authenticating which gives saml, how do we secure and integrate with API gateway.
3) if http request coming from OAM auth and how do we secure and integrate it with API gateway.
Very good video! Can you also save the manual configuration you did for setting up IAM and CodeBuild in the AWS Console in a SAM or CloudFormation template?
@Prasad, Really appreciate all your videos and they are very helpful! One question regarding the python dependencies. If I have pip installed additional dependencies such as `googlemaps` in a local virtual environment, how should I include this additional dependency in the deployment process with CodePipeline and CloudFormation?
i think lambda will take care of everything,tell me if i am wrong
I know I’m late to respond but you would package the dependencies as Lamda Layer and then deploy the Lambda Layer as an additional resource in your template
Thanks Prasad for this video! I have a question though.
If my python code has additional dependencies and it requires a virtual env for python to be deployed, how should I make use of code build here or if you can suggest a better way?
Or will Sam template take care of it somehow?
I was wondering if I have to use Sam build first?
Wonderful video, still working!!
Thank you for the tutorial. I have a doubt actually... In the online policy json file there's this line
"Resource": [ "arn:aws:s3:::codepipeline*" ]
Here, what is meant by codepipeline? Is it the S3 bucket name to store the artifacts after build step.... Do I need to change it with the name of my S3 bucket?
How to create a similar pipeline with Jenkins and Nexus instead of AWS code services?
Thanks
Can you post the link to the code? the resolution makes it hard to read in parts.
Found this code on github, not sure if it is the right link, but hope it helps. github.com/pdomala/LambdaCDDemo/blob/master/CFnServiceRole.json
@@gohjy1995 Thankyou
Great and Awesome explanation.. Thanks Buddy! Is there a link to the code in the github?
Hi Prasad I have 50 number of lambda functions in one git repo with python lambda functions and node js lambda functions , how can we achive codepipeline in aws in this scenario , please let me know .
Excellent presentation there & I love that crisp font. How did you get that crispness?
Hi Prasad: I would like to understand what are the responsibilities of an individual being a DevOps engineer in terms of AWS Lambda service..?
Great video. It was very useful for me. Subscribed
nice explanation, can you please share the GitHub repository for these JSON files.
Expecting appspec file at location /home/ec2-user/appspec.yml but it is not found there. Please either run the CLI from within a directory containing the appspec.yml file or specify a bundle location containing an appspec.yml file in its root directory
How can we run smoke tests after lambda deployment ?
Thanks. Explained very nicely.
I have my dotnet core application artifacts that should be packaged and deployed to lambda. But when I try to upload the artifacts as zip or from S3 and save it , then it removes the zip file or S3 link.
How do we know that artifacts are deployed to lambda?
I have used CloudFormation to create the lambda.
Excellent job!
Thank you for showing us how to do this. Could you please share the code?
How can we deploy lambda to different AWS account. We have different account for dev, test and production.
Great talk.. great job.
what was the purpose of the pipeline vs just running the build as aws cli command?
Where is the link in the description for git repo?
Does this include in freetier? they mentioned 1$ per month for 1 pipeline.
Thanks a lot Prasad. this is really good tutorial
Can you please provide the link for iam policies
it will be very helpfull
really helpful man, thanks for the video!!
Great Tutorial Prasad !
Hey Prasad !! First of all superb explanation of AWS CI-CD. Can you please provide the git link for all the code that you used for my POC purpose, specially these yaml files. Plz share asap have a deadline to meet ;-)
Did you get the link of it ?
Hello sir thank you so much. One thing I just wanted ask you how we can create multiple lambdas from single Github repo. Thanks
You can add multiple Lambda functions in your SAM template. Please have a look at my video on Building Lambda based REST API where I used multiple Lambdas within the same template. th-cam.com/video/50rBFasH3OE/w-d-xo.html
@@PrasadDomala thank you so much sir.
Repo here: github.com/pdomala/LambdaCDDemo
Thumbs up so people will see
Really good one
awesome video but we need to integrate with lambda only not api gateway you have any suggestion for this?
API Gateway is not mandatory. If you have other ways of triggering your Lambda directly, You can skip API Gateway which is for sending HTTP requests to Lambda from your application.
@@PrasadDomala - Thank you for the video, it is very helpful. Can you tell a couple other possible ways of triggering lambda without API gateway. I am asking this coz, I have an implementation with integrating lambda only and not quite sure on how to configure triggering.
i cant understand this vedio becoz it is high level, still i understood samTemplate.yaml is for creating lambda using cloudFormation, buildspec.yml file for codepipeline where we can get outputTemplate.yaml file, can anyone explain me little more, i am unaware of cloud front so i am not understanding, cloud front is like terraform right for infrastructure as a code, if you help thanks in advance, what is the flow going here
I got an error in the Executechangeset block of the pipeline. Later I check my cloud formation template and there I found this error "The role defined for the function cannot be assumed by Lambda. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException;" . I am working on it. Do you have any idea @Prasad_Domala?
If your Lambda is unable to assume the execution role, there are high chances that the trust policy is missing on the execution role. You should provide 'sts:AssumeRole' permission for 'lambda.amazonaws.com' service under trust relationships.
@@PrasadDomala Thank You for the response.
Hi Prasad/All,
I am getting "Missing authentication token" error everytime m hitting the apigateway to invoke lambda function.
Authorization is set to NONE everywhere.
Could you please help!!
Check custom domain mapping
Hello sir thank you so much. One thing I just wanted ask you how we can add model (schema) to our api gateway through serveless not from the console. Thanks
You mean. Build the API using code and not from console? Yes i will make a video on that soon.
@@PrasadDomala thanks
How do you manage different environments with SAM-CLI? because there's serverless framework from serverless.com which already allows you to manage stages of your app like dev, staging, pre-production and production. I want to be a able to do that with sam-cli as well.
SAM uses CloudFormation in the backend. The approach I would follow for multiple environments is hooking up your repo with some CI/CD tool like terraform and manage the environments using Terraform variables. You can even use Git branching strategy to isolate your resources specific environments. For example, commit to your develop branch should create your develop resources and prod branch should create your production resources and so on. I might demo this process in my future video soon.
@@PrasadDomala Fantastic! A demo is just what I needed, there's an issue in sam-cli repo about this. I would imagine that different environments are just CF Stacks with prefix corresponding to their environment, like `-dev` or `-staging` or `-preproduction`, but it seemed quite complicated. If you may, can you also discuss the concept behind that? how would that look like if you do it using a bash script.
github.com/awslabs/aws-sam-cli/issues/814
There is not link for the repository...
Cannot see any code in description
thank you!
Hi Prasad, Your video is awesome, so I started to follow it. But in between, where you are at the CodeDeploy step, you are able to mention only the template file name as "outputSamTemplate" but in my case, its asking me to enter like this:- ::
Please help.
@Ishara Karunathilaka Yes, actually doing some hit and trial, I was able to achieve the exact same name you have mentioned in reply. Thank you so much...👍🏼
@sanjay kumar Singh first tell me what is the output template name u gave in buildSpec.yml for output template
Great video, but I have the same issue. Here's the entry in my buildspec.yml.
- aws cloudformation package --template-file samTemplate.yaml --s3-bucket my-lambda-pipeline --output-template-file outputSamTemplate.yaml
Any suggestions?
@@ChrisMcDaniel1 I can help you resolve your issue. Lets discuss over this in DM.
@@ChrisMcDaniel1 Please can you share your Skype ID?
Nice tutorial Prasad,
At build stage I am getting below error. after googleing I got to know buildspec.yml having problem but I kept all same except the S3 bucket name. Which I have created and give access public,
Error: Phase context status code: YAML_FILE_ERROR Message: YAML file does not exist
error in DOWNLOAD_SOURCE phase
if you get time please help.
We followed the same procedure as above video,but we are facing issue at endpoint access url at api gateway.Its showing error as : {"message":"Missing Authentication Token"} and {"message": "Internal server error"}.Can you suggest any recommended step
to be involved to override this error.
Make sure there are no authorisers on API Gateway and the backend Lambda is working fine. Also please test the API Gateway resource from AWS Console and check if you are getting expected response from backend Lambda.
@@PrasadDomala As @kavya panchumarthy i got the same error "{"message": "Internal server error"}" and as you said when I checked my AWS console I am getting the same error. And when I created an API manually from existed Lambda code it is working. What could be your suggestion?
@@PrasadDomala And when I disabled the option of "Use Lambda Proxy integration" at Integration Request stage of the API and when I hit the API, I am getting this error {"message": "Missing Authentication Token"}. Any idea?
@@PrasadDomala This is the issue : github.com/awslabs/aws-sam-cli/issues/437
Hi where is the repository link??
could you please share with us the code files
Hi prasad,
When I am using your lambda code it is working
Working working:
import boto3
import json
ec2 = boto3.client('ec2')
def lambda_handler(event, context):
response = ec2.describe_availability_zones()
return {"statusCode": 200, "body": json.dumps(response)}
But when I am using this code or any code with lambda layers i am getting this error: {"message": "Internal server error"}
Code that don't work:
import json
import urllib3
def lambda_handler(event, context):
http=urllib3.PoolManager();
d=http.request( 'GET', 'pricing.us-east-1.amazonaws.com/offers/v1.0/aws/index.json')
data_json=json.loads(d.data.decode('utf-8'))
print(data_json)
service_list= data_json['offers']
response = {"message":"Hello welcome to Lambda demo","List of AWS Services": json.dumps(list(service_list.keys())) }
return {"statusCode": 200, "body": response}
I followed every step you said. But getting the above error when I am changing the code at API gateway and code is working fine at lambda level. Any idea?
Atleast mention the code or policy details in the section ..
Can u plz share your github repo URL?
link please...
Can you share the source code and policy link
Found it -- github.com/pdomala/LambdaCDDemo
github.com/pdomala/LambdaCDDemo
Repo URL???
github.com/pdomala/LambdaCDDemo/blob/master/CFnServiceRole.json
can codecommit be used instead of github?
Yeah! I'm using codecommit.
Yes
Found the repo by searching his name + github: github.com/pdomala/LambdaCDDemo
here is the code: github.com/pdomala/LambdaCDDemo