@cloudmancer noticed you don't have any networking videos. Is that something you can potentially discuss? Creating a VPC, security groups, subnets etc...?
Hey, thank you! I am not the best a python but I have a desire to get better. I have a python tutorial series on the CDK on my list of videos to make, I will keep you updated!
great. I've got two questions: 1. is there a maximum size of the payload as in/out of the functions? I mean you pass the whole converted csv as out -> in. Maybe is "too much"? 2. How can we achieve an email if some step fails? Another lambda or a direct call to SNS? thanks
Hey there, 1. Yes there is a max payload size that a Lambda can return and this is 6 MB for synchronous invocations such as invoking Lambda directly via API Gateway, another Lambda function or a step functions invocation. If you had a particularly large data set, one solution might be to write it into a new file in S3 and pass the new object key down and have the subsequent lambda function read from the new file, rather than passing the raw data down in the state object. 2. Yes this is totally doable. All we would need to do is create a new FailedExecution SNS topic and a new lambda invoke task that could handle the failed execution. Then, rather than passing in the FailState into our addCatch, which fails the execution directly, we would instead pass in this new task. I would be interested to hear your experience if you implement something like this! I hope this helps.
@@cloudmancer thanks for your answer, those are just out of my curiosity. At the moment i'm not yet into step functions: i'd like to experiment and consider a refactoring. My current project involves three lambdas with different tech stack, communicating via sqs.
What videos would you like to see me make next?
Very interesting project, I’m going to try it! You could add something with AI or containers!
You know building something with AI would be really fun, I will come up with a good idea
@cloudmancer noticed you don't have any networking videos. Is that something you can potentially discuss? Creating a VPC, security groups, subnets etc...?
I have it on my list to get into some networking videos with the CDK. I will come up with something for you.
Great video. Any chance you can do it in python?
Hey, thank you!
I am not the best a python but I have a desire to get better. I have a python tutorial series on the CDK on my list of videos to make, I will keep you updated!
great. I've got two questions:
1. is there a maximum size of the payload as in/out of the functions? I mean you pass the whole converted csv as out -> in. Maybe is "too much"?
2. How can we achieve an email if some step fails? Another lambda or a direct call to SNS?
thanks
Hey there,
1. Yes there is a max payload size that a Lambda can return and this is 6 MB for synchronous invocations such as invoking Lambda directly via API Gateway, another Lambda function or a step functions invocation. If you had a particularly large data set, one solution might be to write it into a new file in S3 and pass the new object key down and have the subsequent lambda function read from the new file, rather than passing the raw data down in the state object.
2. Yes this is totally doable. All we would need to do is create a new FailedExecution SNS topic and a new lambda invoke task that could handle the failed execution. Then, rather than passing in the FailState into our addCatch, which fails the execution directly, we would instead pass in this new task.
I would be interested to hear your experience if you implement something like this!
I hope this helps.
@@cloudmancer thanks for your answer, those are just out of my curiosity. At the moment i'm not yet into step functions: i'd like to experiment and consider a refactoring. My current project involves three lambdas with different tech stack, communicating via sqs.
It sounds like your current project would be good for a step functions refactor, let me know if you do and what you come up with!