Webhooks Processing: HTTP API Gateway + SQS +Lambda

🌟 *Master AWS Fundamentals!* 🌟
Ready to dive into the world of cloud computing? Check out this comprehensive course on Coursera: AWS Fundamentals Specialization
imp.i384100.net/WqnRe3
This course covers everything you need to know about Amazon Web Services, from the basics to advanced concepts, making it perfect for both beginners and those looking to enhance their cloud skills. Enroll now and elevate your career with in-demand AWS expertise! 📚🚀
In this video we will look at a real world application of HTT API Gateway and Lambda. Together with Simple queue service they can be used to process webhooks.
Receiving and processing webhooks with AWS HTTP API Gateway, SQS, and Lambda provides the following advantages.
- HTTP API Gateway is highly available and can sustain heavy loads.
- SQS temporarily stores webhooks’ data and enables re-tries
- Lambda functions process webhooks’ data and the number of concurrent functions can be adjusted not to overload the upstream

[@theREALfrancyz]

Thank you for creating a simple project utilizing webhooks and APIs on AWS! This gives me a better understanding even though I don't have real-world experience yet.

[@alexrusin]

Glad it was helpful!

[@perminusgaita]

Something to note, use this design only if you are anticipating bursts in traffic and you want asynchronous processing of your events. Otherwise api gateway to lambda/event bridge will do. This is because sqs queues add latency and also have to be polled to get messages. The polling can add costs as every poll is counted as an sqs execution.

[@alexrusin]

That is a good point. Actually, if you are not expecting bursts in traffic you don't even need to do AWS services. It will be easier to just create a POST route in your application to receive webhooks.

[@watchparty1]

Another good tutorial. Well done.

[@alexrusin]

Thanks again!

[@Fisher0fM3n]

Thank you so much! This helped out a lot

[@alexrusin]

You are welcome. Glad it helped!

[@diegotovaresp]

if i have 2 two headers one is a String and the second one is a byte[] , what i have to put in the Message Attributes input ... i cant read very well in the video thanks for the help

[@alexrusin]

Your headers from the API request will be always coming in as a string (unless they come up with a new HTTP protocol). You can map your headers like so {"hmac":{"DataType":"String","StringValue":"${request.header.x-hmac}"}} in Message Attributes for SQS integration.

[@oleksandrhubachov9201]

I see completelly different UI in API Gateway, switching to old UI does not help. Can't understand on how to connect API Gateway with SQS

[@alexrusin]

Maybe you selected create REST API instead of HTTP API. Be sure you are in the HTTP API. In order to connect API Gateway (HTTP Gateway type) to SQS you can go to integration settings and then choose manage integration settings if the option of creating integration settings is not available. In manage integration settings you can create a new integration and then you can attach it to the gateway.

[@joegenshlea4605]

Do you have a video on how to acknowledge the hmac token and save the body down to S3?

[@alexrusin]

Unfortunately, I don't. It is a good idea, though, to follow up this video with HMAC validation.

[@alexrusin]

I just created such video th-cam.com/video/fe26obf7Png/w-d-xo.html

[@SivaKrishna-zj9jy]

I am not able to view Cloudwatch logs, should I configure some steps to view it?

[@alexrusin]

Hmm, see if there's a check box when you are configuring lambda to create log groups. Maybe it was unchecked.

[@amir-nvim]

do you have any idea how can i send custom header ? but i don't want have default value it comes from webhook

[@alexrusin]

When processing webhooks, you are not the one who sends headers. It is the application you are integrating with that sends you headers. For example Shopify will send X-Shopify-Topic, X-Shopify-Webhook-Id, etc headers. They are standard and will always be there. You can map those headers from HTTP API Gateway to SQS.
If you are building an application that sends webhooks to other applications you can add your headers to POST requests that send webhooks. It is a good idea to prefix custom header with an "X", for example: X-My-Cool-App-HMAC.