AWS SQS | SQS Architecture | Visibility Timeout | Dead Letter Queue | Long Polling
ฝัง
- เผยแพร่เมื่อ 6 ก.ค. 2024
- Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
Here as well, just like Message queue, we have our Producers who actually have messages that they want to be processed and we have our consumers who are waiting to process these messages.
And to propagate these messages, in AWS we make us of AWS SQS aka Simple Queue Service.
Producers push these messages or tasks to the AWS SQS queues for them to be executed with the consumers at hand.
⭐ Kite is a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I've been using Kite for 6 months and I love it!
www.kite.com/get-kite/?...
Timelines for convenience:
0:00 Intro
0:20 AWS Simple Message Queue
1:10 AWS SQS Architecture
2:46 Amazon SQS Visibility Timeout
7:12 AWS SQS Message Lifecycle
8:54 AWS Simple Message Queue (Standard vs FIFO queue)
11:50 Features of AWS SQS
14:48 AWS SQS Dead Letter Queue
15:52 AWS SQS Long Polling
20:24 AWS SQS Recap (Lifecycle of AWS SQS)
22:14 Outro
Please follow we in the links below to stay updated: 🙌
🍀Click on the link below to subscribe: tinyurl.com/qqebnwz
🍀Instagram: / tougherapollo
🍀Twitter: / tougherapollo1
🍀Discord: / discord
#AWSSQS #AWSSolutionsArchitectAssociate2020 #SQSLongPolling - วิทยาศาสตร์และเทคโนโลยี
Timelines for convenience:
0:00 Intro
0:20 AWS Simple Message Queue
1:10 AWS SQS Architecture
2:46 Amazon SQS Visibility Timeout
7:12 AWS SQS Message Lifecycle
8:54 AWS Simple Message Queue (Standard vs FIFO queue)
11:50 Features of AWS SQS
14:48 AWS SQS Dead Letter Queue
15:52 AWS SQS Long Polling
20:24 AWS SQS Recap (Lifecycle of AWS SQS)
22:14 Outro
Instablaster
you explained long vs short polling like 20 times in 5 mins in same exact way 😂though, good vid. thank you 🙂
Nice and detailed. Your diagramming is some of the best I have seen out there.
Very well described and presented. Thanks
Best on SQS so far.. Keep up doing good work..
Hey Bud. You did a great job. Thanks for this! You may get a fan if this is your standard quality :)
Please check out the playlist and let me know if I did. 😊
A great explanation indeed! thanks
Great content. Thank you so much!
Great video bro..we can see the effort put in to it to cover all the areas..keepup the good work
Thanks bro
Great Video!
very nice, thank you
Thank you Bro for your excellent videos. This helped me a lot for clearing my Associate certification exam. Keep up your good work.
Oh, thats awesome. @prakash. Please tag Pythoholic YT on linked In -- it helps support the channel. I will add you to the hall of fame. Many congratulations.
@@Pythoholic Sure Bro
Thanks for the detailed explanation. However, I need one clarification in 13:53 you mentioned about message locking. Is message locking same as visibility timeout?
You are awesome
Thank you for detailed visual explanation. Is it possible to list down all default values of different fields/parameters on last slide like visibility timeout for a message, FIFO queue throughput
Sure
How SQS decides which consumer will pick that message? if there are may consumers requesting the consumtpion which consumer will get first chance?
Awesome video. However, I have a question, what if i don't send Delete message request after reading that message? Does it mean i will end up reading that message again, but again once i read message it gets locked, isn't it? So, why do we need do delete message?
Very nice ! Question ! When Documentation say "It can Scale from 1 Message to 10000" ,what do they mean ? What is scaling and where? Are scaling the SQL Servers? Or what? Thanks in advance my friend!
When it says scale, it can handle the capacity increase of the message count as a part of the request.
Think of this like working on a website. If I say my website can handle more than 10000 requests also ,what would it mean?
That means the capacity provisioning is auto scaled to handle these many requests.
Long pooling is per consumer ?Let's say we have 10 consumer then long pooling 10 sec timer starts for them seperately ?how dos it works here?
Hi
If we have multiple consumers
How can we process message in all consumers?
Bro 1 important question. Please reply me fast if possible. SQS publisher and consumer uses single or multiple thread ? also if producer & sqs consumer run on the same thread. Need ur help. please reply
please share the note if possible
Hi, thank you for the content. I would appreciate it if you could explain request offloading.
Thank you
thanks amanuel
i will do that
Sir, i have a doubt..
Is there anything like a message will be sent to a particular consumer especially?
Or is the case that it sends the messages to any of the consumers present?
Or in more simpler way, when a consumer sends "receive message request " to the queue, will that request contains any parameters representing a particular message?
Thanks in advance
Mostly we might make use of the group ID for this. But if u could tell me a use case I can let u know
@@PythoholicHi sir, I am having 4 messages in queues, but it is not able to read all the messages at a time , maximum it reads 2 messages , even I have long polling also , still I am not able to read all the messages
At 8.43, you made a comment that when the message is being processed by a consumer, it will not be visible to other consumers? Is this true? Or other consumer can make a request to the message and they will get a false
The thing is with SQS, there is a visibility time out. If let suppose you are a consumer and there is a message on the queue and you are able to receive it. If and when visibility timeout is set, the other consumers wont be able to see the message, it means when they send the GetMessage they wont receive the same message. the message id has never come to you for a false case. If there are no messages then you will have to wait.
This means I should poll SQS every 20 seconds MAX, even if set to _long polling_ ? Correct me if I'm getting this wrong
Yeah please check the demos for a clearer understanding of how things work.
@5:20 Incase of failure, how a failure is occured, Both in consumer and producer failure scenario or example would be much clear.
Please check this -- It should help aws.amazon.com/blogs/compute/using-amazon-sqs-dead-letter-queues-to-control-message-failure/
Please share ppt
at 12.42, If each chunk is 64KB (and billed as 1 request) isnt the max size of payload 64kb? what is the logic behind 4 requests of 64KB each ? And, in batches you have 10 messages or 256KB , How does 256KB matter for batch processing?
Thanks for the query, When we say a chunk its not always the max size. If we say max size it means the amount that can be sent at max. When i say its 256 Kb is max. Aws charges the consumers in terms of 64Kb as one request. There is a difference between max size and chunk. In batch processing you wont send the message the same way as you are sending individually. There is a process to send the message in batch.
@@Pythoholic Thank you for taking time to answer my question. So in SQS max message size is 256Kb, when producer sends 256kb message it will be broken into 4 messages of 64kB each and will each chunk have it have its own message id?,, REgarding batching,, you have mentioned 10 messages or 256kB, so does that mean 10 messages adding upto 256KB or or they not related at all? In other words, batching only to increase throughput i.e, 3000 requests/sec?
Yeah its like when you send a sms, even if you type a content with a huge set of characters, the sms service breaks it into multiple messages, similar to that 64kb chunk is considered as a request.
And for batch as we said, as the limit is 300 msg per sec , mesg here is a operation.
With batching you can have 3000. The 3000 transactions represent 300 API calls, each with a batch of 10 messages.
Please share pdf of same
Will be updating the same
On my website soon
Pythoholic thank you so much ...very good tutorials indeed
Thanks for the support