What is a DynamoDB Stream? (And why you should be using it!)
ฝัง
- เผยแพร่เมื่อ 21 ก.ค. 2024
- DynamoDB Streams are a powerful feature that allow applications to respond to change on your table's records. Using streams can enable some powerful use cases such as real time summary dashboards or creating data replicas. In this video, I start by talking about some common use cases for DynamoDB streams. From there I review some of the key features of DynamoDB Streams. Finally, I give a detailed breakdown of how DynamoDB Streams work under the hood.
Looking to get hands on experience building on AWS with a REAL project? Check out my course - The AWS Learning Accelerator! courses.beabetterdev.com/cour...
📚 MY RECOMMENDED READING LIST FOR SOFTWARE DEVELOPERS📚
Clean Code - amzn.to/37T7xdP
Clean Architecture - amzn.to/3sCEGCe
Head First Design Patterns - amzn.to/37WXAMy
Domain Driver Design - amzn.to/3aWSW2W
Code Complete - amzn.to/3ksQDrB
The Pragmatic Programmer - amzn.to/3uH4kaQ
Algorithms - amzn.to/3syvyP5
Working Effectively with Legacy Code - amzn.to/3kvMza7
Refactoring - amzn.to/3r6FQ8U
🎙 MY RECORDING EQUIPMENT 🎙
Shure SM58 Microphone - amzn.to/3r5Hrf9
Behringer UM2 Audio Interface - amzn.to/2MuEllM
XLR Cable - amzn.to/3uGyZFx
Acoustic Sound Absorbing Foam Panels - amzn.to/3ktIrY6
Desk Microphone Mount - amzn.to/3qXMVIO
Logitech C920s Webcam - amzn.to/303zGu9
Fujilm XS10 Camera - amzn.to/3uGa30E
Fujifilm XF 35mm F2 Lens - amzn.to/3rentPe
Neewer 2 Piece Studio Lights - amzn.to/3uyoa8p
💻 MY DESKTOP EQUIPMENT 💻
Dell 34 inch Ultrawide Monitor - amzn.to/2NJwph6
Autonomous ErgoChair 2 - bit.ly/2YzomEm
Autonomous SmartDesk 2 Standing Desk - bit.ly/2YzomEm
MX Master 3 Productivity Mouse - amzn.to/3aYwKVZ
Das Keyboard Prime 13 MX Brown Mechanical- amzn.to/3uH6VBF
Veikk A15 Drawing Tablet - amzn.to/3uBRWsN
Resources:
Getting started with AWS: • Introduction to AWS | ...
☁Topics covered include:
- DynamoDB Streams
- DynamoDB Event Types (Keys only, New Image, Old Image, New & Old Images)
- DynamoDB Stream Use Case Examples (Real Time Dashboards, Data Replication)
- DyanmoDB Stream Shards
- DynamoDB Stream and Lambda Integration
🌎 Find me here:
Twitter - / beabetterdevv
Instagram - / beabetterdevv
Patreon - Donations help fund additional content - / beabetterdev
#DynamoDBStreams
#DynamoDB
#AWS
![[TH] VCT Pacific Stage 2 - Playoffs Grand Final // GEN vs DRX](http://i.ytimg.com/vi/Su6gA_UPs9s/mqdefault.jpg)
I know I'm going to sound hyperbolic here, but this guy's explanations are quite simply the best I've seen for AWS-related concepts.
Thank you so much John. Your comment means a lot to me. Cheers!
@@BeABetterDev Thank you , you are giifted , not everyone can explain so well !
I agree. Great content
The way you also take up scenario based explanations is phenomenal. Its a great add on apart from the core concepts. Thank a ton :)
You're very welcome!
The same thoughts here, very practical.
I watched all the videos and this is best playlist on DynamoDb. I have an interview tommorow and they mentioned knowledge about DynamoDb in JD. I hope I crack it and btw keep up the good job.
I have been looking for a video that will explain dynamodb streams to me in preparation for an exam. This video is prolly the best! This is short but comprehensive
Thanks Sai! Glad you enjoyed.
Love the short videos that are to the point with facts and good tips or how the AWS services works. Thank you!
Glad it was helpful!
Nice to find videos that give use cases and the why's; instead of the how's. thank you !
Glad it was helpful!
Great Video! your explanation and examples really made it easy to understand DynamoDB Streams
Greate video! Explanations are amazing!
A must watch video
This example is amazing, would like to hear more about AWS Elastic Search!
Coming soon! Thanks !
This is great, starting a new role and this easily bootstrapped me into the ecosystem
Glad I could help!
You're really good at this. Keep going!
Thank you so much for the kind words!
Great stuff. Just what i need. Thank you, subbed. 👍
wow this was so interesting, I understood almost everything
so wonderfully explained, thank you very much !!
You're very welcome!
I love how you simplify the content with your used case , you make it seem so cool !
Just a tiny bit of suggestion , it'd be great if you could speak a bit louder in the future videos. You are awesome , keep going !!
Thank you so much Zainaba for the kind words. This was one of my first videos so my audio quality should be much better in my more recent ones. Thanks again and stay safe!
Daniel
Concise, yet comprehensive. Its a shame I can only like this video only once?
Thanks so much Eyad for the kind comment. Glad you enjoyed )
brilliant! thanks for this video
i had an issue with dynamodb streams and had to understand the foundation of dynamodb streams feature and this video helped a lot.... i believe the community needs more videos/tutorials and really real code examples... please, give us more and we can always help as well (start using aws resources and connecting them is something new to me... i was just used to normal ec2, s3, blablabla)!
cheers!
Thanks Rejaine! I'm constantly putting out new videos on all sorts of AWS topics - especially those concerned with hooking up resources across services.
Thanks again!
@@BeABetterDev thank you!!!
I also found a dynamodb developer guide which has everything.. it just impressive!!! and it has 1 chapter about dynamodb streams api and kinesis adapter.. i was really confusing about this kinesis client library if we have the dynamodb streams api, but now I believe i got it and it has awesome samples to practise! =)))
thank you so much again!!! all your content are amazing!!!
Thanks again Rejaine! If you have any suggestions for videos I'd love to hear them!
AWS Simplified a walkthrough of a Java application that uses the Amazon Kinesis Client Library and the Amazon DynamoDB Streams Kinesis Adapter. An example of data replication, in which write activity from one table and applied to a second table, with both tables' contents staying in sync to another consumer... Hehe
That may be a bit complicated for this audience but I'll add it to my list. Sounds like you have quite an interesting project on your hands!
Clear and to the point. Perfect!!
Thank you Ashish!
Respect for this quality of content!
Thanks Auroshis!
Terrific series. Please do one on Route 53.
Thanks ben!
I am surprised that not many people `liked` this video. I work for AWS, and I think this video is very amazing.
Hi G O. I am currently working with DynamoDB. Need your help with “Invalid attribute value type” error.
Thank you G O!
thank you so much nice video, would like to hear more
Thank you!
Very useful and informative. 👍
Glad it was helpful!
@@BeABetterDev I am currently studying for my first AWS certification: SAA-C02. I wanted to point out that the great value of this particular video is how it illustrates the integration of several AWS services. Until I watched this I really didn't grasp the full potential of lambda event sourcing. This is a shortcoming of the course that I'm taking which really tends to dwell on individual services rather than their integration. Is my understanding that SQS and KDS more or less follow the template that you illustrated with Dynamodb streams.
The video is really helpful but I did not find the code in the description can you please provide me one
Can you tell me what's price when using Dynamo db stream with lambda
You are just awesome. Thanks
Thanks Rajesh!
Do you think DynamoDB Stream would be suitable for streaming stock prices to web users who have different watch list?
I have one question:
How is this better than having a SNS when the player is playing and then writing in the DynamoDB table and a lambda for the ElasticSearch? I am asking because I see how these DynamoDB events could be useful but what makes them better than directly writing when an event is triggered by the player at the beginning.
Thanks!
thanks for the amazing video! one quick question though - when you say events ordering will be preserved does it mean within shard? As under the hood you explained multiple shards will be created which is acting like a container to store the events, so there is a possibility of situation like E1 -> S1 and E2-> S2?
Hi there,
Yes events are preserved, but are bound to the shards. With DynamoDB Streams, the 'shards' map to the DynamoDB Partition Key on your table.
amazing job
Thanks Abishek!
Having some background on elasticsearch, and now watched this video and video on GSI, can I have more information regarding when to use dynamodb when to use es?
Hi Eric, I'm going to be covering your question in an upcoming video. Check back soon!
I have a requirement to stream dynamoDb data to lambda which is in different account. As per the documentation, I can only stream dynamoDb data into the lambda of same account.
How can I fulfil my requirement? Can anyone help me out!
Excellent material! Clear and concise!
Thank you for the kind words!
Amazing explanation. Just 1 question
My problem statement was:
The bathSize that Aws gives us which can be till 10k , does that mean all shards including 10k or is it that individual shards will have 10k
I added batch size as 100 and I see that 100 are not even happening, before that only lambda is hit
Note: payload is NOT > 6MB and time is also 300 sec(whihc is not completed)
Due to this multiple small files are created in my s3(my lambda does this process), and my etl running on this costs heavily on GET requests.
Is there any way around?
please do the implementation after architecture videos, than only it solves the whole purpose. thanks
At the 11:15 mark - you mention that "it won't cost anything extra on the DynamoDB side". You may want to clarify what you mean. You may have been referring to something other than pricing. There is a cost to DynamoDB streams. Every month, the first 2,500,000 DynamoDB Streams read request units are free and then $0.02 per 100,000 DynamoDB Streams read request units thereafter. (N. Virginia)
Good point david. I was referring to cost on the dynamodb table itself but you bring up a good point - streams are not free.
Hi can you post some videos on java + lambda
could you provide ppt it would be really helpful!
Thanks
Dynamodb single table design can cater for quite a large amount of use cases.
Perfect 👌 just the voice quality is less good than previous videos..
Not bad if you have headphones on. The content is still as good as ever
you are a Teacher..
TL;DR - DynamoDB is simply an equivalent to RelationalDatabase Triggers in a more convoluted and expensive way :X
Demn.
I swear I fucking love you!
I love you too twisted bam!
audio is low
Hi welcome to my videos where my audio is really really low.
Hi Andrew. Sorry about that - this was one of my earlier videos where I wasn't aware of audio normalization. If you watch some of my more recent videos you'll notice the quality is much better and sound levels consistent. Thanks.
Could've used an easier analogy
The video is really great but the audio is not up to the mark.
Can you please step it up a notch
Hi Pramesh. This was one of my first videos, apologies for the poor audio quality. The quality in my more recent videos is much better.
@@BeABetterDev Yes sir,
I did see your other videos. I have been binge watching. Loving serverless and your videos. Thank you very much.
Please, normalize the audio level...it sounds like you're in a vehicle passing by over and over 🙁 Other than that, good video. 👍
Hi Peyator,
Thanks for the feedback. I definitely agree with you - my audio was horrible in this video. And I do actually live beside a street so good observation on the vehicles haha.
I've drastically improved my audio in my more recent videos - this one was one of the first videos I put together for my channel. Hopefully you give them a chance :)
Thanks for watching and for your feedback.
Daniel
Video quality of your videos are poor. Sound is low. Increase the sound. If we feel high volume, we can reduce. But, for low volume already, hard to increase.
Hi Jewel, this was one of my first videos. My more recent ones are much better quality.
@@BeABetterDev ok. Got it
damn the mic is so low volume =_=
Hi there I'm sorry about that - the quality is much better in my more recent videos.
very very low audio..
Sorry about this NC. This issue has been fixed in my newer videos.
your voice is very low its better if u can speak some loudly
helpful video #shobhitwalia1
I watched all the videos and this is best playlist on DynamoDb. I have an interview tommorow and they mentioned knowledge about DynamoDb in JD. I hope I crack it and btw keep up the good job.
Thanks Ujjval! I hope you did well on your interview and that my videos helped you ace it!
Cheers,
Daniel
Thanks
Welcome!