Adding and Getting Files from Amazon S3 with Serverless.
ฝัง
- เผยแพร่เมื่อ 23 ก.ค. 2024
- In this video we'll be creating two new API endpoints. One to add JSON data to our Amazon S3 bucket, and one to get files back.
By the end of this video you will have:
- Added two more endpoints to your serverless application
- Created a wrapper to easily read and write files from Amazon S3
- Uploaded JSON data to your S3 bucket via an endpoint
- Downloaded JSON data from your S3 bucket
Complete code for this file is available here:
github.com/SamWSoftware/Serve... - วิทยาศาสตร์และเทคโนโลยี
Thank you very much, Sam. Enjoyed this tutorial.
Glad you enjoyed it
That was awesome dude! Thanks a lot! 1000 thumbs up!
Glad you liked it and thanks for the support!
crisp and clear explanation!! cheers !!
Glad you think so!
Thanks for the great video. It’s really helpful and the closest I found to production work. However note that Lambda doesn’t have a big payload so large files will require different approaches (e.g. signed Urls)
Yes, if you're transferring large files then yes, you'd need to use signed urls or another approach.
Thanks chamo, you are the best!!
Thank you!
I really enjoy all your videos because it direct to the point, one thing i missing here is different types of encryption of uploaded file, would be great if you make a video to explain how this could be implemented in the code
You can configure your S3 bucket so all files are encrypted automatically at upload. Makes this super simple for storing encrypted files. No real coding required.
You could encrypt it before you upload it (in the web app) and then the user can decrypt it again when they download it. that would all be frontend work ( encrypting and decrypting in react )
Another option is to encrypt it manually in the Lambda before storing it in S3. I don't really see the point in this vs option 1.
If you did go for option 2 or 3 then you'd just use a library to do the encryption. This isn't something I've ever done so can't really recommend any packages.
Excellent video. I am also gonna watch that video on image uploading to s3.
I hope that you enjoy it
Great tutorial! Is it possible to create these uploading and downloading functions but deployed as its own s3 static website? To expand, a s3 website interface that is able to get files and upload files listed on the s3 console
Yes definitely. I'm planning to do some videos where I build simple from end websites that interact with the APIs built in this course.
Thanks a lot for sharing this info. Can you also let us know how we can set the expiration lifecycle for the S3 through serverless.
I really enjoy this tutorials, Sam. Thanks for create them.
I was wondering how to associate a file to one register into another table (for example player-points table). Would it be as easy as create an additional field called "Certificate" and save it here? BUT
how could I trigger this process update routine of player to achieve it? or this is a frontend process and it is not neccesary to do it like this?
I really appreciate your commnets about it.
Do you mean allowing the user to upload a file which you save in S3 and then add a field in dynamo which is the url of the uploaded object?
If you do then you can definitely do that. You could modify the update player endpoint to also take an optional 'file' in the body of the request (similar to this video). If that parameter is present then you write the file to S3 as you do in this video. When you get back the details and calculate the URL you would add that as a parameter in the dynamo write data.
This way when you next get that player score, there will be a URL in the data, pointing to that S3 file.
@@CompleteCoding Thanks for the comments.
Could I upload a .zip file using this same Bucket?
Great Video Sam!
Just a quick question if we want to upload a CSV file in the bucket and perform operations on it on the client side will this code work?
Yes, writing and reading csv files should work just the same.
You'll have to convert it from cvs to JSON to be able to process it in the front end
Hello thank you very much for the tutorial, can you help me with a recommendation to download files from an S3 to an EC2, I am trying with lambda and an api rest
You have two main options.
You could either call your S3 get API from your server. If you've already followed this video then you should have an API which you can hit from your server. The other option is to add the logic for getting from S3 to your server directly. You will also need to give your EC2 permission to read from S3 but it saves the extra work of building and deploying your API.
Hope that helps
@@CompleteCoding Thank you very much it is quite helpful, the file is deposited in the S3 on user request as I do it automatically an event of the S3 can invoke a rest service or implement the api
when I run "sls deploy" again, car.json file in S3 is deleted. Any idea to prevent auto delete ?
I'm getting error when i'm trying to see my files in lambda. Does any one having clue ?
You cannot open any files within this directory through the UI. This file can be opened via the command line. This issue only affects files with the file path /home/ec2-user/environment/home/ec2-user/environment. Check documentation
Hi Sam! The way you are explaining is excellent. I just want to ask you a question, I have a scenario where I need to upload multiple images to the s3 bucket using api gateway which I will consume in android app. Could you please make a video on it or refer me any blog or anything else thorough which I could solve my problem. please reply 🙏🙏🙏🙏🙏
I have a video on image upload already
The API part th-cam.com/video/GBn5zi_Hhqk/w-d-xo.html
The react app: th-cam.com/video/6GpaNOYvZCE/w-d-xo.html
Once the image is in S3, if you've made it public then you can just show that in the app. If it is private then you would have to use a signed url
Is that possible from single endpoint Adding and Getting Files from Amazon S3??
Yes, if you wanted to you could add both functionality into a single endpoint.
One example I could think where you might is if you were creating a API to "find similar something". If it was for movies, you would post up the data for the movie, do some logic to find similar movies and then get those movies to return to the user.
@@CompleteCoding Thank you:)
hi, why we need webpack installed?
Webpack takes all of the code required for the function and puts it into a single file. If you don't then ask code in the repo ends up in every lambda. With lots of endpoints the lambda can become too big to deploy
@@CompleteCoding Thank you for clearing that!
I am getting error when run this command: sls deploy
Cannot read file node_modules\yamljs\cli\yaml2json.js due to: EMFILE: too many open files, open 'C:\Users\Masud\projects\learning\serverless-test
ode_modules\yamljs\cli\yaml2json.js'
I've never seen that error before. There seem to be some answers if you google the error message
There's a better approach using regex to check the file type on 17:20
/\.json$/.test(fileName)
That's a good point. I'll put that option into the text version of this video
so this only work with JSON ?... and not uploading actual files like word documents. Dang, back to google I goooo ( great vid btw)
You can store files like those in S3 but you would use another technique called "Signed Urls". That way you can use whatever file type you like - even images or videos
th-cam.com/video/MlKpK0WqTSs/w-d-xo.html
(...) from s3, not a dynamo
great tutorial THANKS
Nice spot. I'm sure there are loads of little fumbles hidden in my videos