This TH-cam channel is gold, I recently saw the playlist where you explaining tests and mocking an express and and mocking a database. Your videos are gold man. One day I’m going to reach this point. You gained a subscriber in me.
Great tutorial! One thing to note; the crypto package adds SIGNIFICANT bundle size to the client, so the computeCheckSum function should be executed on the server EDIT: Digging into this deeper, the file is an input to the checksum, so running the computeChecksum function on the server would defeat the purpose of this tutorial as the whole point was not having your files hit the serverless function... I guess the tradeoff would be to lazy load in the computeChecksum function only when a file is uploaded, ensuring that the initial server payload of your page is not bloated.
Sam? A potential subscriber here who really appreciates this video (and the exactly 1000th thumbed up viewer), but... Any response for this truly legit question from 3 months ago?
Would love to see an expansion of this that goes into multiple files at once and editing a post to swap out images etc. I implemented this recently and feel like there must be a nicer way to do it than I did.
Thanks for the video! Because you’re the s3 guy and have made a lot of videos about it, I think it’d be nice to make an extension video incorporating some of the stuff uploadingthing does and looking at their open source code and showing us how to implement some of the functionality they provide (whatever it is). Afaik, avoiding potential issues where users request a url but don’t upload or something goes wrong and there are a lot of ghost entries in the database or whatnot. I haven’t looked into it myself yet but it’s something I would want to do and a youtube video would be helpful I think to see how uploadthing does it and what problems they solve as an extension to this video. It’s kind of like the video that joshTriesCoding did, but with more code and implementing some of it on our own
Can you PLEASE do a video where you upload a file along with a filled out form - store the file locally and retrieve the file as part of data retrieval for presentation. All done in Nextjs. I'm currently stuck in this for a work project. Thanks in advance
Hi Sam, thanks for the video. I am a product manager and interested in the logic and you gave me good understanding on how things work for social media post/view. One question in my mind was when you said making sure that this post belongs to this user but then you are showing on the home of the app. Why do you need to see if this post belongs to this user? do you mean no one can view the post except the user who created it? Thanks Another question, can I bring up the CloudFlare R2 since they are S3 compatible. do you think we can use it in the same manner you have used S3 now? on the look of the documentations, it seems it can function the same way
I wonder, for security reasons if you should lock down that bucket to not be public and instead generate TTL'd pre-signed read URLs for the items in that bucket. Also you could create this as an API behind and API gateway with cloudfront and cache the images so you don't get hammered with charges for egress from S3.
10:38 ugh, looks like the whole aws identity stuff is completely different now. There's no security credentials tab anymore, and I think they want you to get temporary access keys through "roles" rather than how it's done in this tutorial. Still trying to find a tutorial that shows how to do that. Edit: nevermind. I just realized I've been in the entirely wrong section this whole time. There's the IAM Identity Center, and just the plain old IAM. I think I need to be in the plain old IAM. Gotta love how confusing amazon makes this.
Why does uploading large files say like 160 MB, crashes my application and mainly next auth and I get JWT Session error and need to sign in again, and chery on top is my react query mutation is still running when all this os happening
Hey, if the image was AI generated and fetched from an API, I guess you’d have to upload it from the server directly to s3. Do you know any ressource on how to do that with next js ? All ressources I could find only cover the client-to-s3 situation
can you are a video about your website , where we are using mdx ? Website looks sick !! I want to implement same any pointers ? and does he have a discord ?
Hello friend, thank you for the video, the explanation is very clear! I am implementing the DALL-E 3 API and I want to save the image in a bucket, but I'm not sure whether to do it from the client or directly from the server. Do you think it's too costly to do it from the server using my server function or edge function? The images are around 2 megabytes in size.
hi ! i am currently trying to setup the S3Client config but i am having this error Argument of type '[{ region: string; credentials: { accessKeyId: string; secretAccesskey: string; }; }]' is not assignable to parameter of type '[S3ClientConfig] | []'. can someone tell what should i do or where am i doing wrong
what do you recommend for tracking the progress of the upload to show the user a uploading 53%... etc. i ended up with doing the upload via xhr. what are your thoughts?
Nice video and content! I am also working on a NextJS frontend with a Python backend (NextJS wasn't doing it anymore for me, too much other processes like data parsing had to be done). I don't want to manipulate the DB from NextJS anymore (I mean I do allow for API calls that then in Python will do DB work). What would you recommend here?
There aren't really folders in s3 just file names. So you can name your file Key: `username/filename`, but it's not actually a folder. S3 will let you believe you've created a folder but all you've done is create a file name that starts with `usenrame/`. So visually this can look nice in s3, but it doesn't allow you to do any of the things you can do with folders. You won't be able to get all files from s3 inside a user's folder for example.
Yes it's a feasible approach, just prefex the object key with the `userid/` It looks like i was kind of wrong, you can get AWS to list files that start with a prefex, which is kind of like listing contents of a directory. just determine how you want to organize your files then when you name your files using the Key property, add a prefix to that. Something like this: const putObjectCommand = new PutObjectCommand({ Bucket: process.env.AWS_BUCKET_NAME!, Key: `${user.id}/${generateFileName()}` })
@SamMeechWard is correct here, S3 is object storage not a file system. You can however structure your file names however you wish. e.g. user-1/image-1.jpg
Great tutorial! Gained a sub from me. I am implementing this into my project and having trouble with it in production. In my localhost it is working fine. I did create a new bucket and IAM user for the production build and updated the env files in Vercel. I am getting 403 forbidden errors and errors about "An error occurred in the Server Components render. The specific message is omitted in production builds to avoid leaking sensitive details." Any help would be nice!
It's hard to know exactly what's going wrong from that message. make sure you `console.error()` all your errors in your app, then check the vercel logs to see more detailed error logs
I ran into the exact same problem. Looked at vercel logs and the actual error message is "Only plain objects, and a few built-ins, can be passed to Client Components from Server Components. Classes or null prototypes are not supported." The line that causes this is calling the getSignedUrl command from the @aws-sdk/s3-request-presigner library. Anyone know a fix for this?
am i the only one that feels lost when trying to add this to my project lmao. all the docs examples on payload are way too complex and not real intuitive
s3 guy is back lol, learnt uploads with your other video like a year ago, havent viewed this yet but cannt wait to check it out.
This TH-cam channel is gold, I recently saw the playlist where you explaining tests and mocking an express and and mocking a database. Your videos are gold man. One day I’m going to reach this point. You gained a subscriber in me.
Perfect timing! I was looking for a robust way to do exactly this. Thank you Sam!
Happy to help!
you have no idea how much you saved my life with every s3 video, you have the best content out there!
This video is very useful for my project thanks for sharing this! Also, love how detailed the steps are in your website.
Great tutorial! Just what i needed.
Understanding S3 was a breeze with your video mate, Thanks for this content!
Very very useful tutorial. I needed that for my project ! Thanks a lot !
Great tutorial! One thing to note; the crypto package adds SIGNIFICANT bundle size to the client, so the computeCheckSum function should be executed on the server
EDIT: Digging into this deeper, the file is an input to the checksum, so running the computeChecksum function on the server would defeat the purpose of this tutorial as the whole point was not having your files hit the serverless function... I guess the tradeoff would be to lazy load in the computeChecksum function only when a file is uploaded, ensuring that the initial server payload of your page is not bloated.
Wooo it was fantastic, learned a lot of things, thank you very much
22:29
Awesome. Thanks
with the file type check, you might as well do if type.includes("image") || type.includes("video"), which is kind of succinct.
GLAD YOU ARE BACK!!!!!!
🤘✌️
What's stopping a malicious user from sending a gazzilion requests to the image hosted on s3 and billing you a billion dollars?
Sam?
A potential subscriber here who really appreciates this video (and the exactly 1000th thumbed up viewer), but...
Any response for this truly legit question from 3 months ago?
@@Why-Ron75 yes i am also still waiting, decided to not do the project because cant get answer for my question here or anywhere
This was one of my concerns as well and had to use uploadthing
@@imransefat8770 I appreciate this comment! Thank you 🖖🏽
@@imransefat8770 thanks, how much did it cost you per user?
Would love to see an expansion of this that goes into multiple files at once and editing a post to swap out images etc.
I implemented this recently and feel like there must be a nicer way to do it than I did.
Hey maybe can you share a repo where you implemented multiple file upload ?
Yeah I'd also like to take a peek at that
Hi Sam,
Do you have a video or a course where you show how to set Private buckets and how to use them with NextJS ?
thanks for the video, was very helpful
Thanks for the video! Because you’re the s3 guy and have made a lot of videos about it, I think it’d be nice to make an extension video incorporating some of the stuff uploadingthing does and looking at their open source code and showing us how to implement some of the functionality they provide (whatever it is). Afaik, avoiding potential issues where users request a url but don’t upload or something goes wrong and there are a lot of ghost entries in the database or whatnot. I haven’t looked into it myself yet but it’s something I would want to do and a youtube video would be helpful I think to see how uploadthing does it and what problems they solve as an extension to this video. It’s kind of like the video that joshTriesCoding did, but with more code and implementing some of it on our own
Great video, very useful
so thorough! Thank you!
Very useful video, thank you so much!!! :)
I personally use SST for this, it takes a few extra steps but mostly it works if you full in wanna deploy things with AWS. But it simplifies things.
Do you use next js with SST?
Can you PLEASE do a video where you upload a file along with a filled out form - store the file locally and retrieve the file as part of data retrieval for presentation. All done in Nextjs. I'm currently stuck in this for a work project. Thanks in advance
look for multer npm for this with the drag and drop npm library
@@wchorski OK but I'm doing this with TypeScript... I hope it's smooth 👀
great video!
thanks a lot
Could You try this with DigitalOcean Spaces?
Hi Sam, thanks for the video. I am a product manager and interested in the logic and you gave me good understanding on how things work for social media post/view. One question in my mind was when you said making sure that this post belongs to this user but then you are showing on the home of the app. Why do you need to see if this post belongs to this user? do you mean no one can view the post except the user who created it? Thanks
Another question, can I bring up the CloudFlare R2 since they are S3 compatible. do you think we can use it in the same manner you have used S3 now? on the look of the documentations, it seems it can function the same way
Thanks for the video!
How did you autocomplete those several lines of code at a time when calling a function or making an object? Is it an extension or a setting?
really nice ❤
I wonder, for security reasons if you should lock down that bucket to not be public and instead generate TTL'd pre-signed read URLs for the items in that bucket. Also you could create this as an API behind and API gateway with cloudfront and cache the images so you don't get hammered with charges for egress from S3.
what happens if the url is expired in the database how do you handle that
10:38 ugh, looks like the whole aws identity stuff is completely different now. There's no security credentials tab anymore, and I think they want you to get temporary access keys through "roles" rather than how it's done in this tutorial. Still trying to find a tutorial that shows how to do that.
Edit: nevermind. I just realized I've been in the entirely wrong section this whole time. There's the IAM Identity Center, and just the plain old IAM. I think I need to be in the plain old IAM. Gotta love how confusing amazon makes this.
can you do it with lambda function
Why does uploading large files say like 160 MB, crashes my application and mainly next auth and I get JWT Session error and need to sign in again, and chery on top is my react query mutation is still running when all this os happening
Im doing the same process but I don't know why it is not generating presigned url, can anyone help me out here
Hey, if the image was AI generated and fetched from an API, I guess you’d have to upload it from the server directly to s3. Do you know any ressource on how to do that with next js ? All ressources I could find only cover the client-to-s3 situation
can you are a video about your website , where we are using mdx ? Website looks sick !!
I want to implement same any pointers ?
and does he have a discord ?
Hello friend, thank you for the video, the explanation is very clear! I am implementing the DALL-E 3 API and I want to save the image in a bucket, but I'm not sure whether to do it from the client or directly from the server. Do you think it's too costly to do it from the server using my server function or edge function? The images are around 2 megabytes in size.
hi ! i am currently trying to setup the S3Client config but i am having this error Argument of type '[{ region: string; credentials: { accessKeyId: string; secretAccesskey: string; }; }]' is not assignable to parameter of type '[S3ClientConfig] | []'. can someone tell what should i do or where am i doing wrong
what do you recommend for tracking the progress of the upload to show the user a uploading 53%... etc.
i ended up with doing the upload via xhr. what are your thoughts?
Nice video and content! I am also working on a NextJS frontend with a Python backend (NextJS wasn't doing it anymore for me, too much other processes like data parsing had to be done). I don't want to manipulate the DB from NextJS anymore (I mean I do allow for API calls that then in Python will do DB work). What would you recommend here?
U are goated. No cap
How does one organise files in s3 like for example
Userid (as folder name)
Img1
Img 2
etc so the files are referenced in folders under the user's id
There aren't really folders in s3 just file names. So you can name your file Key: `username/filename`, but it's not actually a folder. S3 will let you believe you've created a folder but all you've done is create a file name that starts with `usenrame/`. So visually this can look nice in s3, but it doesn't allow you to do any of the things you can do with folders. You won't be able to get all files from s3 inside a user's folder for example.
@@SamMeechWard what of the bucket key/folder architecture. Someone suggested that to me. Is it a feasible approach?
Yes it's a feasible approach, just prefex the object key with the `userid/`
It looks like i was kind of wrong, you can get AWS to list files that start with a prefex, which is kind of like listing contents of a directory.
just determine how you want to organize your files then when you name your files using the Key property, add a prefix to that. Something like this:
const putObjectCommand = new PutObjectCommand({
Bucket: process.env.AWS_BUCKET_NAME!,
Key: `${user.id}/${generateFileName()}`
})
@SamMeechWard is correct here, S3 is object storage not a file system. You can however structure your file names however you wish.
e.g. user-1/image-1.jpg
Great tutorial! Gained a sub from me. I am implementing this into my project and having trouble with it in production. In my localhost it is working fine. I did create a new bucket and IAM user for the production build and updated the env files in Vercel. I am getting 403 forbidden errors and errors about "An error occurred in the Server Components render. The specific message is omitted in production builds to avoid leaking sensitive details." Any help would be nice!
It's hard to know exactly what's going wrong from that message. make sure you `console.error()` all your errors in your app, then check the vercel logs to see more detailed error logs
@@SamMeechWard I fixed it by not having "-" as bucket name and added www. and just incase for my domain. Thanks again
I ran into the exact same problem. Looked at vercel logs and the actual error message is "Only plain objects, and a few built-ins, can be passed to Client Components from Server Components. Classes or null prototypes are not supported."
The line that causes this is calling the getSignedUrl command from the @aws-sdk/s3-request-presigner library.
Anyone know a fix for this?
how about multiple files at once?
Is there a way to block all public access and still make this work?
Yes you can used signed urls. That process is in my other s3 video
wanna see multipart upload , and a loading percentage bar
🔥
❤❤❤
❤
Nah man… public bucket. 6:06 and I’m out. You should have done pre-signed urls.
He did pre-signed urls later. What is your solution in this case? Do you prefer to keep s3 as private? How would you upload files then?
am i the only one that feels lost when trying to add this to my project lmao. all the docs examples on payload are way too complex and not real intuitive
Amazing! Thanks a lot !
Can you do in React with Amplify?