wow fantastic tutorial. thank you. I hit several speedbumps but solved all of them in the end! I will say for true no coders beware of the "backtick". that mistake cost me 2 hours. As I thinker with set up I may ask a future question but for now this has been excellent. 🙏
Absolute Brilliance - If you love no code and ever find yourself thinking why would I use Bubble's cost overhead with storage, this is a must-watch. Fast, to the point, and very helpful. This is the exact reason I love youtube.
Hey @Launchable AI , thank you for the helpful video. The second part of your video (where you create a custom file uploader) can be replaced with my free plugin Better Uploader. It exposes the selected files as Data URI's which can be used in the API connector to upload the files. If you are not uploading large video files, the encoding doesn't make a difference in terms of storage (for example, my application only supports uploading PDFs). If you want to avoid the encoding penalty, I am working on an action that can upload the binary data directly to your S3 bucket. Wishing you all the best!
Just stumbled across this very useful many thanks! My application is document management system, while bubble is quick to start the costs soon can become astronomical
That's awesome. I was just wandering if you could make a video on how to make the same thing but for uploading larger files (10mb+). I would really appreciate that.
Hey Mr. Vulpes. You can definitely make it work. For large files you need to use pre-signed URLs. iirc, the process is to use the s3 API to generate a URL, split your file into chunks, then send the chunks to the endpoint. I don't recall the details offhand. Perhaps in the not too distant future I'll have a chance to make a video of it, I'm not sure 🤷♂
Fantastic tutorial. Managed to get it to work - just one thing that is not ideal and wonder if you have any comment on this. Uploaded images take 3 or 4 seconds to load. In other words - if I upload an image to s3 and use that imge on a page - when I go to load the page, the page loads and then the image loads maybe 5 seconds later. Obviously, this is not ideal. I am using this so that I can upload images for my website to S3 rather than to bubble for a few reasons. For use on other sites as well as cost. Any tips on how to speed things up. In comparison - if I navigate to S3, select the file and select open - it's almost instant.
Hm, good question. You could look at something like CloudFront, or another CDN, which is more optimized for this sort of thing. I haven't tried it though. Another option would be a sort of hybrid approach: - store the images on S3 as the primary store - when they get loaded on Bubble, store them also on bubble - when they get loaded again, load from bubble (assuming they're now stored in "Bubble-native" storage - periodically clear bubble file storage - rinse and repeat This isn't as clean, but may be simpler to implement. If I think of a better solution, I'll let you know! And if you figure something out, please let us know how you get on!
Great tutorial! i'm just starting with Bubble and AWS and was able to make it work pretty easily. Does anyone know how I'd be able to store image in folders within S3. For example, have my files sorted as bucket/user-id/filename. The way I understand it is I'd need to pass the full path (folder+filename) as the key but then I get errors because (i think) API gateway's parsing the '/' char as another parameter... Help please :)
Thank for this very thorough tutorial. I am running into a problem where the uploaded images to S3 do not render on AWS (click on the file in S3, then click open, and I get garbled text), and subsequently cannot display in my repeating group. I'm able to select and upload files, see the name and UId, etc. but cannot render the images. Did I overlook a setting or permission with S3? Thanks!
This could be due to permissions on the bucket. If you're blocking public access (the default now, since the tutorial was made), you'll need to generate an access token. If you open the bucket, can you view the file by clicking the "Open" button, instead of the file name?
Hello, thank you for the tutorial. I was wondering if you could use a presigned URL to upload and eliminate the Get altogether? You can then save the URL to the database and then return the url into the repeating group?
I think so yeah. I've been talking to someone else about this. I've done something similar in the past (don't recall the specifics atm), so I know it's possible in Bubble. If you try it, let me know how it goes!
Great tutorial! Thank you! One question - when pulling data back into bubble, is there a way of speeding it up? I'm thinking of using external storage for images but when i call the images back into the app, its take quite a while to see them.
I don't know of a great way offhand. One thing I've done with images is this: - when you download the image from an external source, save it to the Bubble DB by attaching it to a thing - when loading, only fetch from the external source if the data thing doesn't have it's image already - periodically delete unaccessed/old images, to avoid just storing everything in Bubble The basic idea is just "cache" recent or commonly accessed images in Bubble. It's not a great solution lol, but I don't know of another one currently.
Can provide assistance with the AWS File Uploader Plugin? I have a template from Zeroqode and the AWS plugin, however, I’m having trouble with the Bubble side functions. I want to use the plugin to upload course promotional videos and lesson material for instructors. I’ve been working on this for two months now and haven’t been able to figure it out yet. If anyone could guide me or provide me with some helpful knowledge, I’d really appreciate it.
This is great thank you! When you do the "Run javascript" is that in the browser client side? If so in theory could the user grab that bubble-upload-demo url by inspecting the webpage then upload stuff to your s3 bucket?
That's correct, it's client side. In theory the user could grab the URL and upload to your bucket. So it's important to lock-down your API via CORS, to only allow requests coming from your app's domain.
Nice job you done there. Say, would it work the same for RDS SQL and would it be possible to automatically retrieve the URL once file uploaded in S3 and then input it to RDS SQL? Also, is it possible for multiple files upload to S3?
I think there is an RDS + API Gateway integration, if that's what you mean. As for multi file upload to S3, I don't recall atm if there's a multi-upload action/endpoint. You could, of course, just loop over files, but might be worth checking the docs for multi-upload if, eg, you're uploading lots of small files where iteration would be slow.
Great tutorial! However, I'm following every step but getting this error message, what would be the reason (The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.)? I've noticed that the API region is US (by default, I couldn't/didn't change it) but my bucket region is AP, does that make a difference?
Hey Linz. I'm not quite sure what's going on there. Are you getting that response in Bubble? S3 buckets are a bit tricky, region wise. As far as I can tell, sometimes they behave as global resources, and sometimes as regional ones, depending on what sdk or function you're calling. I'm sure this probably isn't the case, but I've been confused by it. You could try setting up another bucket in a different region (change your region in the top right of the AWS console). I don't think you can change a bucket's region once its created, but you can change your region before creating. If you want to share a bit more info on the error, maybe I can help more!
Hello, thank you very much for the tutorial. Is there any way to save in the bubble database the URL of the file that is uploaded to the bucket? Also, is there any way to organize the files by folders inside the bucket? Thank you very much.
I don't recall offhand whether the response from the API call returns the path of the file, but that's what I'd check first. If it doesn't include it, you can likely reconstruct it, because you know the bucket name and filename (key), which should be all you need.
Perfect! It worths each minute. I have been moving around and guarantee its content and didatic technique are very rare.Congratulations! ... Taking the chance... I am facing error message when trying to GET method (pls see blw)... Any clue would be very appreciated! Regards. API Connector There was an issue setting up your call. Raw response for the API Status code 403 {"message":"Missing Authentication Token"}
Glad it was helpful! Regarding the error, I'm not sure. There are a few things that could be going on there. The most likely is that you're missing the authentication header. Do you have a header named "Authentication", with a value of "Bearer abcd1234" (with the value of course changed)?
@@LaunchableAI I have this same error, and do not have an Authentication header. Should I add one and provide the value? Not sure where to find the value...
Hi! Thanks for extreme useful content. Do you know about wasabi? Can you make a video about the integration with them? I ask you because I read that aws could be risk with very large bills, is that right?
Glad you like it :) I didn't know about Wasabi, but thanks for pointing them out! They look like a very useful service. I'm going to explore them and see if they might fit into an upcoming project. Regarding large AWS bills, yes, that is a risk (I've had it happen). I'd recommend setting up budgets and alerts if you use AWS services, to monitor your spend.
What if I want to get a table of data from S3. I have a very large 200GB file and don’t know where to host it to call it and display on repeating group.
That's a great question. Can you split up the table/file, somehow? You won't be able to load 200GB at one go; no one's computer has that much memory. You'll likely need to do some incremental parsing. I'm not sure what the right approach would be. First thing I would try is paginating the results. Have a web server that loads the data, then loads increments (results 1-100, 101-200, etc) when you click "view next page". The memory problem will still be tricky though if you don't split the data though, b/c a server with 200GB of RAM will be expensive to run.
For some reason my files are not loading to the bucket from Insomnia. What could have gone wrong in the process? I followed every step in the video until that point.
Hm, not sure. Are you selecting multi-part form for your body, and the "file" type, for the file input? Is your endpoint + stage correct? I've gotten tripped up not having the full and correct endpoint, including stage/method on the end.
Thank you for this amazing tutorial! Subbed and bell turned on from now on! Is it possible to use this approach to populate a user profile page? For example. a Facebook like page that gets the data from AWS instead of bubble. I'm currently trying to build something like this but the bubble data prices are just insane...
Thanks for the kind words :) You could definitely do something like that. You could pull down profile pic, videos, etc., on the user's feed or profile. One thing to keep in mind (which I'm going to add to the description) is that with the "Get data from an external API setup", as used in this tutorial, Bubble will store the files you download in your app's file storage. So you will need to occasionally clean out your file storage. I've been trying to find a better way to do this.
@@LaunchableAI Is this way authenticated? As in, no one could abuse it from the front-end through the browser or anything. I want to be able to setup a CMS on my site where I can download or upload w/ API.
@@vibetribepdx4686 I don't spend much time on security in this video, other than to mention limiting CORS to your domain. This will offer some protection from abusing the API, so I would recommend doing it, but it's not complete. I'd also suggest implementing API keys via API gateway, which will help. (e.g., docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-setup-api-key-with-console.html). Those 2 mechanisms would probably offer sufficient security for most use cases. If you think you'd need greater security measures than this, let me know, and I'll try to cover options in more detail. Good luck with your CMS!
Thank you so much. This was one of the best tutorials I've seen so far. Please keep up the good work. I have been issues with setting the URL Path Parameters in Integration Request. I have the value set exactly how you did, but I get an errror saying that the path is invalid. Error: "Invalid mapping expression parameter specified: method.request.path.bucket" Seems like a weird error and been stuck on this for a long time, and wondering if you know anything about this
Sorry I missed your comment :s Did you figure it out? My suspicion would be a missing symbol somewhere in the path variable definition, but it's hard to say without seeing your setup :s
This is exactly what I've been looking for a couple of weeks. Great work!!! Thank you very much. Just a simple question: what's the limit size of the files to be uploaded if there is any?
Thanks :) Glad it's helpful. There is a limit of 10Mb when using the service integration. Thanks for bringing this up. I'm going to add a note to the video description. If you need to transfer larger files, the docs recommend generating a presigned URL with a lambda function, then using that to upload/download to/from S3. This would be a little bit more complicated, but not too bad.
@@LaunchableAI Great, thanks for the answer. I keep getting "has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status." after enabling CORS, any ideas with that. I've been trying several things but can't solve it
@Zeus Neila Yeah CORS issues are tricky. Have you tried the steps from the CORS section of the video (starting around 36:00)? First option, in API gateway, select your resource -> click actions -> click "Enable CORS". Second option, check the Method Response for the endpoints, and make sure that you have "Access-Control-Allow-Origin" for your domain (or all domains, with *). If you've done this for all resources and endpoints, and it's still not working, let us know!
@@LaunchableAI yup, already done it. At first I received the same error than the video. After doing the enable cors and make sure "Access-Control-Allow-Origin" is present for all endpoints (OPTIONS and POST) in Method Response, I get the "has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status." error. "Access-Control-Allow-Origin" is in Integration Response too with '*' for all endpoints.
ok, coming back to say it is working. The problem was I wanted to upload my image inside a folder within my bucket passing it as filename (filename = "folder/myimage.png"). Apparently it didn't like my idea. I'll try to figure out how to do it, but no more cors problems. Thanks anyway for your kind attention.
It's definitely doable. There's probably a few different ways you could do it. I think I'd first look at setting up an S3 trigger, so that whenever a file gets added to the bucket, a lambda function fires. You could call the Bubble workflow API or data API (e.g., set up an externally accessible back-end workflow, or write data directly to your database) from that lambda function, and pass whatever data you need. Here's a link to some AWS docs on how to start: docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html If you try it, let me know how you go!
@@LaunchableAI Awesome! and that has no limits to the size of the file, right? Also, I keep getting CORS errors whenever I try to use audiofiles in the sample uploader, it works with images, but not audio for some reason. I checked the Binary Media Types, and I have "audio/mpeg" added, but for some reason audio files won't go through. UPDATE: I fixed the issue, it now works. I am currently dividing my files into sections that are under the 10 MB limit for my MVP, but when I get to more advanced stuff, I'll try the Lambda function you mentioned. Thanks so much! Awesome tutorial!
@@therealdoy99 Glad you got it working! Note that if you don't want to split your audio files, you can use pre-signed URLs to upload larger files. See here if you're interested: docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html Good luck with your project!
@@LaunchableAI Awesome! Thank you so much! This just solved a bunch of massive problems on my app! The only thing is that the links expire after 7 days, is there any way around that?
@@therealdoy99 There's always a way lol. You can set an expiration time when you generate the link. So in python + boto3, for example: url = boto3.client('s3').generate_presigned_url( ClientMethod='get_object', Params={'Bucket': 'BUCKET_NAME', 'Key': 'OBJECT_KEY'}, ExpiresIn=3600) ExpiresIn is 3600 seconds, I think standard practice is to have a function that generates a pre-signed url as needed, and make it short-lived (e.g., an hour). So you could, for example, set up a lambda function that generates a pre-signed url for you bucket, and call that from the bubble workflow where you're uploading files. Then you use the returned URL as your upload endpoint. You can of course make them longer-lived (e.g., ExpiresIn=99999999999999999999), but this is less secure, because if anyone gets that link, they'll be able to write to your bucket forever. Hope this helps!
I didn't cover protecting the API with a key in this tutorial, though it may be a helpful topic to cover. Here's a reference from AWS outlining the different options you have to do so: docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-control-access-to-api.html In my experience, I most commonly see a 403 error when I've entered the wrong resource path. Can you double check that you're using the format {api_url}/{stage}/{resource} ? Let me know if that helps/doesn't. Good luck!
@@LaunchableAI I just wanted to get back to you, I started your tutorial again and I got it to work! Thank you so much for taking your time to make this tutorial.
wow fantastic tutorial. thank you. I hit several speedbumps but solved all of them in the end! I will say for true no coders beware of the "backtick". that mistake cost me 2 hours. As I thinker with set up I may ask a future question but for now this has been excellent. 🙏
Absolute Brilliance - If you love no code and ever find yourself thinking why would I use Bubble's cost overhead with storage, this is a must-watch. Fast, to the point, and very helpful. This is the exact reason I love youtube.
Thank you! Glad you liked it :)
Hey @Launchable AI , thank you for the helpful video. The second part of your video (where you create a custom file uploader) can be replaced with my free plugin Better Uploader. It exposes the selected files as Data URI's which can be used in the API connector to upload the files. If you are not uploading large video files, the encoding doesn't make a difference in terms of storage (for example, my application only supports uploading PDFs). If you want to avoid the encoding penalty, I am working on an action that can upload the binary data directly to your S3 bucket. Wishing you all the best!
Just stumbled across this very useful many thanks! My application is document management system, while bubble is quick to start the costs soon can become astronomical
Glad it's helped!
That's awesome. I was just wandering if you could make a video on how to make the same thing but for uploading larger files (10mb+). I would really appreciate that.
Hey Mr. Vulpes. You can definitely make it work. For large files you need to use pre-signed URLs. iirc, the process is to use the s3 API to generate a URL, split your file into chunks, then send the chunks to the endpoint. I don't recall the details offhand. Perhaps in the not too distant future I'll have a chance to make a video of it, I'm not sure 🤷♂
This is awesome! Thanks for making the video!
Glad you liked it!
Fantastic tutorial. Managed to get it to work - just one thing that is not ideal and wonder if you have any comment on this. Uploaded images take 3 or 4 seconds to load. In other words - if I upload an image to s3 and use that imge on a page - when I go to load the page, the page loads and then the image loads maybe 5 seconds later. Obviously, this is not ideal. I am using this so that I can upload images for my website to S3 rather than to bubble for a few reasons. For use on other sites as well as cost. Any tips on how to speed things up. In comparison - if I navigate to S3, select the file and select open - it's almost instant.
Hm, good question. You could look at something like CloudFront, or another CDN, which is more optimized for this sort of thing. I haven't tried it though.
Another option would be a sort of hybrid approach: - store the images on S3 as the primary store
- when they get loaded on Bubble, store them also on bubble
- when they get loaded again, load from bubble (assuming they're now stored in "Bubble-native" storage
- periodically clear bubble file storage
- rinse and repeat
This isn't as clean, but may be simpler to implement.
If I think of a better solution, I'll let you know! And if you figure something out, please let us know how you get on!
Great tutorial! i'm just starting with Bubble and AWS and was able to make it work pretty easily. Does anyone know how I'd be able to store image in folders within S3. For example, have my files sorted as bucket/user-id/filename. The way I understand it is I'd need to pass the full path (folder+filename) as the key but then I get errors because (i think) API gateway's parsing the '/' char as another parameter... Help please :)
Is there any way to upload files via the API connector? It kind of works but the image uploaded to S3 is empty after using the post api call
Thank for this very thorough tutorial. I am running into a problem where the uploaded images to S3 do not render on AWS (click on the file in S3, then click open, and I get garbled text), and subsequently cannot display in my repeating group. I'm able to select and upload files, see the name and UId, etc. but cannot render the images. Did I overlook a setting or permission with S3? Thanks!
This could be due to permissions on the bucket. If you're blocking public access (the default now, since the tutorial was made), you'll need to generate an access token. If you open the bucket, can you view the file by clicking the "Open" button, instead of the file name?
Thank you. When I click open, the file doesn't render. just a small square (like missing pic icon)
@@LaunchableAI
Hey mate, I have this same issue. Did you solve it in the end?@@dropT10
Did you ever figure this out? I have the same issue
I'm sorry I have not. I am using bubble's file manager for now. @@peterbrink4421
Hello, thank you for the tutorial. I was wondering if you could use a presigned URL to upload and eliminate the Get altogether? You can then save the URL to the database and then return the url into the repeating group?
I think so yeah. I've been talking to someone else about this. I've done something similar in the past (don't recall the specifics atm), so I know it's possible in Bubble.
If you try it, let me know how it goes!
Great tutorial! Thank you! One question - when pulling data back into bubble, is there a way of speeding it up? I'm thinking of using external storage for images but when i call the images back into the app, its take quite a while to see them.
I don't know of a great way offhand. One thing I've done with images is this:
- when you download the image from an external source, save it to the Bubble DB by attaching it to a thing
- when loading, only fetch from the external source if the data thing doesn't have it's image already
- periodically delete unaccessed/old images, to avoid just storing everything in Bubble
The basic idea is just "cache" recent or commonly accessed images in Bubble.
It's not a great solution lol, but I don't know of another one currently.
Can provide assistance with the AWS File Uploader Plugin? I have a template from Zeroqode and the AWS plugin, however, I’m having trouble with the Bubble side functions. I want to use the plugin to upload course promotional videos and lesson material for instructors. I’ve been working on this for two months now and haven’t been able to figure it out yet. If anyone could guide me or provide me with some helpful knowledge, I’d really appreciate it.
This is great thank you! When you do the "Run javascript" is that in the browser client side? If so in theory could the user grab that bubble-upload-demo url by inspecting the webpage then upload stuff to your s3 bucket?
That's correct, it's client side. In theory the user could grab the URL and upload to your bucket. So it's important to lock-down your API via CORS, to only allow requests coming from your app's domain.
He did it!
Nice job you done there. Say, would it work the same for RDS SQL and would it be possible to automatically retrieve the URL once file uploaded in S3 and then input it to RDS SQL? Also, is it possible for multiple files upload to S3?
I think there is an RDS + API Gateway integration, if that's what you mean. As for multi file upload to S3, I don't recall atm if there's a multi-upload action/endpoint. You could, of course, just loop over files, but might be worth checking the docs for multi-upload if, eg, you're uploading lots of small files where iteration would be slow.
Great tutorial! However, I'm following every step but getting this error message, what would be the reason (The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.)? I've noticed that the API region is US (by default, I couldn't/didn't change it) but my bucket region is AP, does that make a difference?
Hey Linz. I'm not quite sure what's going on there. Are you getting that response in Bubble? S3 buckets are a bit tricky, region wise. As far as I can tell, sometimes they behave as global resources, and sometimes as regional ones, depending on what sdk or function you're calling. I'm sure this probably isn't the case, but I've been confused by it. You could try setting up another bucket in a different region (change your region in the top right of the AWS console). I don't think you can change a bucket's region once its created, but you can change your region before creating.
If you want to share a bit more info on the error, maybe I can help more!
Hello, thank you very much for the tutorial. Is there any way to save in the bubble database the URL of the file that is uploaded to the bucket?
Also, is there any way to organize the files by folders inside the bucket?
Thank you very much.
I don't recall offhand whether the response from the API call returns the path of the file, but that's what I'd check first. If it doesn't include it, you can likely reconstruct it, because you know the bucket name and filename (key), which should be all you need.
Perfect! It worths each minute. I have been moving around and guarantee its content and didatic technique are very rare.Congratulations! ... Taking the chance... I am facing error message when trying to GET method (pls see blw)... Any clue would be very appreciated! Regards.
API Connector
There was an issue setting up your call.
Raw response for the API
Status code 403
{"message":"Missing Authentication Token"}
Glad it was helpful!
Regarding the error, I'm not sure. There are a few things that could be going on there. The most likely is that you're missing the authentication header. Do you have a header named "Authentication", with a value of "Bearer abcd1234" (with the value of course changed)?
@@LaunchableAI I have this same error, and do not have an Authentication header. Should I add one and provide the value? Not sure where to find the value...
Hi! Thanks for extreme useful content.
Do you know about wasabi? Can you make a video about the integration with them?
I ask you because I read that aws could be risk with very large bills, is that right?
Glad you like it :)
I didn't know about Wasabi, but thanks for pointing them out! They look like a very useful service. I'm going to explore them and see if they might fit into an upcoming project.
Regarding large AWS bills, yes, that is a risk (I've had it happen). I'd recommend setting up budgets and alerts if you use AWS services, to monitor your spend.
What if I want to get a table of data from S3. I have a very large 200GB file and don’t know where to host it to call it and display on repeating group.
That's a great question. Can you split up the table/file, somehow?
You won't be able to load 200GB at one go; no one's computer has that much memory. You'll likely need to do some incremental parsing. I'm not sure what the right approach would be. First thing I would try is paginating the results. Have a web server that loads the data, then loads increments (results 1-100, 101-200, etc) when you click "view next page". The memory problem will still be tricky though if you don't split the data though, b/c a server with 200GB of RAM will be expensive to run.
For some reason my files are not loading to the bucket from Insomnia. What could have gone wrong in the process? I followed every step in the video until that point.
Hm, not sure. Are you selecting multi-part form for your body, and the "file" type, for the file input? Is your endpoint + stage correct? I've gotten tripped up not having the full and correct endpoint, including stage/method on the end.
Thank you for this amazing tutorial! Subbed and bell turned on from now on! Is it possible to use this approach to populate a user profile page? For example. a Facebook like page that gets the data from AWS instead of bubble. I'm currently trying to build something like this but the bubble data prices are just insane...
Thanks for the kind words :)
You could definitely do something like that. You could pull down profile pic, videos, etc., on the user's feed or profile.
One thing to keep in mind (which I'm going to add to the description) is that with the "Get data from an external API setup", as used in this tutorial, Bubble will store the files you download in your app's file storage. So you will need to occasionally clean out your file storage. I've been trying to find a better way to do this.
@@LaunchableAI Is this way authenticated? As in, no one could abuse it from the front-end through the browser or anything. I want to be able to setup a CMS on my site where I can download or upload w/ API.
@@vibetribepdx4686 I don't spend much time on security in this video, other than to mention limiting CORS to your domain. This will offer some protection from abusing the API, so I would recommend doing it, but it's not complete. I'd also suggest implementing API keys via API gateway, which will help. (e.g., docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-setup-api-key-with-console.html). Those 2 mechanisms would probably offer sufficient security for most use cases.
If you think you'd need greater security measures than this, let me know, and I'll try to cover options in more detail.
Good luck with your CMS!
@@LaunchableAI how we can mutiple file uploads in one request?
Thank you so much. This was one of the best tutorials I've seen so far. Please keep up the good work.
I have been issues with setting the URL Path Parameters in Integration Request. I have the value set exactly how you did, but I get an errror saying that the path is invalid. Error: "Invalid mapping expression parameter specified: method.request.path.bucket" Seems like a weird error and been stuck on this for a long time, and wondering if you know anything about this
Sorry I missed your comment :s Did you figure it out? My suspicion would be a missing symbol somewhere in the path variable definition, but it's hard to say without seeing your setup :s
This is exactly what I've been looking for a couple of weeks. Great work!!! Thank you very much. Just a simple question: what's the limit size of the files to be uploaded if there is any?
Thanks :) Glad it's helpful.
There is a limit of 10Mb when using the service integration. Thanks for bringing this up. I'm going to add a note to the video description.
If you need to transfer larger files, the docs recommend generating a presigned URL with a lambda function, then using that to upload/download to/from S3. This would be a little bit more complicated, but not too bad.
@@LaunchableAI Great, thanks for the answer. I keep getting "has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status." after enabling CORS, any ideas with that. I've been trying several things but can't solve it
@Zeus Neila Yeah CORS issues are tricky. Have you tried the steps from the CORS section of the video (starting around 36:00)? First option, in API gateway, select your resource -> click actions -> click "Enable CORS". Second option, check the Method Response for the endpoints, and make sure that you have "Access-Control-Allow-Origin" for your domain (or all domains, with *). If you've done this for all resources and endpoints, and it's still not working, let us know!
@@LaunchableAI yup, already done it. At first I received the same error than the video. After doing the enable cors and make sure "Access-Control-Allow-Origin" is present for all endpoints (OPTIONS and POST) in Method Response, I get the "has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status." error. "Access-Control-Allow-Origin" is in Integration Response too with '*' for all endpoints.
ok, coming back to say it is working. The problem was I wanted to upload my image inside a folder within my bucket passing it as filename (filename = "folder/myimage.png"). Apparently it didn't like my idea. I'll try to figure out how to do it, but no more cors problems. Thanks anyway for your kind attention.
Is there a way to sync the files in my S3 database to a database within bubble? I am trying to make an audiobook app.
It's definitely doable. There's probably a few different ways you could do it. I think I'd first look at setting up an S3 trigger, so that whenever a file gets added to the bucket, a lambda function fires. You could call the Bubble workflow API or data API (e.g., set up an externally accessible back-end workflow, or write data directly to your database) from that lambda function, and pass whatever data you need.
Here's a link to some AWS docs on how to start: docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html
If you try it, let me know how you go!
@@LaunchableAI Awesome! and that has no limits to the size of the file, right? Also, I keep getting CORS errors whenever I try to use audiofiles in the sample uploader, it works with images, but not audio for some reason. I checked the Binary Media Types, and I have "audio/mpeg" added, but for some reason audio files won't go through.
UPDATE: I fixed the issue, it now works. I am currently dividing my files into sections that are under the 10 MB limit for my MVP, but when I get to more advanced stuff, I'll try the Lambda function you mentioned. Thanks so much! Awesome tutorial!
@@therealdoy99 Glad you got it working! Note that if you don't want to split your audio files, you can use pre-signed URLs to upload larger files. See here if you're interested: docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html
Good luck with your project!
@@LaunchableAI Awesome! Thank you so much! This just solved a bunch of massive problems on my app! The only thing is that the links expire after 7 days, is there any way around that?
@@therealdoy99 There's always a way lol. You can set an expiration time when you generate the link. So in python + boto3, for example:
url = boto3.client('s3').generate_presigned_url(
ClientMethod='get_object',
Params={'Bucket': 'BUCKET_NAME', 'Key': 'OBJECT_KEY'},
ExpiresIn=3600)
ExpiresIn is 3600 seconds, I think standard practice is to have a function that generates a pre-signed url as needed, and make it short-lived (e.g., an hour). So you could, for example, set up a lambda function that generates a pre-signed url for you bucket, and call that from the bubble workflow where you're uploading files. Then you use the returned URL as your upload endpoint.
You can of course make them longer-lived (e.g., ExpiresIn=99999999999999999999), but this is less secure, because if anyone gets that link, they'll be able to write to your bucket forever.
Hope this helps!
At the 25 min. mark, you did not show how to add the API key for testing. I'm getting a 403 error. Thank you for your time and this tutorial.
I didn't cover protecting the API with a key in this tutorial, though it may be a helpful topic to cover. Here's a reference from AWS outlining the different options you have to do so: docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-control-access-to-api.html
In my experience, I most commonly see a 403 error when I've entered the wrong resource path. Can you double check that you're using the format {api_url}/{stage}/{resource} ? Let me know if that helps/doesn't. Good luck!
@@LaunchableAI Thank you for responding! I will check that and get back to you.
@@LaunchableAI I just wanted to get back to you, I started your tutorial again and I got it to work! Thank you so much for taking your time to make this tutorial.
@@marcosmonteiro6470 Awesome! Glad you got it running 🥳
@@LaunchableAI @Marcos Monteiro I have the same problem .... any video that you can recommend?