@@WebDevCody yeah that's what I'm learning now. here's a thought: am I safer with the safer platform that seems to have less of a problem with DDoS but seems less willing to refund, or the f'ups that are vercel/netlify that are scared of the negative publicity and seemingly will... +1 for vps rn
@@twitchizle if we just delete the iam user that is using the services, will it not charge? Or will it still charge because we have a cluster or bucket assigned?
I like how people rant about cloud solutions. And there is a good example how you can go bankrupt. And we're back on square one - on-premise is the powerhouse of the cell. I prefer dead server and downtime than pay money for stuff I do for fun.
Wouldn’t you have this same issue on prem anyways? If you don’t have any sort of rate limiting or DDOS protection then it doesn’t matter whether you’re on-prem or on cloud. Your machines will end up using more resources like electricity, your cooling systems will need to work harder, and your application will become unavailable to other users, irrespective of whether it’s cloud or not. Using the cloud might even have been beneficial in this case because cloud can easily scale
This. The main reasons I'd go for a cloud solution is if low latency is a must, or if you need a lot of scalability, or if you lack the hardware expertise to manage a server.
@@rico454 i agree, the cloud will for sure scale your wallet no matter what ^^ unless you make money with something that needs to be online no matter what happens it will only cost you money if it is online no matter what happens, scaling will come with a very big cost at the end if you are not careful enough and uncontrolled scaling can hurt a lot more than a server running a little bit warmer for some seconds.
@@rico454 In what world electricity bill could be hundreds of thousands of dollars? Beside, there is a upper bound to how much on-premise can scale, if you get DDoS-ed and you run out of resource, your server simply goes down. The danger of cloud is that you have access to seemingly unlimited resource and as a result, you don't have an option to just let the server die.
You're getting in there. The perfect setup is a VPS with Cloudflare as a reverse proxy. Cloudflare also provides a way to write IaC through their Wrangler CLI, if that's your cup of tea. One thing is for sure, all this usage pricing nonsense would go out of the window. A WAF for $3000/month?!!!!!
I feel you, and somewhat agree, but I don't think it's the right solution for everyone. Most people are on free tiers forever that won't incur in payments or have built-in killswitches, by the time they have a "scare" like his and go 250 overbudget(if that happens), you would be even with them after all those years of VPS's not to mention the hours you spent both learning and maintaining.
@@alexanderhuliakov6012 Used it a lot for my projects, and you don't notice it much at all. The physical location of your server vs. the client would have a much bigger impact than this
When you get a DDOS attack you will probably see a large number of different source IPs. Maybe a dynamic rate limit based on the last x hours with an minimum and maximum can do the trick. If you move to a different solution, I would be happy to see it in oneof your videos.
@@StanOgn there are articles where you can read how to configure Cloudflare and Vercel. The main issue is caching, so to avoid two layers of caching (in Cloudflare and Vercel), some adjustments are needed.
I think it's great you are sharing this experience. It really drives home for me the importance of always needing to consider rate limiting up front when beginning a project. Probably even best to implement that rate limiting at the application level too.
I think this is the solution. Isn't nextjs middleware capable of setting request limits. I just asked chat gpt and got: // middleware.js or a specific middleware file in your Next.js project import { NextResponse } from 'next/server'; // Example of a very basic in-memory rate limiter const rateLimitMap = new Map(); export function middleware(request) { const ip = request.ip; const currentTime = Date.now(); if (rateLimitMap.has(ip)) { const { lastRequestTime } = rateLimitMap.get(ip); if (currentTime - lastRequestTime < 1000) { // Limit to 1 request per second for example return new Response('Rate limit exceeded', { status: 429 }); } } rateLimitMap.set(ip, { lastRequestTime: currentTime }); return NextResponse.next(); }
@@belkocik It's not fun with initial setup, but it's not that difficult either. A bit more tricky if you want to do everything right with not running as root and per-app users in terms of getting NodeJS up and running for those accounts. My usual setup was something like Domain from NameCheap CloudFlare as DNS, with proxying for protection NGINX as reverse proxy to multiple NodeJS apps
a vps with containers is the best thing you can do to reduce cost. that saved me from a huge bill.. Strongly advice you to migrate to a vps or use it for new projects
I don't understand, I thought Cloudfront had a defaut DDOS protection like Cloudflare, if don't it's scare me. Would you post a video setting Cloudflare in your project? Because I search a lot for it include in official SST discord and I had not find a proper way to do it.
@@WebDevCodythey handle everything basically if you know what you’re doing basically you can set up a reverse proxy on your backend and block all requests bypassing cloudflare
@Kimitri They do have it. I use CloudFront professionally and we recently got bitten by quite a few requests passing through WAF without getting blocked. It turns out that WAF takes a while to detect that something is going on, meaning in that few seconds it takes to react, hundreds of thousands of requests would still pass by unrestricted (even if you tell it to ratelimit after 10000 request per minute). With ratelimiting it appears to check for violations after X time and not for every request. So if an attacker is able to send 30000 requests in 30 seconds, it’s likely all of them would pass through even though you restrict at 10000. After those 30 seconds it would detect some IP sent way more than 10k requests and then blocks it. It seems more retroactive than proactive in that regard.
Nice insights for AWS & SST, thanks! I've mainly deployed via EC2 so far and usually without load balancing if it's just a side project. What I use then to handle simple DoS attacks/request spams are some iptables conntrack rules on the actual machine / in the deployed container. The single container also helps having a predictable bill, but ofc won't scale. Whenever a side project actually gets lots of traffic, then I'll upgrade to multiple instances with load balancing.
It pains me when I hear about bad actors. I mean we have enough we are dealing with but bad people are real. They are very real and your situation is real. I am sorry this happened to you. On a good side you found a valuable lesson through this and share it with us.
Yeah, just kind of sucks that you just can’t trust anyone. At least there’s way of prevent it I was just too lazy to actually do the work on my side projects when I should have done it from the get-go.
if you run npm run build on the server itself it will use a good amount of memory assuming it's one of the lower tiers so make sure you build it off the vps and transfer the build files or bake everything pre-built into a container
I think there's is an option to just kill the ec2, lambdas, distributions once it reaches a certain budget. I think it's called budget actions? Like we can set alarms in budget, also actions can be set.
I never used Cloudflare so I have one question if someone can answer me. So, to set Cloudflare to protect my Nextjs application against DDOS it will work like a CDN or a Reverse Proxy? If CDN so basically it will be a CDN above a CDN because SST use Cloudfront ? And if a Reverse Proxy, so basically I'll lose the pros of use Cloudfront to distribute the static content around the world because all the requests will comes from Cloudflare ?
@@WebDevCody I watched half of your video, and just suggested to put a Cloudflare proxy to avoid these situations. Cloudflare has a free plan that gives the DDOS protection. just the free plan has simple Bot mitigation. From docs: All Cloudflare plans offer unlimited and unmetered mitigation of DDoS attacks. Customers are not charged for attack traffic ever, period. There’s no penalty for spikes due to attack traffic, requiring no chargeback by the customer.
How come you dont use Vercel, Netlify, or something similar? I used to use a cheap VPN for everything, then moved to Vercel for the convenience of CI/CD, DDoS protection, scalability etc.
@@nickwoodward819 Yeh they do in some cases, but as we've learned from twitter, tweeting about it gets that sorted fairy quick. But for what they offer for the price, its extremely entising
Can't you use the free plan from Vercel? Since it doesn't require a payment method (ex. credit card,) you won't get charged. Or is the free plan too limiting?
@@karlembeastusing the free plan can still get you charged though most likely. Just because you don’t have a linked card doesn’t mean you don’t sign a contract with them. It’s likely illegal to just not pay, you’d need to speak to their support
I usually do the poor mans (hard budget limit) DDOS protection for my side-projects. If I go over $10 bucks then just shut down the project. 😀😅. Great video though.
Id totally use a vps. It’s cheap. It’s fun to setup; it gives you a ton of flexibility. And it’s just 20 bucks a month including a Plesk license. VPS all the way
VPS with containers where maybe putting restrictions on the container on how much resources it would scale because I've seen single container go up upto 300% and bring down everything
For the size of your projects, I would go with a VPS to keep it as simple as possible (less moving parts). At least until you start having millions of requests a day and making a boatload of money. I understand the fun on setting this systems up, but maybe, just maybe, this time you over-engineered it. Still great insight so I appreciate the video. Thanks.
That sucks. Have you thought about having your app in a docker container and let the service manage the amount of cpu cores depending on load but you can set a max cap? I've never done it before so I'm curious on your input if you've done something similar and if there are drawbacks.
Don't do it, not worth it on AWS. First off, you have Fargate and if you want to autoscale that it's complicated and expensive anyways (1vCPU + 1GB RAM is 20$ I think), you will then also need an IP which is 3.5$ + you will need a load balancer, which is 20$ + 2 IP (7$) for a grand total of ~50$/month. Congrats, you pay 50$ for 1vCPU and 1GB RAM. It is absolutely criminal. For reference: on Hetzner you can get a server with Intel Xeon E5-1650V3, 256GB RAM + 2 500GB Datacenter SSD disks for this price. Pro tip: Don't ever fall for lambda. Yes, you get 1M requests free, but you are locked in, have a weird programming model + if something like in the video happens and you fuck up caching/don't cache some stuff, AWS will skin you alive. Pro tip 2: Don't ever host your shit on AWS, pick Linode or something similar and put CloudFlare in front. Hell, pay 5$/month on DigitalOcean and you will get CD for free + have a way better experience because of UI.
About 2 years ago I reached the limit of my free tier firebase just by forgetting my dependency array in a useEffect in React .... very very quickly! . Anyway... it took me quite some time to figure out what what happening. Maybe someone tried to access your api in this way by accident?
Why cloudfront didn't rate limit it? It has rate limit by default, you can't write while (true) fetch. script, after few request's it will start returning 429 error So was all this bandwidth made from thousand of IPs or how?
I believe like he briefly mentioned in the video that he didn’t use Cloudflare. Likely used Route53. Which out of the box just comes with DNS and not WAF. You’re correct, with CF you get an ootb firewall - not sure how good it is but I know it works at least. He was asking in the video if he should use cloud front, probably is a good plan
@@aspirine17 did he say that though? Like I say, he likely just used Route53. If he does, yeah the default settings should protect. Though personally I add additional rate limits and block suspicious activity in cloudflare for free under their “10 free rules”
Amazon could just do what everyone has been asking for years and make a killswitch for when the bill goes over a threshold but then again Bezos may need to buy a new space weiner so nevermind
DoS and DDoS are two different attacks. I think you got the DoS attack, not DDoS if most of the requests are coming from a few IP addresses. DDoS is not easily preventable using simple rate limiting. To prevent DDoS, consider dynamic rate limiting like exponential backoff strategy, mix of global limits, per IP & per user rate limits, browser checks and ML based captchas, CSRF tokens, no direct IP access (domain-bound to server app), no direct A record to DDoS mitigator like Cloudflare - use AAAA record and set firewall to block IPv4 to your virtual servers and Cloudflare will handle IPv4 traffic and route it to your IPv6 only servers. These will prevent IPv4 based devices/botnet to join the DDoS attack.
Ah you just prompted me to check my hobby account and people have been spamming my sites with tons of bots trying to find vulnerable wordpress paths. :( It's a placeholder site that I don't share the URL of! Nothing is safe :(
Cody I think CF has got DDOS protection on free plan too... Also can you try hetzner. I am currently hosting the server on it used by like 2000-3000 folks at my college. Some things I miss is dev preview deployments, edge middleware... Speed is descent
i've seen a couple of other comments asking this, but why wouldn't you use something else like Cloudflare to proxy all of your traffic before it hits AWS?
Oh, I hadn't gotten to the point in the video before commenting this. You mentioned cloudflare has DDoS protection for $20/mo. That's actually not entirely true, you *can* get DDoS protection for free, though I'm not sure what the licensing agreement is (commercial uses etc etc)
Are things like Vercel free tier really not capable of providing enough functionality for what you need, especially since these are "free side-projects" ?
you can't host commerical applications which make money (which my side projects do sometimes make money) on vercel free plan according to their license.
I'm assuming all cloud solutions have a similar setup/cost. I don't have experience with this but does putting an extra anti ddos vps in front of aws help? No idea on the cost but then you don't have to move your apps. Curious what path you will take. Great video!
To be fair that’s a great idea. It comes with the drawback of too many requests will just make it slower and slower and timeout requests. It would just be like a proxy that filters through traffic. Great idea, I’d love to hear more about this
@@oSpam It's a terrible idea and it defeats the purpose of CloudFront. You have CloudFront (edge locations) to serve as proxy close to the user and cache the response. For example - you host your site in N. Virginia and I make a request in Italy. Request will go from my home, to the nearest data center to my location, to US, come back to data center, be cached and come to me. The next request a user close to me will make, will not make the round trip to US, but will just return the same response that got cached. If you put a proxy in front of cloud front in N. Virginia for example, my request will go from my home, to the proxy in US, to cloudfront edge location, to the server in US, be cached in US cloudfront, be cached on the US proxy, and only then be returned to me. The next request will have to go to US. If you have a static site, you would ideally cache everything for a long time on cloudfront and for some time in the browser, and on each deploy, invalidate the cloudfront cache if needed. If you get DDoSed, you will just pay a lot, but there is nothing you can do in that situation.
I think you can configure you aws to hard shutdown if billing go above a defined value, but I'm not so sure. This can work for pet projects that you don't care if they go off. This can prevent creating an infinite loop while developing and getting an infinite bill too.
@@Dontcaredidntask-q9m yeah after that I went looking, and it seems that I have to code a hard shutdown myself using lambda and billing event. I think I'll try doing that for my pet project. Not really a hard shutdown but just turning off the public access of any api
@@SogMosee I don't know much about cloudfront but with aws sdk you can do pretty much anything that you can do in the console, so in the worst case you can delete the cloudfront configuration.
@@zeusvargasLaughably only if you *pay* for the Pro tier. So they're admittedly leaving you exposed as a customer because you're on the wrong tier. Vercel are a joke.
@@nickwoodward819if you don't pay for the pro tier, they don't have your CC. so there is no way for them to charge you if your hobby tier site gets ddosed.
@@maitre999 Not sure that I care what you think if that's the level of English you use to express your thoughts, but just for the hard-of-thinking at the back: *The cost of the pro tier quite obviously doesn't justify the gutting of basic spending controls in the free plan. No one should expect to be left vulnerable to massive bills because they haven't upgraded* It could be $3 and there would be no justification for *not* having spending caps when they're obviously available.
doesn't digialocean charge a fixed amount per month? I'm not sure why people insist on running AWS (or any cloud) for small projects. Even a cheap shared hosting will do in most cases.
The thing is that free is still cheaper than cheap. This definitely helps weigh the pros and cons, but I’ll still use google cloud run with a max instance of 1 before I use a VPS.
I still remember the day webdevcody told he will never use a bare server for any deployment. Now he realises the risk in not using a simple vps in place for aws services. I have this fear all day when I use aws which make sure i learned how to setup and maintain a $5 vps 😂
i think you should move your frontend to the edge and backend to your own vps, and it would be much cheaper if your backend does not eat too much resources, like when you write them in go or rust
@@WebDevCodyCDn? Yeah you’re right. They probably didn’t watch the video 😅 I think too many people are suggesting bad rush moves. They need to watch your other video on why to never use a VPS for production 😂
have a script that shuts down the machine if get a billing alert etc. But yeah, I wont touch aws for personal stuff. But yes of course, your service goes down during that time. But that's fine for personal stuff. And your service might be down from the ddos anyway!
This services are not meant for hobby projects. I allocate a fixed amount of money per month on my hobbies: books, manga, and my two websites. I make sure I never spend more than that
They do. The budgeting service he showed has the option to cut off all services at a certain price. He may not want to do that for what I can only guess to prevent disruption for users.
Even if you pay for waf I think they still charge you per request? Idk you’d think off of this would be very black and white to understand, but it’s not
I`m looking into AWS for their GPU instances(AI stuff) - otherwise I woudnt touch that behemoth with a 10 foot pole - too confusing,too big,too enterprisey and huge risk of this stuff happening if you dont have a PHD in AWS.
Please make a video of - your deployment process to AWS including docker config and shell scripts for full-stack and backend projects. - setup with cloudflare - setup with services like digital ocean Thank you 🙏🙏
I keep seeing AWS horror stories for normal people and small businesses. Seems absolutely ridiculous to not have a way to set a cap. I have an app that is going to need infrastructure setup soon and I am wondering if there are any other options.
AWS are probably the one Sponsoring the attack. There is a conflict of interest if you have to pay cloud service provider for ddos attach. You got attack now you have to pay for a service
@@doreto95 Didn't they recently implement soft and hard limits (at least I'm sure Vercel did) because of a scandal where somebody got charged $100k on their hobby project?
Incorrect, you should change this comment - Vercel and Netlify *DO* charge you for attacks, and if their protection was adequate enough they wouldn't provide spending caps to Pro account holders like it was a benefit
Hey guys, sure Netlify or Vercel charge for execution time etc. BUT they handle DDOS protection for you without you having to pay extra. AND they are willing to not charge users if their protection failed.
from all the comments I've read from people smarter than myself, you should probably just use cloudflare.
Does cloudflare protect backend routes? Say when using astro? I'm guessing not and they have a spend cap?
@@nickwoodward819 It'll always protect, unless you disabled prot for that record or ddos ips
@@nickwoodward819it sits in front of ALL your stuff; you point your DNS to cloudflare and it proxies to your API or UI.
@@WebDevCody yeah that's what I'm learning now. here's a thought: am I safer with the safer platform that seems to have less of a problem with DDoS but seems less willing to refund, or the f'ups that are vercel/netlify that are scared of the negative publicity and seemingly will...
+1 for vps rn
do a video on how to add cloudflare
lol “$3,000/month so that’s obviously not gonna happen.” I’m sorry, I burst out laughing lol
@@nobody124... huh?
Wtf 😢
Relax. @@nobody124...
they way he said it, just so nonchalant, actually funny
@@nobody124... you dropped this 🤴🏿
getting a huge bill like one of those infamous 100k+ dollars bill aws stories is what scares me
you dont have any user, you wont get one of those
@@twitchizle if we just delete the iam user that is using the services, will it not charge? Or will it still charge because we have a cluster or bucket assigned?
@@daphenomenalz4100 yes they will charge if any instances are active and users are hitting your website.
It's the $5000 bill that will end your start up but not get enough traction on social media that should scare you
Set budget alarms.
I like how people rant about cloud solutions. And there is a good example how you can go bankrupt. And we're back on square one - on-premise is the powerhouse of the cell. I prefer dead server and downtime than pay money for stuff I do for fun.
Wouldn’t you have this same issue on prem anyways? If you don’t have any sort of rate limiting or DDOS protection then it doesn’t matter whether you’re on-prem or on cloud. Your machines will end up using more resources like electricity, your cooling systems will need to work harder, and your application will become unavailable to other users, irrespective of whether it’s cloud or not.
Using the cloud might even have been beneficial in this case because cloud can easily scale
This. The main reasons I'd go for a cloud solution is if low latency is a must, or if you need a lot of scalability, or if you lack the hardware expertise to manage a server.
@@rico454 nah, they aren't likely to be auto scaling, so the service will just go down rather than produce a massive bill
@@rico454 i agree, the cloud will for sure scale your wallet no matter what ^^
unless you make money with something that needs to be online no matter what happens it will only cost you money if it is online no matter what happens, scaling will come with a very big cost at the end if you are not careful enough and uncontrolled scaling can hurt a lot more than a server running a little bit warmer for some seconds.
@@rico454 In what world electricity bill could be hundreds of thousands of dollars? Beside, there is a upper bound to how much on-premise can scale, if you get DDoS-ed and you run out of resource, your server simply goes down. The danger of cloud is that you have access to seemingly unlimited resource and as a result, you don't have an option to just let the server die.
You're getting in there. The perfect setup is a VPS with Cloudflare as a reverse proxy. Cloudflare also provides a way to write IaC through their Wrangler CLI, if that's your cup of tea. One thing is for sure, all this usage pricing nonsense would go out of the window. A WAF for $3000/month?!!!!!
I feel you, and somewhat agree, but I don't think it's the right solution for everyone.
Most people are on free tiers forever that won't incur in payments or have built-in killswitches, by the time they have a "scare" like his and go 250 overbudget(if that happens), you would be even with them after all those years of VPS's not to mention the hours you spent both learning and maintaining.
Is using Cloudflare as a reverse proxy ok for API? It's at least more latency for all requests, right?
@@alexanderhuliakov6012its good for rate limiting thepublic api, but yeah its adding more latency
@@alexanderhuliakov6012 Used it a lot for my projects, and you don't notice it much at all. The physical location of your server vs. the client would have a much bigger impact than this
Great video! Would love to see a video on VPS setup and how you would migrate over. Thinking about doing the same with my side projects
When you get a DDOS attack you will probably see a large number of different source IPs. Maybe a dynamic rate limit based on the last x hours with an minimum and maximum can do the trick.
If you move to a different solution, I would be happy to see it in oneof your videos.
Would having Cloudflare as a first layer would have prevented it?
If correctly set yes.
yeah probably, everyone here says their service is great
@@WebDevCodycloudflare is literally the best we use it in production for 4M monthly active users site
@@WebDevCody what about if it was on vercel, they say they have some ddos protection...
@@StanOgn there are articles where you can read how to configure Cloudflare and Vercel. The main issue is caching, so to avoid two layers of caching (in Cloudflare and Vercel), some adjustments are needed.
I think it's great you are sharing this experience. It really drives home for me the importance of always needing to consider rate limiting up front when beginning a project. Probably even best to implement that rate limiting at the application level too.
Yeah I obviously need to make my side project more precessional before I bankrupt myself
Just love all the content you produce and so appreciative of it.
I think this is the solution. Isn't nextjs middleware capable of setting request limits. I just asked chat gpt and got:
// middleware.js or a specific middleware file in your Next.js project
import { NextResponse } from 'next/server';
// Example of a very basic in-memory rate limiter
const rateLimitMap = new Map();
export function middleware(request) {
const ip = request.ip;
const currentTime = Date.now();
if (rateLimitMap.has(ip)) {
const { lastRequestTime } = rateLimitMap.get(ip);
if (currentTime - lastRequestTime < 1000) { // Limit to 1 request per second for example
return new Response('Rate limit exceeded', { status: 429 });
}
}
rateLimitMap.set(ip, { lastRequestTime: currentTime });
return NextResponse.next();
}
interesting to see you change from being AWS enthusiast to seeing the value of a VPS. Am a fan of Vultr for these types of systems.
hey cody a full tutorial on switching to a VPS would be really good!
Would love too see how to set up a VPS
@@belkocik you put your shit in docker and if you want multiple webapps you use nginx to map the ports to their correct domain, thats pretty much it.
@@belkocik It's not fun with initial setup, but it's not that difficult either. A bit more tricky if you want to do everything right with not running as root and per-app users in terms of getting NodeJS up and running for those accounts.
My usual setup was something like
Domain from NameCheap
CloudFlare as DNS, with proxying for protection
NGINX as reverse proxy to multiple NodeJS apps
Funny how technology goes full circle. Back to on premise, it might be slower but it's great for free products.
a vps with containers is the best thing you can do to reduce cost. that saved me from a huge bill.. Strongly advice you to migrate to a vps or use it for new projects
That's the main reason I avoid pay as you go plans - a ddos attack could really drain your wallet.
I don't understand, I thought Cloudfront had a defaut DDOS protection like Cloudflare, if don't it's scare me. Would you post a video setting Cloudflare in your project? Because I search a lot for it include in official SST discord and I had not find a proper way to do it.
Cloudfront has shield standard which says they handle some ddos 🤷♂️
@@WebDevCodythey handle everything basically if you know what you’re doing basically you can set up a reverse proxy on your backend and block all requests bypassing cloudflare
@Kimitri They do have it. I use CloudFront professionally and we recently got bitten by quite a few requests passing through WAF without getting blocked. It turns out that WAF takes a while to detect that something is going on, meaning in that few seconds it takes to react, hundreds of thousands of requests would still pass by unrestricted (even if you tell it to ratelimit after 10000 request per minute). With ratelimiting it appears to check for violations after X time and not for every request. So if an attacker is able to send 30000 requests in 30 seconds, it’s likely all of them would pass through even though you restrict at 10000. After those 30 seconds it would detect some IP sent way more than 10k requests and then blocks it. It seems more retroactive than proactive in that regard.
Nice insights for AWS & SST, thanks!
I've mainly deployed via EC2 so far and usually without load balancing if it's just a side project. What I use then to handle simple DoS attacks/request spams are some iptables conntrack rules on the actual machine / in the deployed container. The single container also helps having a predictable bill, but ofc won't scale. Whenever a side project actually gets lots of traffic, then I'll upgrade to multiple instances with load balancing.
It pains me when I hear about bad actors. I mean we have enough we are dealing with but bad people are real. They are very real and your situation is real. I am sorry this happened to you. On a good side you found a valuable lesson through this and share it with us.
Yeah, just kind of sucks that you just can’t trust anyone. At least there’s way of prevent it I was just too lazy to actually do the work on my side projects when I should have done it from the get-go.
If you ever decide to host your nextjs projects in a vps, would you please make a little bit detailed video on that?
Yeah doing it soon
Yeah a VPS is probably the most reasonable choice in this scenario. Thanks for sharing your experience, I think I'm going to move my projects as well.
But a lot of VpS providers still charge Ingress fees. So the 1TB of data might have still added up? Any confirmation on this?
@@oSpamdepends on the provider. Vultr is $0.01 per GB when going over your plan's limit(2TB or more)
would cloudflare stop this ? , also would love a tutorial on a Digital Ocean type VPS droplet , especially how to host multiple sites
Sheild "standard" is an option in the CloudFront settings. It may not have been enabled.
I'm pretty sure it's always enabled by default
if you run npm run build on the server itself it will use a good amount of memory assuming it's one of the lower tiers so make sure you build it off the vps and transfer the build files or bake everything pre-built into a container
Cloud flare gives it for free .. create a few nodes K8s cluster using tailscale and use cloud flare in front of it and you are golden.
What about Digital Ocean? They even have applications tab now, without the need for us to set up the servers. Let me know if I am wrong :)
I think there's is an option to just kill the ec2, lambdas, distributions once it reaches a certain budget. I think it's called budget actions? Like we can set alarms in budget, also actions can be set.
I'll have to read up on that
I second budget actions
I never used Cloudflare so I have one question if someone can answer me. So, to set Cloudflare to protect my Nextjs application against DDOS it will work like a CDN or a Reverse Proxy? If CDN so basically it will be a CDN above a CDN because SST use Cloudfront ? And if a Reverse Proxy, so basically I'll lose the pros of use Cloudfront to distribute the static content around the world because all the requests will comes from Cloudflare ?
What about just using Cloudflare as your CDN?
Cloudflare?
Doesn't cloudflare have limited support for node?
@@neociber24 what about node? Use Cloudflare proxy, nothing to do with node
what's your question? I talk about cloudflare in this video.
@@WebDevCody I watched half of your video, and just suggested to put a Cloudflare proxy to avoid these situations. Cloudflare has a free plan that gives the DDOS protection. just the free plan has simple Bot mitigation. From docs:
All Cloudflare plans offer unlimited and unmetered mitigation of DDoS attacks. Customers are not charged for attack traffic ever, period. There’s no penalty for spikes due to attack traffic, requiring no chargeback by the customer.
I think he meant cloudflare for managing DNS and you will get free DDOS protection I guess@@neociber24
How come you dont use Vercel, Netlify, or something similar? I used to use a cheap VPN for everything, then moved to Vercel for the convenience of CI/CD, DDoS protection, scalability etc.
You still get charged if attacks get through, which they do. Limiting spending caps to pro plans is also BS
@@nickwoodward819 Yeh they do in some cases, but as we've learned from twitter, tweeting about it gets that sorted fairy quick. But for what they offer for the price, its extremely entising
Can't you use the free plan from Vercel?
Since it doesn't require a payment method (ex. credit card,) you won't get charged.
Or is the free plan too limiting?
@@karlembeastusing the free plan can still get you charged though most likely. Just because you don’t have a linked card doesn’t mean you don’t sign a contract with them. It’s likely illegal to just not pay, you’d need to speak to their support
@@karlembeast Just because you've not provided a card doesn't mean you don't owe the money
And this is some content i always looking for thanks mate
I usually do the poor mans (hard budget limit) DDOS protection for my side-projects. If I go over $10 bucks then just shut down the project. 😀😅. Great video though.
Is there guide to how to do this?
i sshould prolly splurge this fiscal year and push mine from 5 to 10$ lol
Love to see a vid on cloudflare as layer over s3 etc.
Id totally use a vps. It’s cheap. It’s fun to setup; it gives you a ton of flexibility. And it’s just 20 bucks a month including a Plesk license. VPS all the way
Just bought a raspberry pi and create your own homelab to host your services
Could maybe use Varnish if you want the cache benefit
VPS with containers where maybe putting restrictions on the container on how much resources it would scale because I've seen single container go up upto 300% and bring down everything
For the size of your projects, I would go with a VPS to keep it as simple as possible (less moving parts). At least until you start having millions of requests a day and making a boatload of money.
I understand the fun on setting this systems up, but maybe, just maybe, this time you over-engineered it.
Still great insight so I appreciate the video. Thanks.
I 💯 agree at this point and I’m moving alll my side projects into a single vps and hosting behind cloudflare
That sucks. Have you thought about having your app in a docker container and let the service manage the amount of cpu cores depending on load but you can set a max cap? I've never done it before so I'm curious on your input if you've done something similar and if there are drawbacks.
Don't do it, not worth it on AWS. First off, you have Fargate and if you want to autoscale that it's complicated and expensive anyways (1vCPU + 1GB RAM is 20$ I think), you will then also need an IP which is 3.5$ + you will need a load balancer, which is 20$ + 2 IP (7$) for a grand total of ~50$/month. Congrats, you pay 50$ for 1vCPU and 1GB RAM. It is absolutely criminal. For reference: on Hetzner you can get a server with Intel Xeon E5-1650V3, 256GB RAM + 2 500GB Datacenter SSD disks for this price.
Pro tip: Don't ever fall for lambda. Yes, you get 1M requests free, but you are locked in, have a weird programming model + if something like in the video happens and you fuck up caching/don't cache some stuff, AWS will skin you alive.
Pro tip 2: Don't ever host your shit on AWS, pick Linode or something similar and put CloudFlare in front. Hell, pay 5$/month on DigitalOcean and you will get CD for free + have a way better experience because of UI.
Does rate limiting prevents this?
About 2 years ago I reached the limit of my free tier firebase just by forgetting my dependency array in a useEffect in React .... very very quickly! . Anyway... it took me quite some time to figure out what what happening. Maybe someone tried to access your api in this way by accident?
Love how your voice comes through - what mic do you use?
Mxl 990, got it on Amazon for like $100
Why cloudfront didn't rate limit it?
It has rate limit by default, you can't write while (true) fetch. script, after few request's it will start returning 429 error
So was all this bandwidth made from thousand of IPs or how?
I believe like he briefly mentioned in the video that he didn’t use Cloudflare. Likely used Route53. Which out of the box just comes with DNS and not WAF. You’re correct, with CF you get an ootb firewall - not sure how good it is but I know it works at least. He was asking in the video if he should use cloud front, probably is a good plan
But still, he had cloudfront why didnt it block requests
@@aspirine17 did he say that though? Like I say, he likely just used Route53. If he does, yeah the default settings should protect. Though personally I add additional rate limits and block suspicious activity in cloudflare for free under their “10 free rules”
Amazon could just do what everyone has been asking for years and make a killswitch for when the bill goes over a threshold but then again Bezos may need to buy a new space weiner so nevermind
request limiter may help?
DoS and DDoS are two different attacks. I think you got the DoS attack, not DDoS if most of the requests are coming from a few IP addresses. DDoS is not easily preventable using simple rate limiting. To prevent DDoS, consider dynamic rate limiting like exponential backoff strategy, mix of global limits, per IP & per user rate limits, browser checks and ML based captchas, CSRF tokens, no direct IP access (domain-bound to server app), no direct A record to DDoS mitigator like Cloudflare - use AAAA record and set firewall to block IPv4 to your virtual servers and Cloudflare will handle IPv4 traffic and route it to your IPv6 only servers. These will prevent IPv4 based devices/botnet to join the DDoS attack.
I trust what you say. I didn’t have monitoring setup or logging to view ip addresses so at this point I’m in the dark 😂
you can also limit number of concurrent lambda instances, the default is 1000
I get so scared by this stuff that I login to my Vercel account to check it even though its on a hobby plan with no billing setup at all 😅
Ah you just prompted me to check my hobby account and people have been spamming my sites with tons of bots trying to find vulnerable wordpress paths. :( It's a placeholder site that I don't share the URL of! Nothing is safe :(
Is it not possible to set rate limit on fetching the API on your end? Since WAF is also using just IP address to set the restrictions.
I’m getting charged for CDN requests, not from api requests
Doesn't AWS provide some sort of free built in ddos prevention?
I think aws shield advanced does but it’s 3k a month
Cody I think CF has got DDOS protection on free plan too...
Also can you try hetzner. I am currently hosting the server on it used by like 2000-3000 folks at my college.
Some things I miss is dev preview deployments, edge middleware... Speed is descent
On Hetzner you have around 20 TB free, but after you start to pay pro consumed TB, but they also have an anti ddos in place
Yea I am surprised that even entry level vps have 20TB bandwidth. So it's great@@neevot
i've seen a couple of other comments asking this, but why wouldn't you use something else like Cloudflare to proxy all of your traffic before it hits AWS?
Oh, I hadn't gotten to the point in the video before commenting this. You mentioned cloudflare has DDoS protection for $20/mo. That's actually not entirely true, you *can* get DDoS protection for free, though I'm not sure what the licensing agreement is (commercial uses etc etc)
Isn't Cloudflare a good solution for this?
Any idea why/who ddos'd you? Might be interesting to see a tear down of that. Was it from many ips or just 1?
I didn’t setup real monitoring and logging either 😂 no clue what the ip
So in theory a new inexperienced Dev could setup an app on AWS, get ddos's and end up with a bill of around 100K?
Pretty much
cloudflare ddos is best in class
Are things like Vercel free tier really not capable of providing enough functionality for what you need, especially since these are "free side-projects" ?
you can't host commerical applications which make money (which my side projects do sometimes make money) on vercel free plan according to their license.
I think if you contact AWS support and clarify you got an attack, they can reduce the costs
I'm assuming all cloud solutions have a similar setup/cost.
I don't have experience with this but does putting an extra anti ddos vps in front of aws help? No idea on the cost but then you don't have to move your apps.
Curious what path you will take. Great video!
To be fair that’s a great idea. It comes with the drawback of too many requests will just make it slower and slower and timeout requests. It would just be like a proxy that filters through traffic. Great idea, I’d love to hear more about this
@@oSpam It's a terrible idea and it defeats the purpose of CloudFront.
You have CloudFront (edge locations) to serve as proxy close to the user and cache the response. For example - you host your site in N. Virginia and I make a request in Italy. Request will go from my home, to the nearest data center to my location, to US, come back to data center, be cached and come to me. The next request a user close to me will make, will not make the round trip to US, but will just return the same response that got cached.
If you put a proxy in front of cloud front in N. Virginia for example, my request will go from my home, to the proxy in US, to cloudfront edge location, to the server in US, be cached in US cloudfront, be cached on the US proxy, and only then be returned to me. The next request will have to go to US.
If you have a static site, you would ideally cache everything for a long time on cloudfront and for some time in the browser, and on each deploy, invalidate the cloudfront cache if needed. If you get DDoSed, you will just pay a lot, but there is nothing you can do in that situation.
I think you can configure you aws to hard shutdown if billing go above a defined value, but I'm not so sure. This can work for pet projects that you don't care if they go off. This can prevent creating an infinite loop while developing and getting an infinite bill too.
No you can't.. you can setup billing alerts that tell you when you hit a limit, but nothing is going to shut down
@@Dontcaredidntask-q9m yeah after that I went looking, and it seems that I have to code a hard shutdown myself using lambda and billing event. I think I'll try doing that for my pet project. Not really a hard shutdown but just turning off the public access of any api
@@EduarteBDO can you programmatically turn off cloudfront access too? I know you can add deny policy to apigw, but apigw wasnt what got dos here
@@SogMosee I don't know much about cloudfront but with aws sdk you can do pretty much anything that you can do in the console, so in the worst case you can delete the cloudfront configuration.
vercel has Basic DDoS Mitigation on Pro plan
so the up sell is that below their Pro plan they're inadequate?
@@zeusvargasLaughably only if you *pay* for the Pro tier. So they're admittedly leaving you exposed as a customer because you're on the wrong tier. Vercel are a joke.
@@nickwoodward819if you don't pay for the pro tier, they don't have your CC. so there is no way for them to charge you if your hobby tier site gets ddosed.
@@nickwoodward819 pro tier 20$, i think you are the joke 😂 Maybe you want that your bill will be negative one also?
@@maitre999 Not sure that I care what you think if that's the level of English you use to express your thoughts, but just for the hard-of-thinking at the back:
*The cost of the pro tier quite obviously doesn't justify the gutting of basic spending controls in the free plan. No one should expect to be left vulnerable to massive bills because they haven't upgraded*
It could be $3 and there would be no justification for *not* having spending caps when they're obviously available.
doesn't digialocean charge a fixed amount per month? I'm not sure why people insist on running AWS (or any cloud) for small projects. Even a cheap shared hosting will do in most cases.
The thing is that free is still cheaper than cheap. This definitely helps weigh the pros and cons, but I’ll still use google cloud run with a max instance of 1 before I use a VPS.
use free cloudflare dns + ddos protection instead?
I still remember the day webdevcody told he will never use a bare server for any deployment. Now he realises the risk in not using a simple vps in place for aws services.
I have this fear all day when I use aws which make sure i learned how to setup and maintain a $5 vps 😂
XDD
Is it possible to add cloudflare to a sst project?
Yeah, just don’t use route53 and instead point your domain to cloudflare and then point cloudflare to your cloudfront distribution I think
This is one of the “easy” benefits you get with Vercel - if something is clearly ddos, they block it or take the charge.
Yeah, I'm not relying on their judgment as to whether I should pay them... The fact that they haven't got a spend cap is *nuts*
They have some automatic mitigation but they dont pay your bill, unless you are a popular content creator
@@nickwoodward819 I think they added it the last week
This just isn't true. Plenty of examples online of nobodies getting their DDoS bills refunded.@@AndersGustafsson87
is that written in writing in their SLA? "we will pay for any DDOS bandwidth your app receives"? I highly doubt it
I would love to see as an example how to setup and deploy nextjs project to vps.
Does SST work with nextjs14?
yes
i think you should move your frontend to the edge and backend to your own vps, and it would be much cheaper if your backend does not eat too much resources, like when you write them in go or rust
the front end is what caused this high charge; it was all the requests to the VPN cache
@@WebDevCodyCDn? Yeah you’re right. They probably didn’t watch the video 😅
I think too many people are suggesting bad rush moves. They need to watch your other video on why to never use a VPS for production 😂
Love these practical real world videos
Hope you don’t have to make too many of them 😅
🤣
VPS with cloudflare on front. Good enough for semi small projects
What's with these D-DoS attacks? what do they even gain from this? are they even competitors?
have a script that shuts down the machine if get a billing alert etc. But yeah, I wont touch aws for personal stuff. But yes of course, your service goes down during that time. But that's fine for personal stuff. And your service might be down from the ddos anyway!
This services are not meant for hobby projects. I allocate a fixed amount of money per month on my hobbies: books, manga, and my two websites. I make sure I never spend more than that
step one: put a hard limit/budget cap on your third party service
I don’t know if AWS doesn’t support this but other third party 4:18 applications do
They do. The budgeting service he showed has the option to cut off all services at a certain price. He may not want to do that for what I can only guess to prevent disruption for users.
But why ppl do ddos? I'm afraid now for lunching my software 😢
Don't be, it's not nearly as common as you might think. You should be worried about not finding users.
so you that means that any attacker can actually do an ddos on side project website that doesnt pay for waf are vulernable thats not good for amazon
Even if you pay for waf I think they still charge you per request? Idk you’d think off of this would be very black and white to understand, but it’s not
If you contact AWS support and prove that you got attacked, its the first time, and you've now setup WAF, they'll refund you the cost of the attack
My question is how can you manage to spend $11 per month with 5+ apps, cloudfront, S3, possibly ELB, RDS, Redis cache, etc?
2 of my apps use convex free tier, 1 of my applications used planetscale free tier for a year, another 2 application use supabase db free tier
I’m surprised you didn’t use CF. It’s like budget deployment 101
I`m looking into AWS for their GPU instances(AI stuff) - otherwise I woudnt touch that behemoth with a 10 foot pole - too confusing,too big,too enterprisey and huge risk of this stuff happening if you dont have a PHD in AWS.
Having same concerns for side projects eventually i think i will go through same path to vps
Im new to your channel, are u using a bot to like the comments? (just out of curiousity)
Can we get a video on deploy next.js using sst , preety plz
search the channel, you'll find this video
Cloud is such a waste of money. Still don't understand why everyone drinks that Koolaid..
Welcome to reality) great video!
Please make a video of
- your deployment process to AWS including docker config and shell scripts for full-stack and backend projects.
- setup with cloudflare
- setup with services like digital ocean
Thank you
🙏🙏
Rate limits are easily circumvented.
I keep seeing AWS horror stories for normal people and small businesses. Seems absolutely ridiculous to not have a way to set a cap. I have an app that is going to need infrastructure setup soon and I am wondering if there are any other options.
Can you not just code a rate limit , i am sure the code for rate limit and little bit ddos prventation code will be easy no?
not when the static assets are hosted on a CDN; then you have no code that executes.
Cloudflare is your answer my dude
I think so
3:39 AWS pricing is absurd once you start to do something even remotely interesting. I can't wait until SST has first class Cloudflare support.
sorry for your money loss and thank you for sharing the knowledge
Sad that happened 🙁 have you tried to contact AWS and explain them the situation? Maybe you get a refund for that? 🤔
We’ll see
more vids like this, thanks
AWS are probably the one Sponsoring the attack. There is a conflict of interest if you have to pay cloud service provider for ddos attach.
You got attack now you have to pay for a service
great video
Cloudflare free tier is so much better than payed AWS Cloudfront
Why not use Vercel or Netlify? Then you are not longer responsible for DDOS attacks and you don't have to pay additionally for this protection.
This is not true. Netlify or Vercel will still charge you for execution time and bandwidth.
@@doreto95 but vercel has basic DDoS Mitigation on Pro plan
@@doreto95 Didn't they recently implement soft and hard limits (at least I'm sure Vercel did) because of a scandal where somebody got charged $100k on their hobby project?
Incorrect, you should change this comment - Vercel and Netlify *DO* charge you for attacks, and if their protection was adequate enough they wouldn't provide spending caps to Pro account holders like it was a benefit
Hey guys, sure Netlify or Vercel charge for execution time etc. BUT they handle DDOS protection for you without you having to pay extra.
AND they are willing to not charge users if their protection failed.
jesus christ, serverless is convenient for damn it's expensive
I hope this fixes your problem