Oh no this is going viral which means the comments are going to utter shit. FAQ HERE - IF YOU ASK SOMETHING I ANSWER HERE YOU'RE PERMABANNED "What is the edge runtime" - it's an ALTERNATIVE TO NODE that is VERY MINIMAL and VERY FAST. That is why it has no cold starts. That's also why it has no native ability. (mentioned at 8:54) "Regional edge is contradictory; You made that up!" - As I said AT THE START. Edge references TWO THINGS. Runtime and location. If you limit your edge functions to run in a specific region, they're called "regional edge functions". Learn more here: vercel.com/blog/regional-execution-for-ultra-low-latency-rendering-at-the-edge Seriously if y'all don't cut the shit I"m turning off comments.
Something to add to the FAQ maybe, as I can't seem to find the original comment questioning it: - Why does the cold start take several seconds? - Why would we care about it? The number of needed cold starts approaches zero in the long run
YES! So spot on. It's even better when you listen to a dev that has "discovered" security principles and having them walk the team through all the things they should have been doing from the beginning.
@@Kimitri I agree, the level of fucking idiocy devs go to while there are simple point-and-click solutions is astounding. I feel like there are two different worlds - the actual dev world and YT tech space where everything is THE NEW THING. Like SHUT THE FUCK UP oh my god. Yes, it's fun to learn the new things, but wait for them to mature before switching. Fucking tech youtube is a bunch of hipsters ordering essential oils for a flu.
I feel that web developers went from optimizing LAMP stacks and dove directly into serverless, to quickly realize that it's expensive if you're not going to optimize, and then went back to optimizing. Next video will be about "how I ditched serverless to configure my own Varnish/Redis cache"
So true. You need to ask yourself how this man make money and you will see that this is a propaganda campaign disguised as tech content, and dont you dare debunk or you will get ban.
@@o_glethorpe I don't necessarily think that he's doing propaganda, but rather milking content for views. No one on earth changes their mind on tech stacks like this guy
I've heard memcached has better performance than redis as it it written in a lower level language and supports multithreading, but admittedly haven't tested it yet.
idk guys I feel at this point when I was building with Express and EJS and deploying on a VPS with a simple NGINX reverse proxy we reached the peak and then we started over-engineer..
Agree 100%. Maybe this is useful for use cases that I’m not aware of but to me this seems like major extra complexity. Connecting to your database using REST?! Your existing code won’t always work because the runtime doesn’t support certain functions? And if they do it’s because the vendor is maintaining some custom converter? What problem are we solving here? Is it purely for cost reduction while achieving low global latency?
Theo does the experimentation and research and as a result I get to spend a lot less time doing it to find really great tools. I hope he keeps changing his stack every week!
@@curiouslycory I concur. Your comment epitomizes the phrase *_"'Tis wise to learn from one's mistakes, but wiser still to learn from the mistakes of others"_*
Chasing the framework of the week, the tech of the month whatever it is, never satisfied. This is not targeting real developers with real companies, just the usual twitter's "TODO app devolver" thats basically just clickbaity sponsored content "omg it only takes .00023seconds to create this full fledge app with this tech" the full fledge app being a todo app auth 😂
What you are referring to as Edge Runtime is in fact specifically the Cloudflare Workers runtime (Vercel is powered by Cloudflare Workers), that runs on V8 Isolates. There is nothing that prevents you from having another edge runtime with an approach that does not use Isolates. AWS Lambda@Edge is also on an "edge runtime", but they use their own solution to achieve this (on which you can only use Node and Python as far as i know), it's just much inferior to Cloudflare's.
that's good to know! Sorry if this is a stupid question, but doesn't that mean it's possible to achieve the same without going through Vercel at all? I've never used either, but it looks like if one were to use CloudFlare workers directly, you'd have more flexibility (don't need to be constrained to using node, for instance), which I'm sure has its trade-offs since I'm guessing Vercel provides some conveniences in case you do want to use their service instead.
@@DisFunctor I've never used Vercel but yeah you can achieve the same result using CF Workers directly. I am sure there's some sort of service/features that Vercel adds to the equation though
I really like how you give an explanation with a visual diagram to support it. It's really helping me understand some of the more infrastructure-side tech.
Seems like the Vercel Edge Runtime is only for JS app, so not for me :) An inter-continental round trip takes 150ms, so in most situations, we don't even need multi regional deployments. I'm fine with doing a little bit of server management and never have a "cold start" and having super fast DB queries... On AWS if we pick things like Spot instances we can save a lot on ECS costs, when done right you can save like 90% on the on demand price. Serverless is fine for doing Async use cases, but I don't think it's a good choice for Web Servers where latency is critical.
Edge runtimes can only run JS or WASM generally, and you'll only see response times this good in JS. Crazy enough, having a fully virtualized engine for your language has benefits sometimes
I use lambda running node/express for SSR and find that less than 1% of user requests end up being cold starts now that the site has a decent level of traffic. With a cold start, the TTFB on a good connection rises from around 200ms to around 400-450ms, which isn't bad enough to make me want to switch. Plus the JS code is 20mb+ uncompressed on the server side so I would have to do something with that.
curious what your setup looks like? would you mind sharing a little more detail? I'm assuming you're using some template engine to serve HTML from lambdas and routing reqs through AWS APIGateway.
@@nsuid7499 Yeah, a page request first goes to CloudFront (CDN), if it hasn’t been cached it goes to API Gateway (HTTP API) which then calls the Lambda function to generate the HTML for the page. The site uses the Angular framework ("boo!"), so it’s their rendering engine, but I imagine it would work just as well with any other template engine. The lambda functions need to get an item from DynamoDB to produce the page (house price information site). The average execution time for a warm Lambda function is 50-75s of which I think the database query is 10-15ms. For users 200ms seems to be the time it takes to get anything back, so I guess the rest is all network stuff like directs, DNS, connection etc. Cold start on average seems to add around 200-250ms, and they are a very small percentage of total requests.
It's surprising to see how "Frontend development" went from "Develop HTML, JS, CSS" --> "Develop SPA but backend handles database and hosting" --> "Decide if a feature/page is served on serverless or edge and take care of it". Does this mean small app developers no longer need a dedicated backend engineers?
Hey guys, chill, I was talking about smaller apps, ofc backend devs are important, nobody is debating that. Maybe I used not enough words to say what was on my mind
Edge location is basically only useful for static content. Making a brochure website for a big client? Edge location is a solid choice. Want anything with live data? Rethink it.
The funny thing is, I thought when serverless was released it had the same sort of pricing, and people where massively touting it because you could get millions of runs for pennies - am I misremembering that? I’d imagine, once there was uptake and people began using serverless that it became worth it for providers to begin charging for potential availability, tracking the minutes and charging that way. I would expect a similar shift for edge.
Don't let the terminology mislead you. It's really just "Web standard APIs inside v8 isolates" It's on the "edge" because providers like cloudflare can achieve incredibly high multi-tenancy on far fewer servers compared to VMs A 1-5MB bundle is much more reasonable to replicate to 100+ regions compared to 300MB to 1GB+ docker images It's objectively cheaper than "serverless" VMs. And much better suited to applications like frontend servers which don't really need much more than a subset of Web APIs
@PursuitofKnowledge how come? Blockchain is a similar concept, isn't it? Everybody in the network has the ownership of something, and at the same time, everybody has the ability to share it with multiple users at the time. It's like a giant system built on top of some sort of community
Everyone is pushing some kind of agenda, you should learn as much as possible and create your own opinions, dont just go with the flow and never assume to yourself any opinion just because some dude with fancy hair said so.
Hey Theo. I don't know if you actually measured the difference in request time to the DB but from what I know there is signifcant overhead to an HTTP request unless you are using Connection: keep-alive. Does the edge runtime support keep-alive connections? (P.S. I checked the FAQ first)
My brain fried when trying to figure out the subtle differences between edge/regional I'm just happy to be able to throw my SvelteKit frontend up to Cloudflare pages with specifying which routes can be prerendered. Then for anything more complicated I connect Cloudflare Pages to my Go Fiber API and DB backend through Cloudflare Tunnel. Edge functions are still spooky to me, even though I do use fetch().
hmm.. like a lot of times in fact. I suspect most people watching Theo are not real senior engineers and those who are senior just use Theo's content as one uses newspapers, just to hear about topics in the market and selectively do their own deep research on technologies.
@@lex-fridman To be honest that's what anyone should be doing with any info on the internet in general. Sure there are more or less reliable sources, but in the end, you make the call.
@@lex-fridmansystem architect here. Some time ago I've consulted my friends about tech stack for their startup idea. The issue is that in my country salaries of developers are pretty high, so I've decided to put in serverless everything that was possible. My main job is in big corpo so I'm not familiar with such stuff and Teo's videos helped me a lot. For my standards the result app was sht af, performance sucked, but its more pragmatic than hire expensive team of java+js devs for startup with unknown future. If everything will go right and guys get some investment we ofc hire a normal team but for now it's not an option. So, all this stuff is not suitable for big corpos where "Serious Enginners" work but useful for testing business ideas.
Edge runtime feels like an oportunity for CQRS revival to me. Serverless was already a good fit for it but with those limitations specifically, it almost reads like the intent of `exec` in pure FP variations of CQRS. Meaning, you aren't meant to do IO in it, you just take a command and a state (usually local to the system), and emit events out of it instead of running long-living requests (which can be triggered elsewhere reacting to events similar to redux-sagas), and you just need to keep a minimal internal state, only what is necessary for your `exec` to make decisions on what events to emit. Plus events are meant to be JSON.stringify-able
This is serverless stuff is getting way too overengineered. At this point just throw the whole app into a ec2 box with ubuntu and call it a day. If you ever reach page-load performance issues, start by trying to optimize in other places first (maybe start by removing huge JS frameworks). In the worst case, if you ever need to get a super-optimized backend, just hire prime to rewrite the entire thing in rust. I mean at this point we’re rearchitecting the way the entire internet works from scratch: building new db drivers, new cache systems, new ways to split up databases across many servers, new rate-limiters, etc. Just kidding Theo. Love your vids.
I hear operations people quietly sigh on this video. Leaving some things aside: The main problem is distribution of data services - and also the privacy aspects of that.
it's only been a month since your T3 tutorial and i feel like it's already getting outdated. Will you make an update video where you build it with all the new things you have been talking about since then?
Under the hood Vercel deploys to AWS Lambda. To deliver world class dynamic sites in production, Vercel has turned AWS Lambda into an edge-first compute layer
Wouldn't read-only DB regions be a solution for the global edge problem? (Planetscale can manage that) Or I can't guarantee my global edge function calls my closest read-only database?
You shoul'd take a look at Fermyon. It's basically everything you want when you talk about edge runtime. It's basically webassembly on the server, it allows you to have no cold start and spin up and down you infra as needed.
When talking about edge locations, you assumed that there always has to be a cold time or some slow database connection build-up, there are solutions where this is not the case (warm standbys). Additionally you can also send HTTP requests to database API from edge locations, this is not only limited to edge runtimes.
Thank you for showing that you could opt into edge on pages as well as API routes, I was trying to find out if you could for like an hour yesterday and couldn't find that anywhere. Everything I see on Vercel is just showing API routes
This concept of regional vs. global edge runtimes is interesting. It makes sense that you want your code running close to the database if you are making many requests. I feel like the major difference between regional and serverless is the cold starts. If serverless somehow solves this issue, that would seem like the best of all worlds, right? No cold start, close to DB, everything is compatible. I guess we gotta wait and see.
Damn the edge looks really impressive. I might take a deep dive into v8 isolates to see how they work and what other tech is using them. Would be really cool to see a follow up video on moving ping to the edge and seeing a before and after in performance.
"I hate serverless" - yes, someone I can agree with "I'm just going to move everything to the edge" - Oh great, a new tech stack with a gimmick marketing name.
Wow this was crazy informative. I didn’t think edge was much of a big deal and I didn’t understand how much it differed from serverless so I never bothered with it. But now that you broke it down, I’m definitely going to try it out! The amount of value I get out of your channel sometimes is absurd. Thanks Theo. Keep up the good work
Yes, but you lose our on the hyper fast cold starts promised by edge runtime using web APIs you use the included runtime whereas using another language could mean you're shipping libs that end up resulting in larger payload with slower start time The tradeoff does make sense for heavier use cases like image processing
@@rzr1191 Do you have data to back that up? rust WASM has incredibly low cold start times/binary sizes, on lambda rust can be almost an order of magnitude faster to start and 3x less size than an equivalent node function. Why would edge suddenly mean that WASM has worse performance? Also, my main benefit is the fact that I don't have to use typescript (absolute garbage language)
Edge is also used to talk about edge devices. For instance, whether or not a computer vision model is handled online somewhere, or on the physical device, or the "edge device" or "edge processing"
And when I think that for more than half of my and my clients web apps/sites I just use a VPS and it just works fine, with loading times that are good. :( When did deploying stuff become so overcomplicated?
Edge as a location IS IMPORTANT! Just like a CDN; the location edge out-does Regional edge when the data is cacheable (eg 10 latest blog posts; or newest rentals in your city). Edge locations is also important when you can use a distributed DB like the SQLite offering many edge providers now offer. This video seems already outdated. (Old school thinking)
@@rewrose2838 That it? If cold starts and prisma are the bottlenecks it sounds to me like optimizing prisma and hosting on a regular server are the most obvious solutions. People really just jumped on the serverless bandwagon for the *hypothetical* benefit of scalability only to suffer the very real downside of cold starts because they don't actually have users
What presentation tool do you use to explain all about serverless? quick drawing is what kept my attention when you were pointing out the subtle differences between serverless vs edge. Kudos.
I have one server, in Sydney, Australia. It's good enough for now, and I've built my whole site's system to use it. Who knows what will happen when I officially release it. That price though... Edit: What the hell is vercel? am I so much of a dinosaur I can't understand it?
i believe it started out mostly as a frontend hosting solution, then slowly started to support more backend solutions. now they offer simple ways to host fullstack jamstack apps like next, remix, sveltekit etc.
I used to use Google Cloud Functions (Firebase) a few years ago and the cold starts where just down right unacceptable. I built an entire infrastructure on Cloudflare Workers and have never been more happier serving code and sharing libraries
This was incredibly informative! Where can we read more about practical coding for the edge runtime (including handling non-edge-ready dependencies and dealing with data)?
Thoroughly absorbing, isn't it? For a continued absorption, consider a book I highly esteem. "AWS Unleashed: Mastering Amazon Web Services for Software Engineers" by Harrison Quill
I'd like to emphasize, this is probably only a solution for small complexity apps. If you have a massive system with multiple user types, serving multiple different UIs, might not be the best approach
Do you have a video of you building an app on the edge with planetscale? I find it hard to use planetscale without prisma and would love to see how you do it
I feel like one area that has quite some room for improvement is moving all data related things inside the database. Being in the same data center is great but nothing beats being on the same chip. For requests that don’t do rendering you can avoid spinning up any runtime altogether.
This is the elephant in the room for all of these services he's using. They all have pretty generous free tiers but balloon in cost for any serious applications. Vercel's pricing is rough enough for my company to be exploring building out infrastructure on AWS just to get Next with all the features it offers without Vercel.
@@JoRyGu Tbh vercel does makes it easier to use cutting edge things like he is talking about a lot. But this is something which is holding me back too. The balloon in the cost is why I opted for self hosted supabase instead of Planetscale. I haven't used amplify yet, but do you know how much worse it is than vercel? Or does it support all the latest features of next 13?
@@JoRyGu you should watch my video about AWS cost if you sincerely believe this. The bandwidth costs are in line with any other CDN of similar quality. Be smart about where you put your big assets and you’ll be fine. Your Vercel services should return HTML and JSON, if you’re moving terabytes of either you’re probably making a mistake
Tbh, my biggest gripe with vercel rn is that their free tier does not allow commercial usage. Almost any other provider does (e.g. netlify, heroku formerly, cloudflare pages, etc.). It's such a shame cause vercel is still probably the most convenient but their free tier is just a deal breaker. Idk why they did that but to me it's simply a bad business decision.
I had observed a very interesting company, it called Azion Technologies! They are building a lot of thing with Edge Computing, I have used a lot the Azion Edge Runtime to accelerate our apps
Stupid question: WHY are edge functions free of cold start delays? I though edge functions were just serverless functions running on the edge? I feel like a bunch of people came up with a new set of buzzwords without rigorously defining them.
Guys eat me, but I am running my projects on Raspberry PI Cluster -> 4x Rasbperry PI 4B 8GB ram hidden behind the firewall, 1GB ethernet connection, all of them on SSD (not SD card) and... this is the 1,5 year and they do just fine :D
Yeah but pal heres the thing, everytime we ain't gonna use planetscale or database which provides apis to interact with their db , so that's a big issue. In that cases, we have to do everytime connecting to db, fire request and close the connection, for every request.
Postgres with PGBouncer and you have direct connection, pooling manager, don't need to think about open/closing sessions (since it's claims to be milliseconds but Im too often face with the situation when it adds up 100ms on top of the request). Keep any RDBMS on single server until it's possible to handle all your requests, divide you relational and non-relational data between nosql databases and rdbms, move out analytical requests out of your RDBMS to the warehouse It's all about architecture and your needs, there are no single solution for all
Would it be a big win to have the auth built into the database, e.g. request data from a table (db checks permissions and returns data in a single request/response if they have permission and says no otherwise), rather than sending multiple requests to see if they have that permission before requesting the data.
Vercel is using aws or i am wrong? Also for the cold start time, its because your code is to big. Sound crazy, but we are in an time where you need to optimize the weight of the backend as we do with the front-end. For the DB connection, use DynamoDb as mush as possible and dont use external db of aws. Serverless DB from aws is bad also. So lightweight code, DynamoDb with a aws non-serverless db and it will be fast without cold start lag
16:30 This is my only issue with this video - if the user has slow internet, then global edge will be much, much faster, no? Assuming that the request times are equal assumes that the user has the same internet speed as the server. Anyways, you are absolutely right when you consider lots of serial requests.
Yeah it’s awesome! Btw is it possible already to use just “edge” keyword instead of “experimental-edge” for deployments or it’s different runtimes where the experimental one is for new features like regional edge?
Does anyone know of an edge compatible cuid generator? The paralleldrive one seems to use node:crypto which doesn't work in edge, and I don't believe Drizzle has a helper for it.
The paralleldrive one _should_ work but doesn't due to a weird bug in bundling for app router. Works in /pages for some reason 😅 I've raised it with vercel and hope they have it handled soon. Suffering with uuid v4 for now 😅
Can somebody link me a video where Theo is talking about Firebase or why he would never consider it If such exists? Just curious of Firebase functions vs vercel edge comparison
The hilarious part to ‘serverless’ is that there still is a server or servers behind it. ‘Serverless’ doesn’t mean no server.. a computer can be a server. So going serverless literally would be like writing code on paper and never having it get complied or presented to you in physical form..
What is that drafting/presentation software on 3:39? My projects are usually quite simple so I have been using Cloudflare workers and KV Storage to host static website exports and to build basic APIs. Now they have made their D1 SQL databases for workers open for testing.
Theo can you speak about how u protect your gateways that is in front of your apis? Do u use jwts? So u have a specific GW that has sec embedded? So mostly about security arround ClientGWAPIs Thank you
Oh no this is going viral which means the comments are going to utter shit. FAQ HERE - IF YOU ASK SOMETHING I ANSWER HERE YOU'RE PERMABANNED
"What is the edge runtime" - it's an ALTERNATIVE TO NODE that is VERY MINIMAL and VERY FAST. That is why it has no cold starts. That's also why it has no native ability. (mentioned at 8:54)
"Regional edge is contradictory; You made that up!" - As I said AT THE START. Edge references TWO THINGS. Runtime and location. If you limit your edge functions to run in a specific region, they're called "regional edge functions". Learn more here: vercel.com/blog/regional-execution-for-ultra-low-latency-rendering-at-the-edge
Seriously if y'all don't cut the shit I"m turning off comments.
lol
Something to add to the FAQ maybe, as I can't seem to find the original comment questioning it:
- Why does the cold start take several seconds?
- Why would we care about it? The number of needed cold starts approaches zero in the long run
I just got here stop yelling at me
@@xN811x typescript compiling all the node_modules
The internet is full of idiots. You should know this by now.
First serverless, then edge, next thing you know theo's in your kitchen installing a server for less latency.
they might as well start torrenting databases and some parts of the backend to reduce latency
Eventually we are back in the Stone Age of locally running services
@First Last is the AGI gay by any chance?
@@ea_naseer at this point blockchain makes more sense
@@ea_naseer Might not be that crazy, running microservers on edge and using a token algorithm to handle the data persistance.
It's funny watching devs reinvent everything they shunned 😂
This needs more upvotes
YES! So spot on. It's even better when you listen to a dev that has "discovered" security principles and having them walk the team through all the things they should have been doing from the beginning.
Just wait until they fully move client side rendering back to the server
@@meorung05 You mean server prerendered HTML. The heresy! :D
Was thinking the exact same thing.
"Put everything on a VPS" and stop thinking
Best decision i ever made
see: my video on getting DDOS'd last week lol
@@t3dotggno one said you cant put it behind cloud-flare anyways
@@t3dotgg so put everything on a vps with cloudflare protection
@@Kimitri I agree, the level of fucking idiocy devs go to while there are simple point-and-click solutions is astounding. I feel like there are two different worlds - the actual dev world and YT tech space where everything is THE NEW THING. Like SHUT THE FUCK UP oh my god. Yes, it's fun to learn the new things, but wait for them to mature before switching. Fucking tech youtube is a bunch of hipsters ordering essential oils for a flu.
I feel that web developers went from optimizing LAMP stacks and dove directly into serverless, to quickly realize that it's expensive if you're not going to optimize, and then went back to optimizing. Next video will be about "how I ditched serverless to configure my own Varnish/Redis cache"
So true. You need to ask yourself how this man make money and you will see that this is a propaganda campaign disguised as tech content, and dont you dare debunk or you will get ban.
@@o_glethorpe I don't necessarily think that he's doing propaganda, but rather milking content for views. No one on earth changes their mind on tech stacks like this guy
I've heard memcached has better performance than redis as it it written in a lower level language and supports multithreading, but admittedly haven't tested it yet.
well you cant optimise your php if aws is the one running it on serverless.
@@System0Error0Message I hate php. I hope it dies and is replaced with c++ backends and wasm clients.
idk guys I feel at this point when I was building with Express and EJS and deploying on a VPS with a simple NGINX reverse proxy we reached the peak and then we started over-engineer..
@@vinos1629 of course there is a reason, adding to your resume…
Agree 100%. Maybe this is useful for use cases that I’m not aware of but to me this seems like major extra complexity. Connecting to your database using REST?! Your existing code won’t always work because the runtime doesn’t support certain functions? And if they do it’s because the vendor is maintaining some custom converter? What problem are we solving here? Is it purely for cost reduction while achieving low global latency?
Build for now and not the future, the majority of projects will never need to use edge
I don't think you guys understand just how hard it is to do VPS management correctly at scale.
@@vinos1629 and I don’t have to hire engineers like y’all to manage it full time while maintaining cheaper, faster, more scalable solutions 🤷♂️
You are the meme JavaScript developer that has a different stack every week, but pretends they don't.
😂😂😂
Are you a Developer who doesn't like to improve your stack ? I mean updates the stack and change are two very separate things buddy. 🎉🎉❤ Theo
Theo does the experimentation and research and as a result I get to spend a lot less time doing it to find really great tools.
I hope he keeps changing his stack every week!
@@curiouslycory I concur. Your comment epitomizes the phrase *_"'Tis wise to learn from one's mistakes, but wiser still to learn from the mistakes of others"_*
Chasing the framework of the week, the tech of the month whatever it is, never satisfied. This is not targeting real developers with real companies, just the usual twitter's "TODO app devolver" thats basically just clickbaity sponsored content "omg it only takes .00023seconds to create this full fledge app with this tech" the full fledge app being a todo app auth 😂
this is a marketing video
Soon we will rediscover servers where cold starts, hand shakes, and ORM spin ups aren’t a thing.
don't tell Theo he will rewrite everything again
What you are referring to as Edge Runtime is in fact specifically the Cloudflare Workers runtime (Vercel is powered by Cloudflare Workers), that runs on V8 Isolates. There is nothing that prevents you from having another edge runtime with an approach that does not use Isolates. AWS Lambda@Edge is also on an "edge runtime", but they use their own solution to achieve this (on which you can only use Node and Python as far as i know), it's just much inferior to Cloudflare's.
When I was watching this I kept thinking "this sounds exactly like CF workers", good to know it basically is
that's good to know! Sorry if this is a stupid question, but doesn't that mean it's possible to achieve the same without going through Vercel at all?
I've never used either, but it looks like if one were to use CloudFlare workers directly, you'd have more flexibility (don't need to be constrained to using node, for instance), which I'm sure has its trade-offs since I'm guessing Vercel provides some conveniences in case you do want to use their service instead.
@@DisFunctor I've never used Vercel but yeah you can achieve the same result using CF Workers directly. I am sure there's some sort of service/features that Vercel adds to the equation though
I really like how you give an explanation with a visual diagram to support it. It's really helping me understand some of the more infrastructure-side tech.
Seems like the Vercel Edge Runtime is only for JS app, so not for me :) An inter-continental round trip takes 150ms, so in most situations, we don't even need multi regional deployments. I'm fine with doing a little bit of server management and never have a "cold start" and having super fast DB queries... On AWS if we pick things like Spot instances we can save a lot on ECS costs, when done right you can save like 90% on the on demand price. Serverless is fine for doing Async use cases, but I don't think it's a good choice for Web Servers where latency is critical.
i agree but sometimes It can be a really good choice if you dont mind cold starts. very low prices and you get auto scaling out of the box
Edge runtimes can only run JS or WASM generally, and you'll only see response times this good in JS. Crazy enough, having a fully virtualized engine for your language has benefits sometimes
Yep. That fear of cold starts is grossly overblown.
And my shared server that responds in less than 50ms with PHP!
I use lambda running node/express for SSR and find that less than 1% of user requests end up being cold starts now that the site has a decent level of traffic. With a cold start, the TTFB on a good connection rises from around 200ms to around 400-450ms, which isn't bad enough to make me want to switch. Plus the JS code is 20mb+ uncompressed on the server side so I would have to do something with that.
curious what your setup looks like? would you mind sharing a little more detail? I'm assuming you're using some template engine to serve HTML from lambdas and routing reqs through AWS APIGateway.
@@nsuid7499 Yeah, a page request first goes to CloudFront (CDN), if it hasn’t been cached it goes to API Gateway (HTTP API) which then calls the Lambda function to generate the HTML for the page.
The site uses the Angular framework ("boo!"), so it’s their rendering engine, but I imagine it would work just as well with any other template engine.
The lambda functions need to get an item from DynamoDB to produce the page (house price information site).
The average execution time for a warm Lambda function is 50-75s of which I think the database query is 10-15ms. For users 200ms seems to be the time it takes to get anything back, so I guess the rest is all network stuff like directs, DNS, connection etc.
Cold start on average seems to add around 200-250ms, and they are a very small percentage of total requests.
Dev Test
I like your visualization around 5:00 on the impact of moving the service further away from the db.
It's surprising to see how "Frontend development" went from "Develop HTML, JS, CSS" --> "Develop SPA but backend handles database and hosting" --> "Decide if a feature/page is served on serverless or edge and take care of it". Does this mean small app developers no longer need a dedicated backend engineers?
yeah backend devs are often useless lol
(for small apps)
@@Uk1fbcka terrible opinion. Who do you think wrote the code for edge to work? Frontend Devs? Lmoa
It's fullstack all along. I've been fullstack since I entered the industry for all I know.
@@Uk1fbcka words of a front end dev who can't center a div and happen to be stuck in vim.
Hey guys, chill, I was talking about smaller apps, ofc backend devs are important, nobody is debating that. Maybe I used not enough words to say what was on my mind
Edge location is basically only useful for static content. Making a brochure website for a big client? Edge location is a solid choice. Want anything with live data? Rethink it.
The funny thing is, I thought when serverless was released it had the same sort of pricing, and people where massively touting it because you could get millions of runs for pennies - am I misremembering that? I’d imagine, once there was uptake and people began using serverless that it became worth it for providers to begin charging for potential availability, tracking the minutes and charging that way. I would expect a similar shift for edge.
Don't let the terminology mislead you. It's really just "Web standard APIs inside v8 isolates"
It's on the "edge" because providers like cloudflare can achieve incredibly high multi-tenancy on far fewer servers compared to VMs
A 1-5MB bundle is much more reasonable to replicate to 100+ regions compared to 300MB to 1GB+ docker images
It's objectively cheaper than "serverless" VMs. And much better suited to applications like frontend servers which don't really need much more than a subset of Web APIs
I want the "Edge" to just become everyone's PCs/phones and everything just running like torrents and not off any owned server whatsoever.
sounds similar to a blockchain distribution
@@Uk1fbcka lol no it doesnt
@PursuitofKnowledge how come? Blockchain is a similar concept, isn't it? Everybody in the network has the ownership of something, and at the same time, everybody has the ability to share it with multiple users at the time. It's like a giant system built on top of some sort of community
@@Uk1fbcka I would suggest looking up what torrents are.
@@pursuitofknowledge5566 yeah sounds like a good idea after all lol
I don't know what makes everything change so much. You learn one thing and next month it's not the right way.
Everyone is pushing some kind of agenda, you should learn as much as possible and create your own opinions, dont just go with the flow and never assume to yourself any opinion just because some dude with fancy hair said so.
he's just selling his service...serverless is fine for 99% of the apps
Hey Theo. I don't know if you actually measured the difference in request time to the DB but from what I know there is signifcant overhead to an HTTP request unless you are using Connection: keep-alive.
Does the edge runtime support keep-alive connections?
(P.S. I checked the FAQ first)
My brain fried when trying to figure out the subtle differences between edge/regional I'm just happy to be able to throw my SvelteKit frontend up to Cloudflare pages with specifying which routes can be prerendered. Then for anything more complicated I connect Cloudflare Pages to my Go Fiber API and DB backend through Cloudflare Tunnel. Edge functions are still spooky to me, even though I do use fetch().
Sometimes Theo isn't a reliable source.
hmm.. like a lot of times in fact. I suspect most people watching Theo are not real senior engineers and those who are senior just use Theo's content as one uses newspapers, just to hear about topics in the market and selectively do their own deep research on technologies.
@@lex-fridman To be honest that's what anyone should be doing with any info on the internet in general. Sure there are more or less reliable sources, but in the end, you make the call.
@@lex-fridmansystem architect here. Some time ago I've consulted my friends about tech stack for their startup idea. The issue is that in my country salaries of developers are pretty high, so I've decided to put in serverless everything that was possible. My main job is in big corpo so I'm not familiar with such stuff and Teo's videos helped me a lot.
For my standards the result app was sht af, performance sucked, but its more pragmatic than hire expensive team of java+js devs for startup with unknown future. If everything will go right and guys get some investment we ofc hire a normal team but for now it's not an option.
So, all this stuff is not suitable for big corpos where "Serious Enginners" work but useful for testing business ideas.
I would like to correct you. It's not "sometime" , you can never trust Theo, this is just an vercel sponsored channel
@@lex-fridman and you're assuming juniors are not capable of doing that?
Edge runtime feels like an oportunity for CQRS revival to me. Serverless was already a good fit for it but with those limitations specifically, it almost reads like the intent of `exec` in pure FP variations of CQRS.
Meaning, you aren't meant to do IO in it, you just take a command and a state (usually local to the system), and emit events out of it instead of running long-living requests (which can be triggered elsewhere reacting to events similar to redux-sagas), and you just need to keep a minimal internal state, only what is necessary for your `exec` to make decisions on what events to emit. Plus events are meant to be JSON.stringify-able
I hope we can keep edging together for a long while
I like the strong opinions. Watching it now
World confirmed to be a flat circle 💯
This is serverless stuff is getting way too overengineered.
At this point just throw the whole app into a ec2 box with ubuntu and call it a day.
If you ever reach page-load performance issues, start by trying to optimize in other places first (maybe start by removing huge JS frameworks).
In the worst case, if you ever need to get a super-optimized backend, just hire prime to rewrite the entire thing in rust.
I mean at this point we’re rearchitecting the way the entire internet works from scratch: building new db drivers, new cache systems, new ways to split up databases across many servers, new rate-limiters, etc.
Just kidding Theo. Love your vids.
100%
Theo the type of guy who is so convincing in every single video that he makes me doubt my stack forever 😂
I hear operations people quietly sigh on this video. Leaving some things aside: The main problem is distribution of data services - and also the privacy aspects of that.
it's only been a month since your T3 tutorial and i feel like it's already getting outdated. Will you make an update video where you build it with all the new things you have been talking about since then?
Of course. Unlike my tutorial, none of this tech is ready for general use and recommendation. Hopefully soon…ish…
@@t3dotggthanks! and yeh, whenever it's ready ofc
Under the hood Vercel deploys to AWS Lambda. To deliver world class dynamic sites in production, Vercel has turned AWS Lambda into an edge-first compute layer
Wouldn't read-only DB regions be a solution for the global edge problem? (Planetscale can manage that) Or I can't guarantee my global edge function calls my closest read-only database?
You can guarantee that your edge fn calls the closest db.
You’re thinking too far ahead I wasn’t gonna talk about this for at least a month or two…
You shoul'd take a look at Fermyon.
It's basically everything you want when you talk about edge runtime.
It's basically webassembly on the server, it allows you to have no cold start and spin up and down you infra as needed.
When talking about edge locations, you assumed that there always has to be a cold time or some slow database connection build-up, there are solutions where this is not the case (warm standbys). Additionally you can also send HTTP requests to database API from edge locations, this is not only limited to edge runtimes.
Thank you for showing that you could opt into edge on pages as well as API routes, I was trying to find out if you could for like an hour yesterday and couldn't find that anywhere. Everything I see on Vercel is just showing API routes
This concept of regional vs. global edge runtimes is interesting. It makes sense that you want your code running close to the database if you are making many requests. I feel like the major difference between regional and serverless is the cold starts. If serverless somehow solves this issue, that would seem like the best of all worlds, right? No cold start, close to DB, everything is compatible. I guess we gotta wait and see.
If cold starts are solved, I'll probably delete this video
Respect to the editor going back to screenshot that yes from Jacob 😂
Damn the edge looks really impressive. I might take a deep dive into v8 isolates to see how they work and what other tech is using them.
Would be really cool to see a follow up video on moving ping to the edge and seeing a before and after in performance.
"I hate serverless" - yes, someone I can agree with
"I'm just going to move everything to the edge" - Oh great, a new tech stack with a gimmick marketing name.
Thank you for saying "I couldn't care less" 🙇♂
Wow this was crazy informative. I didn’t think edge was much of a big deal and I didn’t understand how much it differed from serverless so I never bothered with it. But now that you broke it down, I’m definitely going to try it out! The amount of value I get out of your channel sometimes is absurd. Thanks Theo. Keep up the good work
I still like good old PostgreSQL
based
very based.
@@abdirahmann let's see Paul Allen's database.
This is a fantastic video. Really wonderful job explaining the trade offs.
Doesn't like slow apps... writes everything with typescript.
"I want my ideal stack to be serverless." - Ryan Dahl
U can use wasm as ur compile target to run rust/c++ on edge
Yes, but you lose our on the hyper fast cold starts promised by edge runtime
using web APIs you use the included runtime whereas using another language could mean you're shipping libs that end up resulting in larger payload with slower start time
The tradeoff does make sense for heavier use cases like image processing
@@rzr1191 Do you have data to back that up? rust WASM has incredibly low cold start times/binary sizes, on lambda rust can be almost an order of magnitude faster to start and 3x less size than an equivalent node function. Why would edge suddenly mean that WASM has worse performance? Also, my main benefit is the fact that I don't have to use typescript (absolute garbage language)
>You will own no servers, and you will be happy
Yeah, no thanks
Edge - the browser
At this point it feels like Theo keeps over-optimizing his app for content
He will eventually discover the benefit of just running a few containers on a VPS
Simplicity and maintainability is far more important than shaving off 100ms from the page load time.
Edge is also used to talk about edge devices. For instance, whether or not a computer vision model is handled online somewhere, or on the physical device, or the "edge device" or "edge processing"
And when I think that for more than half of my and my clients web apps/sites I just use a VPS and it just works fine, with loading times that are good. :( When did deploying stuff become so overcomplicated?
Edge as a location IS IMPORTANT! Just like a CDN; the location edge out-does Regional edge when the data is cacheable (eg 10 latest blog posts; or newest rentals in your city). Edge locations is also important when you can use a distributed DB like the SQLite offering many edge providers now offer.
This video seems already outdated. (Old school thinking)
Great video. I would if you included a bit on how this helps your customers.
Around 21:30
@@rewrose2838 That it? If cold starts and prisma are the bottlenecks it sounds to me like optimizing prisma and hosting on a regular server are the most obvious solutions. People really just jumped on the serverless bandwagon for the *hypothetical* benefit of scalability only to suffer the very real downside of cold starts because they don't actually have users
What presentation tool do you use to explain all about serverless? quick drawing is what kept my attention when you were pointing out the subtle differences between serverless vs edge. Kudos.
Thanks for this great edging tutorial theo
I have one server, in Sydney, Australia. It's good enough for now, and I've built my whole site's system to use it. Who knows what will happen when I officially release it.
That price though...
Edit: What the hell is vercel? am I so much of a dinosaur I can't understand it?
i believe it started out mostly as a frontend hosting solution, then slowly started to support more backend solutions. now they offer simple ways to host fullstack jamstack apps like next, remix, sveltekit etc.
vercel is the sponsor of the channel ;)
am dead at your edit 🤣🤣🤣🤣, a dinosour!!, bruh!!💀
Simple solution: just get rid of all databases. No databases = no database requests. BOOM!
Yeah probably one of his next videos : "I'm done with dynamic pages"
I used to use Google Cloud Functions (Firebase) a few years ago and the cold starts where just down right unacceptable. I built an entire infrastructure on Cloudflare Workers and have never been more happier serving code and sharing libraries
Excellent watch. Tysm
This was incredibly informative! Where can we read more about practical coding for the edge runtime (including handling non-edge-ready dependencies and dealing with data)?
Thoroughly absorbing, isn't it? For a continued absorption, consider a book I highly esteem. "AWS Unleashed: Mastering Amazon Web Services for Software Engineers" by Harrison Quill
Poor, sad Tux Penguin 😢🐧
One day apps will become so close to us they run locally
tfw you start out as bleeding edge JS YT channel but then evolved into a devops channel because thats where all your bottlenecks occur
…You got me
Theo has an Arch Linux mustache in the thumbnail. Can't unsee.
I'd like to emphasize, this is probably only a solution for small complexity apps. If you have a massive system with multiple user types, serving multiple different UIs, might not be the best approach
I think it depends what do you understand under complex apps, 90% of the apps in the internet and corps can be handled by 1 server with database on it
I use the "far edge" the person's own device with service workers!
Do you have a video of you building an app on the edge with planetscale? I find it hard to use planetscale without prisma and would love to see how you do it
I love when you called out Jacob!!!
I feel like one area that has quite some room for improvement is moving all data related things inside the database. Being in the same data center is great but nothing beats being on the same chip.
For requests that don’t do rendering you can avoid spinning up any runtime altogether.
in memory databases reduce your latency from ms to micro seconds :). You basically buy a server with 1TB DDR4 and you good to go xD
Whats the tool Theo is using for drawing all the stuff ?
excalidraw
But what about ridiculous bandwidth cost of vercel? Is that manageable?
Most of the "bandwidth" will be cached after the first requests. Don't serve massive assets off an expensive CDN and you'll be fine.
This is the elephant in the room for all of these services he's using. They all have pretty generous free tiers but balloon in cost for any serious applications. Vercel's pricing is rough enough for my company to be exploring building out infrastructure on AWS just to get Next with all the features it offers without Vercel.
@@JoRyGu Tbh vercel does makes it easier to use cutting edge things like he is talking about a lot. But this is something which is holding me back too. The balloon in the cost is why I opted for self hosted supabase instead of Planetscale. I haven't used amplify yet, but do you know how much worse it is than vercel? Or does it support all the latest features of next 13?
@@JoRyGu you should watch my video about AWS cost if you sincerely believe this.
The bandwidth costs are in line with any other CDN of similar quality. Be smart about where you put your big assets and you’ll be fine. Your Vercel services should return HTML and JSON, if you’re moving terabytes of either you’re probably making a mistake
Tbh, my biggest gripe with vercel rn is that their free tier does not allow commercial usage.
Almost any other provider does (e.g. netlify, heroku formerly, cloudflare pages, etc.).
It's such a shame cause vercel is still probably the most convenient but their free tier is just a deal breaker. Idk why they did that but to me it's simply a bad business decision.
I had observed a very interesting company, it called Azion Technologies! They are building a lot of thing with Edge Computing, I have used a lot the Azion Edge Runtime to accelerate our apps
got to love how a bad implementation(lambda) gets renamed as a new technology(edge)
awesome video, grateful to have this knowledge for free!
Stupid question: WHY are edge functions free of cold start delays? I though edge functions were just serverless functions running on the edge? I feel like a bunch of people came up with a new set of buzzwords without rigorously defining them.
The edge runtime is not the same as edge computing.
Guys eat me, but I am running my projects on Raspberry PI Cluster -> 4x Rasbperry PI 4B 8GB ram hidden behind the firewall, 1GB ethernet connection, all of them on SSD (not SD card) and... this is the 1,5 year and they do just fine :D
Yeah but pal heres the thing, everytime we ain't gonna use planetscale or database which provides apis to interact with their db , so that's a big issue. In that cases, we have to do everytime connecting to db, fire request and close the connection, for every request.
Sounds like lots of room to innovate in the industry 🥲
Postgres with PGBouncer and you have direct connection, pooling manager, don't need to think about open/closing sessions (since it's claims to be milliseconds but Im too often face with the situation when it adds up 100ms on top of the request).
Keep any RDBMS on single server until it's possible to handle all your requests, divide you relational and non-relational data between nosql databases and rdbms, move out analytical requests out of your RDBMS to the warehouse
It's all about architecture and your needs, there are no single solution for all
@Leo S yep, I agree. ECS/Fargate feels like a winner for scaling with RDS databases. DX feels a bit better too. Long running tasks etc.
I can’t wait for the video showing kysely or drizzle orm setup with T3 stack ❤
you always hit the nerve of time, very useful information
Would it be a big win to have the auth built into the database, e.g. request data from a table (db checks permissions and returns data in a single request/response if they have permission and says no otherwise), rather than sending multiple requests to see if they have that permission before requesting the data.
Edge is serverless
say it with me, vercel sponsored channel
Oh my God I’m so glad someone is finally saying this. I’ve wondered for so long why nobody is talking about this.
Vercel is using aws or i am wrong?
Also for the cold start time, its because your code is to big. Sound crazy, but we are in an time where you need to optimize the weight of the backend as we do with the front-end.
For the DB connection, use DynamoDb as mush as possible and dont use external db of aws. Serverless DB from aws is bad also.
So lightweight code, DynamoDb with a aws non-serverless db and it will be fast without cold start lag
16:30 This is my only issue with this video - if the user has slow internet, then global edge will be much, much faster, no? Assuming that the request times are equal assumes that the user has the same internet speed as the server.
Anyways, you are absolutely right when you consider lots of serial requests.
Why would the speed of their internet affect the latency of a call? Bandwidth and latency are entirely different and unrelated lol
prisma just release a blog that how they speed up the init on lambda function
You should check out my last video lol
@@t3dotgg will do!
Yeah it’s awesome!
Btw is it possible already to use just “edge” keyword instead of “experimental-edge” for deployments or it’s different runtimes where the experimental one is for new features like regional edge?
edge and serverless are the saaaaaaaaaaaaaaaaaaaaaaaaaaaaaammmmmmmmmmmmmmmmmeee
I'm enchanted by this content. I recently read a similar book, and I was truly enchanted. "Mastering AWS: A Software Engineers Guide" by Nathan Vale
Prisma has an edge runtime now. Even at the time that this video was posted prisma had it - yet you claim that prisma can't run in edge.
Alright, how much is Vercel paying you?
Use something like Turso and just move your db to the edge as well.
Jeff Bezos teleports behind you, puts a loaded gun to the back of your head, laughs and says “no, you aren’t.” What do you do?
Does anyone know of an edge compatible cuid generator?
The paralleldrive one seems to use node:crypto which doesn't work in edge, and I don't believe Drizzle has a helper for it.
The paralleldrive one _should_ work but doesn't due to a weird bug in bundling for app router. Works in /pages for some reason 😅
I've raised it with vercel and hope they have it handled soon. Suffering with uuid v4 for now 😅
Can somebody link me a video where Theo is talking about Firebase or why he would never consider it If such exists? Just curious of Firebase functions vs vercel edge comparison
The hilarious part to ‘serverless’ is that there still is a server or servers behind it.
‘Serverless’ doesn’t mean no server.. a computer can be a server. So going serverless literally would be like writing code on paper and never having it get complied or presented to you in physical form..
What is that drafting/presentation software on 3:39?
My projects are usually quite simple so I have been using Cloudflare workers and KV Storage to host static website exports and to build basic APIs.
Now they have made their D1 SQL databases for workers open for testing.
Theo can you speak about how u protect your gateways that is in front of your apis? Do u use jwts? So u have a specific GW that has sec embedded?
So mostly about security arround ClientGWAPIs
Thank you