i'm a long time backend sql guy, so i was skeptical about anything said to replace sql, especially after the past decade of non-sql positive hype (typically presenting the benefits of something without the downsides). as you know, engineering decisions are almost always about trade-offs, rather than something is completely awesome/sucks. however, your excellent video here piqued my interest. i'm going to do diligence on EdgeDB for an upcoming project !
To anyone who is in the know… How do these basic queries scale and what is the experience of tuning queries? Having loving syntax is great but I’m genuinely curious to see if this is just a “rapid prototype” database that you’d drop if you get any traction or advanced requirements
Like a lot of things in our industry, these terms become horrendously overloaded. But I get what they meant: edge as in edges and vertices, i.e. graph theory.
Been using self hosted dockered libsql with their client and had speeds surpassing drizzle but now I'm going to have to try this out so I don't have to write sql
@@Frostbytedigital what’s the point of dockerizing libsql? i know it allows for network requests but isn’t the whole point of sqlite that its embeddable?
I've used edgedb since I saw it on one of fireships top 10 dbs videos way back and have absolutely loved how they've keep making it better and better. 10/10
Gonna have to disagree about including its built on postgres on landing page. As a user experimenting with edgedb, that's a piece of info I'm glad to have. I know the core db engine is rock solid because postgres has been around so long. I can guess what features they might easily add by looking at postgres extensions and features. I know I might be able to make some existing tools interop with it.
They actually allow READ-ONLY access to the underlying Postgres Data, so that you can use your favorite Postgres tooling for some complex visuaization or other tasks.
I used EdgeDB when it was quite new, and it was already amazing back then, albeit with some caveats in the query language. Glad to see it grow and becoming better!
I don't really get why people don't like SQL. Personally, I find it fine to work with. What I do get is that there are different kinds of people out there than me, so I'm happy this is a product people can try out and use if they like it. I'll probably be stuck with SQL for a long time cause of my job, but I'm also happy to say that I don't really mind that at all. One thing that does get frustrating is, as in the example Theo points out at 13:41, different implementations of SQL have different ways of writing it, so you can't just write SQL, you *have* to write MySQL or PostgreSQL or Oracle SQL or SQL Server SQL, etc.
Very satisfied with edgedb after using it for 17 months. It's a force multiplier. Never been this fast in my 20 years in the biz, and never written so few bugs with any other db. Name is fine, its graph edges but now that word is pretty much hijacked by the edge so yeah.
SQL 1989?!? I was using it when I was in college in the early 80s and using Informix and Oracle SQL in '87 when I had my first job. Pretty sure SQL had been around a while before that.
edgedb is AMAZING, solves all the problems old SQL systems have, even have integrated migrations system. the language is very comfortable and modern. the documentation is amazing. and it is a front to postgresql which itself is obviously amazing. I have no idea why it isn't all the rage.
He said upfront that he has been advising them, so yes, presumably if they do well as a company, then he'll continue advising them - getting a payday. Rather than the video itself being the source of the funds, his advice is the source
@@Spiker985Studios Doesn't change that he first intended to make it seem like the video isn't biased by saying it isnt sponsored, even though it is obviously is biased considering that.
The first line of the description is a disclosure. A verbal disclosure is at 45 seconds, just after the "hook" How much more disclosed does it need to be?
the fact that they have sdks for almost every language is so cool. I have been wanting to write my backend in go for a while now but there’s just not enough auth libraries that support it. I’ll definitely have to check this out
Self hosting it is super simple, which I very much appreciate. Main gripe with it is the lack of a language server for .esdl files. It has syntax highlighting, but no autocomplete or formatting, which is very annoying.
It'll go away some number of decades after something else has high enough acceptance that people aren't choosing it anymore. Whether that's two decades after next year, or 10 decades after 200 years from now, it'll probably happen eventually.
Of course. It's way too big and integrated into everything. But why not build the new stuff in the new thing, assuming it's all upsides? But I guess time and trial will tell if it really doesn't have significant downsides.
My job is vendor locked with Oracle. We recently had a problem with ref cursors no longer being able to be used in dblinks. They never officially allowed you but we did it anyways since it worked. Starting with 19c they no longer let you at all. Rip our business logic.
orm is a footgun - if you're going to use postgresql then just make functions (you can return json) - also that SQL example on edgedb website is (purposely?) needlessly obscure with subqueries and could more easily/legibly/compactly be written using common table expression
Alas, I doubt you'll get too much support for the idea that ORM is a footgun. I've been apologizing for my colleagues and I accidentally unleashing that nonsense on the world for a decade and a half. Nobody listens because they don't know what else to do.
The big use cases that I always have to keep an eye out for are: * DIstribution - can I have this in pods across multiple datacenters? * Stored Procedures - can I wrap my queries in sprocs to satisfy security requirements? * OpenAPI (Swagger) - Can I generate my types against a JSON schema for interoperability with my ReactJS UI? * Driver Resource Footprint - How much of a hog is this? ODB uses 4gigs for our biggest queries in spikes, otherwise it just sips at memory
Is not a bad idea having new alternatives, the catch is that SQL has decades of development and it's availability is unparalleled. Also another channel, Asianometry, is covering the history of SQL in a series of videos, and how its ideas are the response to the hierarchical databases of the time.
I'm a big fan of EdgeDB(just have a glance and their Book in the docs, it's amazing 🤩), i truly hope they would get much more traction over the coming months The only thing that i'm really not happy about is the polymorphism implementation. Unfortunately folks went the abstract\extends\inheritance way instead of union types, i think this was a mistake. Yes, i know that i can model union types, they got so many things right, but not that "feature", imho.
12:22 null is a fundamental set in set theory. SQL should provide support for null since relational databases are based on sets and set theory. Why is everyone up*set*? 😊
Null is not fundamental to set theory. Unless you mean the empty set Ø. But that is just the initial object in the category of sets. Most of the time null is problematic since having it means you are no longer in a boolean world but need to acommodate three logical values: true, false and null.
@@gencade2504 Read the linked in description article. It talks about the creators of SQL themselves saying that NULL was a mistake and breaks the relational model. It was a mistake that was too late to fix though so it stuck.
Looks cool! I love that it will probably be easier to master than SQL, given all of SQL's edge cases (pun intended), and it's awesome docs. I doubt it'll eclipse SQL cause SQL is SQL, but I can imagine it being the sorta popular alternative people thought Mongo would be.
Awesome content as always, although there's something I don't really understand (and the EdgeDB site itself doesn't really clarify) - the "N+1 problem" is presented here as being a *syntax* problem - as in, it's a pain in the arse to keep having to do sub-queries to get the data you need across multiple tables. But isn't n+1 actually a *performance* concern rather than syntax? e.g. the classic example, if I have a blog posts table, and an authors table, and I want a query that returns some data from each, a single query will need to run a sub query against the authors table once for every blog post that comes back. I don't understand how edgeDB solves that part?
N+1 is an ORM performance problem, usually due to the ORM lazy loading individual values from child tables (ex. load all blog posts, then iterate the posts and get the author name for each post as a separate query). Since EdgeDBs ORM expresses the query as a whole thing, rather the schema being a graph of objects, so it can transform the query into a join executed in the DB itself, so "no N+1" is a function of knowing the totality of the data requested at request time. That said, the subselect in the select statement in the SQL example, for some engines, an N+1 operation, since some engines don't optimize subselects in the select statement into a join at runtime. I need to look at the docs -- the quality of their execution planner and whether they're converting the query to sql first, or using the primitives the execution planner is using under the covers directly could make a big difference. If it's just transforming to SQL, then I'm not sure why it couldn't be a plugin/layer on top, rather than a full DB.
I spent most of the last couple of days playing with edgedb and it's api etc. Brilliant! haven't seen something this interesting since I first discovered Svelte 👍
I think it's fantastic that one area of computing actually found a standard way to do things (SQL) that has held for decades. Nothing else comes close to that kind of universal adoption rate. In the beginning of every project we have to debate so many technologies (and to be fair, SQL comes in many flavors) and I just think it's neat that there's this one thing that just works. That said EdgeDB looks interesting. I'm excited to try it but it'll be unfortunate to lose the universal standard.
Pretty much seems like it. It is a language neutral set (or more likely, a subset) of SQL, that is a bit more convenient to write. I would hate to be the one to hit the capability wall... With ORM you can at least fall back to raw dogging SQL. Someone doing a redis sometime in a distant future should not really stop you from using it. Nothing is stopping noone from pulling a redis.
It honestly looks pretty cool, and if I had a project that I needed to use a DB for that didn't already have a ton of documents/values put in I'd definitely try it out. I've never really used a relational DB since I got started with Mongo and my couple forays into MySQL didn't go far. Even my attempts at using Prisma weren't too great because it doesn't seem to be as plug and play with Mongo as it is with relational databases, although it could also just be the implementation that's the issue
I would have liked to know more about the implementation details of EdgeDB itself - is it just a wrapper around a Postgres DB with some additional functionality? Is it just a different API that abstracts standard DB behavior? I'll look at the docs of course, but I would have enjoyed for this video to explain it for me, since you have had such intimate knowledge of the project! Edit: found it - though unfortunately the docs were no help at all. Look up the talk "The architecture of EdgeDB" given by Fantix King, very helpful! TL;DR: EdgeDB runs their own middleman server in between the Client and the Postgres DB which provides the added functionality
Pardon if this is a bad question because I don't really understand how the architecture of databases work, but does edgedb's query language build SQL queries under the hood, or is there some sort low level functions / Machine code that edgedb calls directly?
@jon1867 No it's not a bad question at all! Never be afraid to ask about things. For what it's worth, both this video (sorry Theo) and the EdgeDB docs itself don't explain the mechanisms of the technology at all unfortunately. You've basically got it down. As far as I understood the video "The architecture of EdgeDB", EdgeDB's query language compiles into EdgeDB-proprietary binary that gets sent to their custom architected middleman server, that then interprets that binary as SQL, then massages it to the native Postgres binary protocol, then after all of that, it communicates to just a standard Postgres instance. The main time saves that EdgeDB achieves is from a caching layer within that middleman server to store general queries + prepared statements, their own binary implementations + using native Postgres binaries which save data over the wire at every touch point in the stack, writing part of their EdgeDB server with Cython and Rust bindings for the most intensive processes, as well as utilizing a more efficient communication protocol to their own server + to the Postgres instance via uvloop The biggest gap in my current knowledge of the technology is that I don't know _how_ the Postgres database is architected after compile / migration time - there are methods of simulating a Graph database e.g. neo4j within Postgres, but I'm just guessing this based on how frequently they use the term 'graph' in the docs. I'll probably spend some time tomorrow getting something up and running to see actually how the database schema looks after the DB has been instantiated
@@TylerJBrown192 One of the questions that raises is, could edgeQL be used on an existing postgres DB with relational tables? Or would it only work on a database built with edgeDB specifically? If so I think adoption for a tool like this could actually be pretty massively viable. And with performance benefits like that, even if people don't wholly drink edgeQL koolaide they could sprinkle it into important parts of their application.
@@jon1867 Damn, good question, I actually had a very similar thought! But it came from me worrying that if I ever brought in EdgeDB to anything I was working on, I just signed myself / my team up for an insane amount of technology / vendor lock-in haha There's been tons of "it's coming" comments I can see from the team since even 2019, but I wasn't able to find anything concrete about how to generate an EdgeDB schema from an existing Postgres database. All of their Migration / EdgeDB CLI functionality seems to only build upon an existing EdgeDB schema, not to be able to generate a schema like many standard SQL ORM-esque libraries can
How does it build on top of PostgreSQL? Is it a hard fork and changed the actual source code? or a layer on top. What do they get from PostgreSQL that they couldn't get anywhere else?
I would definitely try it for some project - but only if it had support for webstorm editor. Otherwise not much interest dealing with plaintext schemas
am i wrong in thinking query vectorization, columnar storage, distributed query engine are the future for sql. what's the problem with sql that we always see ppl asking what's next for sql... lol a more improved sql
How would it handle very large sets? For example, 1B objects that are related to smaller groups of objects. E.g, (Measurement (billions), Measured Variable (hundreds of thousands), variable tag (thousands))
13:37 i may be wrong, but after more than 20 years coding mostly for accountants I think they might be responsible for Null. Several have told me they must have null to indicate unknown as zero could be a valid known amount while it might also indicate "amount unknown" so null makes sense to them... so an outstanding balance on a customer account for example could be zero coz they just paid, or zero because the invoice hasn't been raised yet. I've tried explaining there are alternatives etc. but they think everything is a spreadsheet and get stressed out if there more than 2 dimensions.
I feel like a single bit (boolean), for an "IsInvoiceRaised" value would be infinitely more helpful Inclusion of data instead of the absence of data, every time
@@aislanarislou They did, they just named it NULL. NULL in SQL means unknown, null in programming languages means "uninitialized pointer" or "nothing".
this is not an advertisement trust me bro 100% legit pinky promise they are all my friends but this is for real you know me i mean you don't know me irl but whatever, suscribe!
Name is a pretty gigantic marketing fail, I'm not sure how people continue to misuse well established terms without realizing its a bad idea. I wrote off EdgeDB when I heard of it in the past since I assumed it was edge compute related and not useful to me. They are lucky to have you making this video about it haha. But yeah, the product looks really interesting, I'll give it a try.
EdgeDB started about 10 years ago where the edge in EdgeDB could not really be missunderstood. Today we directly assume it is about edge computing if it has edge in its name. They can either rebrand and loose traction or keep their name and gain traction at a slower rate. Both options suck.
@@michaelbitzer7295 True! But it sounds like they could have changed the name a long while ago and then they would have had several years to regain that traction, but opted not to. I personally think that was probably a mistake, but we'll see if they overcome that. It looks like they already are working towards that, more awareness of what it *actually* is like this video is good progress!
I'm a DBD(Database developer) for close to 2 years now. One of my understanding is this is practically an ORM? One of the things that I dislike ORMs is the fact that when using ORMs is that they do no understand the data you store and how the Engine will interpret it and create the best execution plan there is. This can cause to performance issues. Some of them are easy to solve with indexes other require you to rewrite the query in a way to make the engine go to the execution plan it will be best for performance. No one like slow responses from a DB. Your app can be written Rust and be connected to a badly maintained DB. It does not matter how fast your App is if your DB provides you data slowly or in some cases a bad data. My biggest question here is how do I do my perf tuning in the PostgreSQL or in the EdgeDB platform?
EdeDB has a fully functional Query Analyzer just like conventional SQL databases. Just put "Analyze" is front of your query like you would for most SQL databases.
It is not an orm. My best guess is that it is pretty much a syntactic sugar on top of SQL. What do you mean by "they do not understand the data you store"?
It's not an ORM. It is a Set-based DB implemented on top of an underlying Postgres database. The distinguishing factor is that EdgeDB Normalizes the data on the backend to better support their set fundamentals and optimize for their performance. It allows Postgres to handle efficiently allocating the page files for quick access while more directly controlling the query planning (something ORMs don't typically do outside of bare query splitting for related models). The cool part is since it's 100% open source, you can dig in and see exactly how far they have taken this concept and the implementation details of it.
Same questions here, having tables with millions of rows and some queries will make it very slow, not a fan of those surpises as well, not sure how this is fixed.
@@trapfethen it is not at all a db implementation on top of psql, psql IS the db itself. It does not touch anything related to page files, psql does that. It does not more directly control query planning over normal sql, it uses the exact same interface. Whether typical orms are smart enough to not send multiple queries over the network is beside the point entirely
What I hate about SQL is that it only returns flat data rather than nested objects. What I hate about ORMs is that I feel I don't have enough control over what query is being built, and that it's usually harder and very library specific to do complex things. So I just want to write some version of SQL so I can optimize the f out of that myself, but I want the results to already be nested on the joins I make (or FKs that exist).
I’m really excited about EdgeDB. My biggest hold up right now to using it is that it is new and I’m worried about running into edge cases. Also, I wish I knew a little bit more about the deployment story. Can it be backed by a postgre instance in RDS?
Why can’t we have an orm that lets you write your table definitions in SQL and parse the types from that? Every new orm comes with a new schema definition language 😅
Because ORMs are a fundamentally bad idea. Ted Neward called this out in his 2006 article on "The Vietnam of Computer Science". Neward's arguments on the impedance mismatch are quite good, and my experience bolsters his arguments much, much further. For example, ORMs fundamentally thrash the CPU cache making efficient performance nearly impossible. They also eat memory like candy, which isn't great in a scalable system. You ideally want the path from disk to the client to be as short as possible. Finally, they generate ridiculous amounts of unnecessary code that drastically slows the production of applications that use them. I've been doing something much slimmer and more effective for over a decade now. I've rewritten the platform multiple times for numerous employers. Results are a 20-100x impact on productivity, measured in comparable lines of code metrics. I've finally gotten sick of doing it again and again and am founding my own organization to release it to the world as OSS. Keep an eye out for this alternative later this year.
So its just a wrapper for SQL... meaning you are still using SQL. Also (slightly related) I am sick of people saying NoSQL is better because of performance/scalability. You can scale SQL lol. You pick NoSQL if it fits your data model better.
The name sucks. Everything else looks at the very least promising. And "built on postgres" is a decent selling point, it means that if you don't like it, you can probably just move back to sql with the same data. Or at least migrate it in some decent way. I'll probably play with it a bit, see what's up. Relational databases are great, it's just sql that sucks. It's crazy to think we still haven't found any better alternative.
So, what I get from this, is that EdgeDB is a different query and definition language for PostgreSQL, right? At least I didn't saw any fundamental difference in how data is stored or modeled. It looks that just the way one writes queries and the way one defines the tables is a bit difference and fixes some issues SQL had, but it's not fundamentally capable of different things, right?
The only thing I'd consider a real competitor to SQL would be a variant of datalog imho. Datomic did it really well but had the disadvantage of being proprietary
Unfortunately edgedb docs just give me "Application error: a client-side exception has occurred (see the browser console for more information)." on mobile... Can't see what I want to see.
Hmm? do you mean Graph Databases? I think Graph DBs like Neo4j are indeed challenging the norm, especially with Cypher language that is being adopted by other Graph DBs, it's just not going mainstream fast enough...
I guess, my concern, when considering switching to a different database is that I'd have to learn how to write queries safely. Like, with SQL you have the potential for SQL injections and there are techniques to mitigate it like prepared statements and such. Is there documentation on how to write EdgeDB queries safely?
(Edit: this query is buggy and fixing it results in a query not better than the one listed in the docs. Moral of the story: Don't drink and SQL) While I agree that EdgeDB is a large improvement over any ORM I've personally used I'm still not a huge fan of the Strawman raw SQL query provided. In my opinion someone who is comfortable with SQL would more realistically write something like this 7-liner: SELECT m.title, avg(r.rating) avg_rating, array_agg(p.name) FILTER (WHERE p.name IS NOT NULL) actors FROM movies m LEFT JOIN reviews r ON r.movie_id = m.id LEFT JOIN actors a ON a.movie_id = m.id LEFT JOIN persons p ON p.id = a.person_id WHERE m.id = $1 GROUP BY m.id;
This query would produce wrong results: duplicated actor names and while `avg` would still be correct, if you'd used a `sum` instead it'd be wrong also.
@@ElvisPranskevichusAh, good catch. I let my personal hatred of nesting selects go a little overboard. The `q` subquery still does bother me when an `AND` or a `FILTER` could do it's job.
BTW I'M JUDGING THE EDGEDB HACKATHON. WINNER GETS $5k. hackathon.edgedb.com/
i'm a long time backend sql guy, so i was skeptical about anything said to replace sql, especially after the past decade of non-sql positive hype (typically presenting the benefits of something without the downsides). as you know, engineering decisions are almost always about trade-offs, rather than something is completely awesome/sucks. however, your excellent video here piqued my interest. i'm going to do diligence on EdgeDB for an upcoming project !
To anyone who is in the know… How do these basic queries scale and what is the experience of tuning queries? Having loving syntax is great but I’m genuinely curious to see if this is just a “rapid prototype” database that you’d drop if you get any traction or advanced requirements
the name is horrendous.
clicked on this video wondering why would someone need an entire DB runnign on the edge....
Lol, do you know that Edge also refers to edges in Graphs ?
I didn't :') @@erenjeager1756
Like a lot of things in our industry, these terms become horrendously overloaded. But I get what they meant: edge as in edges and vertices, i.e. graph theory.
It's not their fault that Guillermo Rauch spams the word edge too much on Twitter.
not 100% but some would be nice, replacing redis for me. in memory postgresql is cool
THEY HAVE DOCS FOR RUNNING ON DOCKER OR BARE METAL LETS GO SELF HOST GANG
Been using self hosted dockered libsql with their client and had speeds surpassing drizzle but now I'm going to have to try this out so I don't have to write sql
@@Frostbytedigital what’s the point of dockerizing libsql? i know it allows for network requests but isn’t the whole point of sqlite that its embeddable?
do i get all the GUI dashabord query builder and other stuff provided when i go with their cloud solution ? thou otherwise what's the point?
@@joe_ervin10 sqlite is embeddable libsql makes it shareable; you need a server to fetch the initial db to clone for an embedded user for example
@@GreatTaiwanI also want to know if we we get a GUI dashboard while running a dockerised solution.
I've used edgedb since I saw it on one of fireships top 10 dbs videos way back and have absolutely loved how they've keep making it better and better. 10/10
Gonna have to disagree about including its built on postgres on landing page. As a user experimenting with edgedb, that's a piece of info I'm glad to have. I know the core db engine is rock solid because postgres has been around so long. I can guess what features they might easily add by looking at postgres extensions and features. I know I might be able to make some existing tools interop with it.
They actually allow READ-ONLY access to the underlying Postgres Data, so that you can use your favorite Postgres tooling for some complex visuaization or other tasks.
Not excited about the vacuuming, though
It's Like GraphQL without the GraphQL burden
And with GraphQL 😂
the dgraph query language is limited like graphql, this looks more powerful.
It's like graphQL, but with edging
Truly the best Edge(ing) experience (tm)
Groan. Also (tm)
I used EdgeDB when it was quite new, and it was already amazing back then, albeit with some caveats in the query language. Glad to see it grow and becoming better!
I don't really get why people don't like SQL. Personally, I find it fine to work with. What I do get is that there are different kinds of people out there than me, so I'm happy this is a product people can try out and use if they like it. I'll probably be stuck with SQL for a long time cause of my job, but I'm also happy to say that I don't really mind that at all.
One thing that does get frustrating is, as in the example Theo points out at 13:41, different implementations of SQL have different ways of writing it, so you can't just write SQL, you *have* to write MySQL or PostgreSQL or Oracle SQL or SQL Server SQL, etc.
Very satisfied with edgedb after using it for 17 months.
It's a force multiplier. Never been this fast in my 20 years in the biz, and never written so few bugs with any other db.
Name is fine, its graph edges but now that word is pretty much hijacked by the edge so yeah.
SQL 1989?!? I was using it when I was in college in the early 80s and using Informix and Oracle SQL in '87 when I had my first job. Pretty sure SQL had been around a while before that.
Yeah it's from 1974. I can't think of what he's confusing with that was introduced in 1989
Theo in the impet to pump-up his friend's DB made a poo-poo.
Theo makes a lot of mistakes in his videos. I keep that in mind when watching these.
According to Wikipedia, it first appeared in 1974, so 50 years ago!
edgedb is AMAZING, solves all the problems old SQL systems have, even have integrated migrations system. the language is very comfortable and modern. the documentation is amazing. and it is a front to postgresql which itself is obviously amazing. I have no idea why it isn't all the rage.
Because it was still in beta last time I checked a few months ago
@@boscodomingo how long ago was that? It’s on version 1.5 already and has a cloud version.
Could you make a video comparing EdgeDB with SurrealDB ?
"they are not a sponsor" [...] "if they suceed, i get a bit of a payday" ah yes
He said upfront that he has been advising them, so yes, presumably if they do well as a company, then he'll continue advising them - getting a payday.
Rather than the video itself being the source of the funds, his advice is the source
This isn't the dunk you think it is
@@Spiker985Studios Doesn't change that he first intended to make it seem like the video isn't biased by saying it isnt sponsored, even though it is obviously is biased considering that.
The first line of the description is a disclosure. A verbal disclosure is at 45 seconds, just after the "hook"
How much more disclosed does it need to be?
@@GoldenretriverYT He never said that the video isn't biased, it is. He just said that it wasn't sponsored since it isn't.
I am building a developer tool that uses EdgeDB as a core feature. Glad it’s getting attention.
the fact that they have sdks for almost every language is so cool. I have been wanting to write my backend in go for a while now but there’s just not enough auth libraries that support it. I’ll definitely have to check this out
No neo4j mentioned?
would have been nice to see the movie example in Cypher
ikr? this is basically just an aura db with a graphql layer on top (but obviously, is handled a bit nicer)
Perhaps because Neo4j is not a relational DB? It's a graph DB. 😊
I've tried it for a while and IMHO it's awesome! especially integrated DB migration, and query language looks good too.
I appreciate the disclaimers, thanks Theo
As much as I love SurrealDB, I must say, this shit is HOT
Self hosting it is super simple, which I very much appreciate. Main gripe with it is the lack of a language server for .esdl files. It has syntax highlighting, but no autocomplete or formatting, which is very annoying.
Working on that.
@@YurySelivanov I’ve been waiting eagerly since the Rust parser blog post. Good to hear it’s still being worked on. Looking forward to it.
Same. And there is one specific bug with their eqdeql js lib. But that's about it
EdgeDB seems to respect and implement aspects of The Third Manifesto, which talks about doing relational more properly in a DBMS than SQL does.
The real problem with NULL in SQL is that there's only 1 kind of NULL, unlike JS that has 3! And that solves all problems with NULL. ALL PROBLEMS!
andy said sql ain't going anywhere no matter whatever new gimmick comes around.
I mean of course it’s not, but that doesn’t mean this isn’t cool
It'll go away some number of decades after something else has high enough acceptance that people aren't choosing it anymore. Whether that's two decades after next year, or 10 decades after 200 years from now, it'll probably happen eventually.
Of course. It's way too big and integrated into everything. But why not build the new stuff in the new thing, assuming it's all upsides?
But I guess time and trial will tell if it really doesn't have significant downsides.
SQL will continue to exist for a long time, just like Cobol.
My job is vendor locked with Oracle. We recently had a problem with ref cursors no longer being able to be used in dblinks. They never officially allowed you but we did it anyways since it worked. Starting with 19c they no longer let you at all. Rip our business logic.
orm is a footgun - if you're going to use postgresql then just make functions (you can return json) - also that SQL example on edgedb website is (purposely?) needlessly obscure with subqueries and could more easily/legibly/compactly be written using common table expression
Alas, I doubt you'll get too much support for the idea that ORM is a footgun. I've been apologizing for my colleagues and I accidentally unleashing that nonsense on the world for a decade and a half. Nobody listens because they don't know what else to do.
I would rewrite "based on PostgreSQL" to be "powered by PostgreSQL"
"Based on", to me, implies that the source is "based on" that thing
"written on top a PostgreSQL foundation"
Finally we can edge in peace
The big use cases that I always have to keep an eye out for are:
* DIstribution - can I have this in pods across multiple datacenters?
* Stored Procedures - can I wrap my queries in sprocs to satisfy security requirements?
* OpenAPI (Swagger) - Can I generate my types against a JSON schema for interoperability with my ReactJS UI?
* Driver Resource Footprint - How much of a hog is this? ODB uses 4gigs for our biggest queries in spikes, otherwise it just sips at memory
The whole market is regressing back to the SQL api. This is a bold move cotton.
Is not a bad idea having new alternatives, the catch is that SQL has decades of development and it's availability is unparalleled.
Also another channel, Asianometry, is covering the history of SQL in a series of videos, and how its ideas are the response to the hierarchical databases of the time.
I'm a big fan of EdgeDB(just have a glance and their Book in the docs, it's amazing 🤩), i truly hope they would get much more traction over the coming months
The only thing that i'm really not happy about is the polymorphism implementation. Unfortunately folks went the abstract\extends\inheritance way instead of union types, i think this was a mistake. Yes, i know that i can model union types, they got so many things right, but not that "feature", imho.
12:22 null is a fundamental set in set theory. SQL should provide support for null since relational databases are based on sets and set theory. Why is everyone up*set*? 😊
Exactly
Null is not fundamental to set theory. Unless you mean the empty set Ø. But that is just the initial object in the category of sets. Most of the time null is problematic since having it means you are no longer in a boolean world but need to acommodate three logical values: true, false and null.
There is no null in set theory. Null is a design 'mistake' in computer science.
@@gencade2504 Read the linked in description article. It talks about the creators of SQL themselves saying that NULL was a mistake and breaks the relational model. It was a mistake that was too late to fix though so it stuck.
Between NULL and timezones, also feeling making sand think was a mistake
Looks cool! I love that it will probably be easier to master than SQL, given all of SQL's edge cases (pun intended), and it's awesome docs. I doubt it'll eclipse SQL cause SQL is SQL, but I can imagine it being the sorta popular alternative people thought Mongo would be.
Prisma frustated me soo much that I love EdgeDB since day 1 I started using it
So, if it's based in Postgres... It's just a wrapper over sql?
yes
Yes but a very different and very good one.
This is going to have huge success on junior devs and some companies that will buy the pitch or ripe some bens. sql is not good enough ....
The beginning has that feeling of watching a movie with somebody, who have seen it and they constantly pause to explain some little stuff
Awesome content as always, although there's something I don't really understand (and the EdgeDB site itself doesn't really clarify) - the "N+1 problem" is presented here as being a *syntax* problem - as in, it's a pain in the arse to keep having to do sub-queries to get the data you need across multiple tables.
But isn't n+1 actually a *performance* concern rather than syntax? e.g. the classic example, if I have a blog posts table, and an authors table, and I want a query that returns some data from each, a single query will need to run a sub query against the authors table once for every blog post that comes back. I don't understand how edgeDB solves that part?
N+1 is an ORM performance problem, usually due to the ORM lazy loading individual values from child tables (ex. load all blog posts, then iterate the posts and get the author name for each post as a separate query). Since EdgeDBs ORM expresses the query as a whole thing, rather the schema being a graph of objects, so it can transform the query into a join executed in the DB itself, so "no N+1" is a function of knowing the totality of the data requested at request time. That said, the subselect in the select statement in the SQL example, for some engines, an N+1 operation, since some engines don't optimize subselects in the select statement into a join at runtime. I need to look at the docs -- the quality of their execution planner and whether they're converting the query to sql first, or using the primitives the execution planner is using under the covers directly could make a big difference. If it's just transforming to SQL, then I'm not sure why it couldn't be a plugin/layer on top, rather than a full DB.
@@alexlapins533 Makes sense!
I spent most of the last couple of days playing with edgedb and it's api etc. Brilliant! haven't seen something this interesting since I first discovered Svelte 👍
Great story about SQL and what may come next. Please lead us further down the path!
And here I'm writing almost all my SQL in stored procs and functions in MySQL Workbench. I feel old.
I think it's fantastic that one area of computing actually found a standard way to do things (SQL) that has held for decades. Nothing else comes close to that kind of universal adoption rate. In the beginning of every project we have to debate so many technologies (and to be fair, SQL comes in many flavors) and I just think it's neat that there's this one thing that just works.
That said EdgeDB looks interesting. I'm excited to try it but it'll be unfortunate to lose the universal standard.
can one migrate from sql to edgedb and the other way around too
SQL 1989... you may want to check with your video editor on that one...
So it is just an abstraction over plain SQL? And what is stopping them from doing a redis?
Pretty much seems like it. It is a language neutral set (or more likely, a subset) of SQL, that is a bit more convenient to write. I would hate to be the one to hit the capability wall... With ORM you can at least fall back to raw dogging SQL.
Someone doing a redis sometime in a distant future should not really stop you from using it. Nothing is stopping noone from pulling a redis.
It honestly looks pretty cool, and if I had a project that I needed to use a DB for that didn't already have a ton of documents/values put in I'd definitely try it out. I've never really used a relational DB since I got started with Mongo and my couple forays into MySQL didn't go far. Even my attempts at using Prisma weren't too great because it doesn't seem to be as plug and play with Mongo as it is with relational databases, although it could also just be the implementation that's the issue
what about surrealDB?
Surreal is too new, tries to cover a LOT. and to implement features, they must build it almost from scratch.
What I like about EdgeDB is that it’s very accessible and easy to learn.
I would have liked to know more about the implementation details of EdgeDB itself - is it just a wrapper around a Postgres DB with some additional functionality? Is it just a different API that abstracts standard DB behavior? I'll look at the docs of course, but I would have enjoyed for this video to explain it for me, since you have had such intimate knowledge of the project!
Edit: found it - though unfortunately the docs were no help at all. Look up the talk "The architecture of EdgeDB" given by Fantix King, very helpful! TL;DR: EdgeDB runs their own middleman server in between the Client and the Postgres DB which provides the added functionality
Yes.
Edit: saw your edit
Pardon if this is a bad question because I don't really understand how the architecture of databases work, but does edgedb's query language build SQL queries under the hood, or is there some sort low level functions / Machine code that edgedb calls directly?
@jon1867 No it's not a bad question at all! Never be afraid to ask about things. For what it's worth, both this video (sorry Theo) and the EdgeDB docs itself don't explain the mechanisms of the technology at all unfortunately.
You've basically got it down. As far as I understood the video "The architecture of EdgeDB", EdgeDB's query language compiles into EdgeDB-proprietary binary that gets sent to their custom architected middleman server, that then interprets that binary as SQL, then massages it to the native Postgres binary protocol, then after all of that, it communicates to just a standard Postgres instance.
The main time saves that EdgeDB achieves is from a caching layer within that middleman server to store general queries + prepared statements, their own binary implementations + using native Postgres binaries which save data over the wire at every touch point in the stack, writing part of their EdgeDB server with Cython and Rust bindings for the most intensive processes, as well as utilizing a more efficient communication protocol to their own server + to the Postgres instance via uvloop
The biggest gap in my current knowledge of the technology is that I don't know _how_ the Postgres database is architected after compile / migration time - there are methods of simulating a Graph database e.g. neo4j within Postgres, but I'm just guessing this based on how frequently they use the term 'graph' in the docs. I'll probably spend some time tomorrow getting something up and running to see actually how the database schema looks after the DB has been instantiated
@@TylerJBrown192 One of the questions that raises is, could edgeQL be used on an existing postgres DB with relational tables? Or would it only work on a database built with edgeDB specifically?
If so I think adoption for a tool like this could actually be pretty massively viable. And with performance benefits like that, even if people don't wholly drink edgeQL koolaide they could sprinkle it into important parts of their application.
@@jon1867 Damn, good question, I actually had a very similar thought! But it came from me worrying that if I ever brought in EdgeDB to anything I was working on, I just signed myself / my team up for an insane amount of technology / vendor lock-in haha
There's been tons of "it's coming" comments I can see from the team since even 2019, but I wasn't able to find anything concrete about how to generate an EdgeDB schema from an existing Postgres database. All of their Migration / EdgeDB CLI functionality seems to only build upon an existing EdgeDB schema, not to be able to generate a schema like many standard SQL ORM-esque libraries can
How does it build on top of PostgreSQL? Is it a hard fork and changed the actual source code? or a layer on top. What do they get from PostgreSQL that they couldn't get anywhere else?
I would definitely try it for some project - but only if it had support for webstorm editor. Otherwise not much interest dealing with plaintext schemas
am i wrong in thinking query vectorization, columnar storage, distributed query engine are the future for sql. what's the problem with sql that we always see ppl asking what's next for sql... lol a more improved sql
this. edge db feels like a step in that direction. Maybe will become inspiration for the future sql?
The iceberg graphic alone had me to go the website just to see it for myself
Looks pretty sweet, assuming the Edge in the name is for edges on a graph?
Correct.
finally a video that youre not just whining but artually sharing information
What do you think about Convex?
Convex is really good if you need websocket-like speed from updated data (like a chat app) but it falters in writing complex queries
I am sold already
Can it be supabase alternative?
Great video. Simply superb.
How would it handle very large sets? For example, 1B objects that are related to smaller groups of objects. E.g, (Measurement (billions), Measured Variable (hundreds of thousands), variable tag (thousands))
13:37 i may be wrong, but after more than 20 years coding mostly for accountants I think they might be responsible for Null. Several have told me they must have null to indicate unknown as zero could be a valid known amount while it might also indicate "amount unknown" so null makes sense to them... so an outstanding balance on a customer account for example could be zero coz they just paid, or zero because the invoice hasn't been raised yet. I've tried explaining there are alternatives etc. but they think everything is a spreadsheet and get stressed out if there more than 2 dimensions.
I feel like a single bit (boolean), for an "IsInvoiceRaised" value would be infinitely more helpful
Inclusion of data instead of the absence of data, every time
If that reason is true, so maybe a valid type UNKNOWN should have created back then......
@@aislanarislou They did, they just named it NULL. NULL in SQL means unknown, null in programming languages means "uninitialized pointer" or "nothing".
this is not an advertisement trust me bro 100% legit pinky promise they are all my friends but this is for real you know me i mean you don't know me irl but whatever, suscribe!
I love to edgeDB at work
Name is a pretty gigantic marketing fail, I'm not sure how people continue to misuse well established terms without realizing its a bad idea. I wrote off EdgeDB when I heard of it in the past since I assumed it was edge compute related and not useful to me. They are lucky to have you making this video about it haha.
But yeah, the product looks really interesting, I'll give it a try.
I'll be real, I hate the name. Tried hard early on to get them to change it lol
@@t3dotgg Yep glad you brought it up in the video at the start cause I nearly clicked off haha
EdgeDB started about 10 years ago where the edge in EdgeDB could not really be missunderstood. Today we directly assume it is about edge computing if it has edge in its name.
They can either rebrand and loose traction or keep their name and gain traction at a slower rate. Both options suck.
@@michaelbitzer7295 True! But it sounds like they could have changed the name a long while ago and then they would have had several years to regain that traction, but opted not to. I personally think that was probably a mistake, but we'll see if they overcome that. It looks like they already are working towards that, more awareness of what it *actually* is like this video is good progress!
So first the vercel stopped edging, just for my database to do so instead
I'm a DBD(Database developer) for close to 2 years now. One of my understanding is this is practically an ORM?
One of the things that I dislike ORMs is the fact that when using ORMs is that they do no understand the data you store and how the Engine will interpret it and create the best execution plan there is. This can cause to performance issues. Some of them are easy to solve with indexes other require you to rewrite the query in a way to make the engine go to the execution plan it will be best for performance. No one like slow responses from a DB. Your app can be written Rust and be connected to a badly maintained DB. It does not matter how fast your App is if your DB provides you data slowly or in some cases a bad data.
My biggest question here is how do I do my perf tuning in the PostgreSQL or in the EdgeDB platform?
EdeDB has a fully functional Query Analyzer just like conventional SQL databases. Just put "Analyze" is front of your query like you would for most SQL databases.
It is not an orm. My best guess is that it is pretty much a syntactic sugar on top of SQL. What do you mean by "they do not understand the data you store"?
It's not an ORM. It is a Set-based DB implemented on top of an underlying Postgres database. The distinguishing factor is that EdgeDB Normalizes the data on the backend to better support their set fundamentals and optimize for their performance. It allows Postgres to handle efficiently allocating the page files for quick access while more directly controlling the query planning (something ORMs don't typically do outside of bare query splitting for related models).
The cool part is since it's 100% open source, you can dig in and see exactly how far they have taken this concept and the implementation details of it.
Same questions here, having tables with millions of rows and some queries will make it very slow, not a fan of those surpises as well, not sure how this is fixed.
@@trapfethen it is not at all a db implementation on top of psql, psql IS the db itself. It does not touch anything related to page files, psql does that. It does not more directly control query planning over normal sql, it uses the exact same interface. Whether typical orms are smart enough to not send multiple queries over the network is beside the point entirely
No... keep the name, I can finally say that I'm 'edging my database' (gen z brainrot is catching up to me fr 🗣)
What I hate about SQL is that it only returns flat data rather than nested objects. What I hate about ORMs is that I feel I don't have enough control over what query is being built, and that it's usually harder and very library specific to do complex things.
So I just want to write some version of SQL so I can optimize the f out of that myself, but I want the results to already be nested on the joins I make (or FKs that exist).
I’m really excited about EdgeDB. My biggest hold up right now to using it is that it is new and I’m worried about running into edge cases.
Also, I wish I knew a little bit more about the deployment story. Can it be backed by a postgre instance in RDS?
Why can’t we have an orm that lets you write your table definitions in SQL and parse the types from that? Every new orm comes with a new schema definition language 😅
Because ORMs are a fundamentally bad idea. Ted Neward called this out in his 2006 article on "The Vietnam of Computer Science". Neward's arguments on the impedance mismatch are quite good, and my experience bolsters his arguments much, much further. For example, ORMs fundamentally thrash the CPU cache making efficient performance nearly impossible. They also eat memory like candy, which isn't great in a scalable system. You ideally want the path from disk to the client to be as short as possible. Finally, they generate ridiculous amounts of unnecessary code that drastically slows the production of applications that use them.
I've been doing something much slimmer and more effective for over a decade now. I've rewritten the platform multiple times for numerous employers. Results are a 20-100x impact on productivity, measured in comparable lines of code metrics. I've finally gotten sick of doing it again and again and am founding my own organization to release it to the world as OSS. Keep an eye out for this alternative later this year.
So its just a wrapper for SQL... meaning you are still using SQL.
Also (slightly related) I am sick of people saying NoSQL is better because of performance/scalability. You can scale SQL lol. You pick NoSQL if it fits your data model better.
The name sucks. Everything else looks at the very least promising. And "built on postgres" is a decent selling point, it means that if you don't like it, you can probably just move back to sql with the same data. Or at least migrate it in some decent way.
I'll probably play with it a bit, see what's up. Relational databases are great, it's just sql that sucks. It's crazy to think we still haven't found any better alternative.
Looks great. I'd be in if I didn't lose 1000 hours of my life buying into RethinkDB.
Thank you for introducing me to Edging...
So, what I get from this, is that EdgeDB is a different query and definition language for PostgreSQL, right? At least I didn't saw any fundamental difference in how data is stored or modeled. It looks that just the way one writes queries and the way one defines the tables is a bit difference and fixes some issues SQL had, but it's not fundamentally capable of different things, right?
The iceburg is coolest thing ever
The only thing I'd consider a real competitor to SQL would be a variant of datalog imho. Datomic did it really well but had the disadvantage of being proprietary
If its open source based on postgress you c extensions to it can use postgress extensions for stuff like vector math
10:20 how do you snapshot your schema changes and manage migrations??
Unfortunately edgedb docs just give me "Application error: a client-side exception has occurred (see the browser console for more information)." on mobile... Can't see what I want to see.
Hmm? do you mean Graph Databases? I think Graph DBs like Neo4j are indeed challenging the norm, especially with Cypher language that is being adopted by other Graph DBs, it's just not going mainstream fast enough...
Building on top of postgres is a good idea. Storage engines are hard to write. It can always be swapped out later.
I guess, my concern, when considering switching to a different database is that I'd have to learn how to write queries safely. Like, with SQL you have the potential for SQL injections and there are techniques to mitigate it like prepared statements and such. Is there documentation on how to write EdgeDB queries safely?
Unless you concatenate EdgeQL queries with data there's 0 chance of any injections.
thank you. I edge to this database
I'm guessing Drizzle use functions rather than a big class holding all the possible methods for Tree Shaking
#Ad
"if they succeed, I get bit of a payday" lmao
"they are not a sponsor" [...] "if they suceed, i get a bit of a payday" ah yes
Pro tip: almost everything on TH-cam is an ad or promotion. People like to earn money so they can buy food, you know.
@@7th_CAV_Trooper and honest people preface it with letting their watchers know.
Those who don't, are hmmmm. I don't wanna take names here. 😂
I just spent so much time learning drizzle after learning prisma after learning graphql.... I still just want to write my own sql lol
SurrealDB has potential. Document, tabular and graph; all in one
until you find a critical feature missing, and have to wait 2 years for it to be implemented from scratch.
Now I'm lost, do I try to use EdgeDB for my new side project or do I stick to drizzle/mariadb?
EdgeDB for sure. I use it. It works great, the dox are great too as well as the community on discord.
I’m building a personal project, I’ll give it a try since they offer a free tier!
Great sound ideas, but the name is going to wreck it from wide adoption.
Guys I got it! Nobel prize moment: THE MOAT IS IN THE FONT!😅
The SQL code in the comparison wouldn't pass code review 😕 What happened to steelmanning?
What do you mean by "If they succeed, I get a bit of a payday"? Are you an investor in EdgeDB?
Advisors often get stock options as compensation for their time/advice
Why not Postgres + JSONB columns?
wow this looks boss
So this is basically an ORM for Postgres with extra steps?
Their code runners are so dope indeed.
Remember when MongoDB and NoSQL was all the rage? Yeah I don’t hear about it that much anymore
MongoDB is literally 28% in stackoverflow's survey, L take
I don't understand it, if I have a postgress instance running can I just use Edgedb?
(Edit: this query is buggy and fixing it results in a query not better than the one listed in the docs. Moral of the story: Don't drink and SQL)
While I agree that EdgeDB is a large improvement over any ORM I've personally used I'm still not a huge fan of the Strawman raw SQL query provided.
In my opinion someone who is comfortable with SQL would more realistically write something like this 7-liner:
SELECT m.title, avg(r.rating) avg_rating, array_agg(p.name) FILTER (WHERE p.name IS NOT NULL) actors
FROM movies m
LEFT JOIN reviews r ON r.movie_id = m.id
LEFT JOIN actors a ON a.movie_id = m.id
LEFT JOIN persons p ON p.id = a.person_id
WHERE m.id = $1
GROUP BY m.id;
This query would produce wrong results: duplicated actor names and while `avg` would still be correct, if you'd used a `sum` instead it'd be wrong also.
@@ElvisPranskevichusAh, good catch. I let my personal hatred of nesting selects go a little overboard.
The `q` subquery still does bother me when an `AND` or a `FILTER` could do it's job.
@@Apoque Yeah, that can be cleaned up. Thanks!