Why I'm switching from Postgres to a real-time database
ฝัง
- เผยแพร่เมื่อ 16 ม.ค. 2024
- My Courses
📘 T3 Stack Tutorial: 1017897100294.gumroad.com/l/j...
My Products
📖 ProjectPlannerAI: projectplannerai.com
🤖 IconGeneratorAI: icongeneratorai.com/
Useful Links
💬 Discord: / discord
🔔 Newsletter: newsletter.webdevcody.com/
📁 GitHub: github.com/webdevcody
📺 Twitch: / webdevcody
🤖 Website: webdevcody.com
🐦 Twitter: / webdevcody
Interesting project. Would love more details and the next steps a user needs to take in order to make their idea real
Your videos are increadly inspirational - thank you Cody. You kickstart my productivity every morning.
I've never tried a real time database before but the way you explain it makes so much sense. I'm definitely going try this now in a future video!
Digging the new thumbnails. Levelling up I see!
thanks!
Really interesting project! Wish you good luck with it.
Convex seems like a useful SAAS and pricing is not too bad. Will throw it into the toolbox 👍
love this project idea! keep the videos coming!
I understand what you've done. It makes sense to use BaaS instead of creating your own sockets ! I did the same in one of my projects. Super good video ! Love the content
The project looks pretty interesting, could you think to make a video about how should we utilize ChatGPT (or any other AI) to get what we want? What kind of prompting going on behind the scenes?
The automatic updates look sick. Would be cool to see a comparison video of different db types and why you'd pick one or the other.
Very interesting! I have two questions. What do you use to deploy your Next.js 14 app on AWS? OpenNext? AWS Amplify? Do all functionalities (server actions, etc.) on Next.js 14 work on AWS?
I usually deploy using open next. Everything works on open next except the middleware acts a bit different compared to vercel, and streaming is experimental. I can't speak about amplify.
I use Supabase to do this now, which uses postgres. It has realtime websocket subs to tables. No manual work required (One line of JS). There's also an async http client and an s3-backed storage abstraction.
I tried setting up supabase with RDS horrible experience, you basically can't do that. So what options do you have have if your cloud stack is in AWS? And you want something managed? What did you go for?
Assistance with the project planning portion would be helpful. First, who is the target user? Second, what features should it have? Third, for each feature, what is the flow the user will go through to use it? Autogenerating mockups will also be helpful. FigJam AI is neat, you say "draw this and this", and it auto generates a diagram. Something like that for rapid idea + visual iteration would be neat.
Awesome video! Just wondering why you didn't use streaming for this? For example, you could stream a JSON structor that updates the parts of the plan that are available as they come in.
As in run a server action and return an array with an unawaited promise? First I haven’t seen any good documentation on that in the next docs and I’m not sure if that’s even a supported approach of a hack that happens to just work. Second, I’m pretty sure actions cancel as I navigate (I’d need to test), which is very limiting because I want the flexibility to run the action on one page, then redirect to another page, then update as the results come in. Convex was just easier
@@WebDevCody makes sense! I was thinking steaming results via an api endpoint using the “edge” runtime. You could fetch the result client side and stream in the data as it’s available. Probably a lot more to manage and a lot more difficult then using convex.
Seems like convex was easy to run with so definitely a good choice. Was just curious if you thought about streaming an as approach.
I took alot of inspiration from your hackathon project using convex and how little boiler plate backend code is needed.
I was wondering if you could touch on how you structure your data differently compared to a relational database?
I’ve found joins quite difficult in convex and end up making more database calls instead. But I think this is because of my table structure.
what issues have you found just doing a .map and doing more queries to get related data? The convex team says the mutations and queries basically run on the same machine as the database, so they run very fast.
I can't save for later for some reason so commenting to remember to come back to this.
Hi! Thanks for this video. Could you make a video where you compare supabase and convex?
very intersting, Cody. Thanks for sharing. We have a similar setup but using Firebase RealtimeDB instead. The problem I'm running into is how to architect for testability with this. Any ideas or opinions?
Testability for what, firebase?
@@WebDevCody unit/low-level integration tests for the component
Damn this is huge! I’ve written a call centre product with Twilio programmable voice. Im going to refactor it using this
I was thinking about the database for my app. I was in the middle planetscale or convex. I needed real time. With this video and your graph, I decided what to use. Firebase can be good options, but is Google today we have firebase next month, maybe not.😅
It's survived almost 12+ years I believe (granted not always under Google's ownership...but still!)
how did you do the sections loading like that?
I tried to login I get.
Sign up unsuccessful due to failed bot validation. Please refresh the page to try again or reach out to support for more assistance.
Great vid! Is it possible to do a comparison between auth0 with clerk
Hey man, can you do a video on how you develop your UI?
I think supabase has realtime as well. It would be great if you could compare those technologies.
from what I understand, supabase requires you to write logic to subscribe to rows changes specifically, vs convex automatically re-runs your queries when any of the database queries you do under the hood update. It's another abstraction over realtime data which makes it so much easier. Also typescript support is much better last time I used supabase's client libraries
@@WebDevCody Thank you, I just checked convex pricing model, they didn't specify any additional cost for the realtime data. On the other hand, supabase realtime can only support 500 websockets connection even you pay $25/m which seems quite expensive. Do you understand what make them so different? Why are the websocket connection convex much more cost effective?
@@WebDevCody Sounds more like Firebase's RTDB/Firestore then perhaps?
I see you mentioned that actually
Sup Cody. I can see that maintaining websocket connections with convex for this project makes sense, but do you have any info on scalability? Would these connections scale to a million concurrent users?
No clue about how well convex scales, you’d want to ask their team about that. My use case outlined in this video is for my small side project. I’m assuming they can handle a lot of concurrent websocket connections
I need unlimited scalability for my 0 users.
How did you create the graphics for the landing page of the marketing page?
We found existing landing page sections I think. There are tons of free resources out there
@@WebDevCody Thanks for the quick reply, I was actually referring to the short video that showcases the webapp on the homepage…
@@tsatsuadogla-bessa8732oh that was made using screen studio, it's a paid software to record your screen and make great marketing videos
@@WebDevCody Thank you very much! Also after this video finally decided to try convex for a small side project for my church! Will let you know how that goes
Do you use a Langchain? It has a possibility to output valid json, so you dont need to recall openai for the correct outputs
Don't use langchain, look up the article titled "The Problem with Langchain". I can't link it in comments or else it'll get deleted.
Another great video!! Thanks for sharing it!!
I was wondering how Convex compares with Vercel?? Have you used it enough to say Convex is a better option??
convex and vercel are two completly different category of tech. Vercel is a host, convex is a BaaS (including the database and file storage).
@@WebDevCodythanks for the clarification! I need to learn more about it.
Can we use supabase instead of convax?
you can use whatever you like.
Sure, it doesn’t compare to the dx of convex from what I’ve seen. The supabase subscriptions require listing out lower level database insertions, table names, etc. and I found supabase typescript support to be kind of crappy
Wondering, and am not sure if you said it in the video, but is this Convex something you going to use as a standard now or is it optional when building projects?
It’s a paid BaaS, so it’s not a standard. I’m just saying I’m using it for this project because the tech aligns with the project needs
@@WebDevCody right! That makes sense, cool that your showed us! Thanks Cody 💪🏻
Thanks a lot. ❤
Firebase also has a realtime database, I use firebase mainly for mobile development
Have you used Supabase? How does Convex compare?
Awesome video
I'd be *very* interested in learning about the sst setup, how to learn aws cdk, the github deploy etc. (happy to pay if you make into a course)
Good job babe!!!
i'm interested about pros and cons of convex vs traditional real time db like Firebase
Is this custom ML's or just wrapping gpt?
Add an option to regenerate sections of the project with additional suggestions. Let us iterate on the plan.
I am sorry for my lack of understanding, but why can't we just update our UI directly when actions are complete and successful(like a fetch API call), why are we sending updates to convex and being dependent on it?
When you run a next server action your page has to wait until the action finishes before you can redirect. Secondly, you can run concurrent actions in nextjs (at least a proper way to do that is not outlined in their docs). Third, it’s not outlines if actions should be used for async functions; there are ways to stream in the results from your actions, but I’ve seen strange timeout issues due to it.
Overall I’d avoid server actions for any async tasks.
You could have put hasura before the postgres and you instantly would have a graphql api with subscriptions + authz. Additionally full type safety with codegen😊
graphql subscription uses websockets?
@@skyhappy yes
@@ilijanl what is authz?
Sounds interesting! Is there any reason why not choosing Firebase or Supabase?
I find convex superior, and I'd avoid google products; they kill to many services all the time
@@WebDevCody sounds reasonable! Personally I love Supabase more and it's not a Google product but nice to know about convex.
@@WebDevCody Been using firebase for quite sometime. A real game changer ... Google kinda sucks when it comes to killing products but till this far firebase has been reliable .
You could keep Postgres and use hasura in front, that gives you realtime graphql subscriptions.
I dont quite understand what the importance or practical difference is between normal db (postgress/"non" real time DB) vs real time DBs. Wouldnt you just fetch/query each refresh or view? Or is that potentially too costly for apps with large customer bases (millions of customers), hence the need for a realtime db?
Imagine you have a page which shows a poll and user votes on answers. Anytime someone votes, you’ll need to loop through all users actively on the page and emit websocket events any time someone votes. Then the users ui needs to refresh your grab the latest votes. To achieve this you have to write a lot of extra websocket code and server infrastructure.
With convex, you write one query, and that query automatically updates any time the database data changes.
Can you make tutorial on convex? I did a basic setup but im kinda lost
I have one on my channel
Do you have supabase crud tutorial
If I understand this correctly, it seems you switched away from PostgreSQL not because of limitations in Postgres, but because of limitations in Next.js. Is that correct?
Your whole diagram looks like a pretty standard backend with Postgres, so Im wondering where the problem is.
the problem is how do I update the UI the instant data processing finishes on my api. For that, I need polling, or SSE, or websockets. I personally didn't want setup my own websocket server or the logic for emitting events when jobs finish, so I opted for using a realtime BaaS which takes care of all of that for me.
@@WebDevCody Ah, that makes sense. Thank you!
Nice man.
What is the use of real time database in this contex?
real-time databases are so nice, that's why I love Firebase Firestore lol
Been using firebase for quite sometime. A real game changer
supabase does have realtime updates as well as file storage
I didn't like the icons, designs and images that were generated. Is it possible to have different prompts for generating them or have chatgpt borrow inspiration from other websites?
But it is a nice product for brainstorming project ideas.
thanks for the feedback!
So convex is basically like Firebase?
No clue, I don’t touch google tools, but I know convex argues they are a better firebase
Yeah I heard you say you havent used Firebase later in the video @@WebDevCody
What's theme you're using?
bearded theme stained blue
Like an experimental idea for the project to go further.
Each section connect to its own AI agent and tell it to implement those ideas.
Later, we'll need global agent, that will put the project together.
Make it short - experimental stuff how the project idea can become real project, almost fully managed by AI agents.
Firebase has all that plus auth and stripe integration...Though it doesn't have client generation so the dx for convex is superior in that regard.
🤷♂️ I've never used firebase; I'm not a fan of google products
@@WebDevCody brother u need ot try it can ez build products under
Firebase has no foreign keys while convex does. Make data modelling way easier with convex. Also auth isn't a big difference when things like clerk exists. even simpler than firebase acutally.
Is convex similar to supabase or pocketbase?
Similar to firestore
Using Redis as intermediate storage and Postgres as final storage, would probably get similar performance.
What's wrong with react-query + pocketbase?
React query isn’t real time, so convex is gonna give me a websocket event at the moment the data I need to listen for changes, but let me know if pocket base provides real time data updates
does it realtime update?
Looks like there pro plan only goes to 50gb of storage. Also I thought firebase and Supabase also had this feature? Prisma orm is building a feature like this but you got to pay for it
Convex is pretty goated not gonna lie
💯
have you checked our SurrealDB?
nope I haven't heard of it
@@WebDevCody then check it out, you'll be amazed at its versatility as a database
you need to submit this to YC bro
Good choice. postgres and RDBMS in general are mostly for strict atomicity and transactions. (Banks, fintech, etc) Your use case is defo not a good fit for postgres
Isn’t hasura better? It has same realtime features that are based on graphql subscriptions
A sort of Firestore from Firebase.
I don't understand why use web socket,we can use different request for things .
why not use web sockets?
I use Firebase quite a bit...
Convex should open his system to have the self hosted opportunity. I am afraid of vendor locking in long term.
If Convex shuts down his service, I have to do the same time...
Thanks for introducing me to Clerk, amazing DX so far.
Clerk is really cool, amazing tool! It saves you so much time.
Stop playing with my emotions!
Is there a reason why are you not using firebase? did you even considered? our is just not too new enough?
I don’t touch google tools personally
I don't get it. Why do you need to wait 2 minutes for everything to finish? Why not just generate initial metadata and directly navigate them to the page where subsequent actions will occur? Since you're already using WebSockets, anything that needs to listen for real-time updates will still do so
I'm not sure what you are confused about, that's exactly what I demoed in this video. I click create plan in the modal, it redirects me to the plan page, the events come in as the actions finish, my UI updates.
@@WebDevCody I apologize for commenting in the first minute. You mentioned that you wait two minutes for everything to finish before navigating to another page, and then proposed a real-time database as a solution. I was pointing out that a similar result can be achieved with what I mentioned, but since you're using Nextjs server actions, as mentioned in the comments, it prevents navigation, making my suggestion irrelevant. However, I still wish you could explain why a similar result can't be achieved without a real-time database like Convex. I'm new to Next.js, so pardon my ignorance.
@@kabriid yeah so a similar result could be achieved without convex, but it would require either doing polling from the front end (a bit inefficient but easy to implement where you keep querying your API to see when the sections are done), you could do Server Sent Events which work fine by require a persistent connection which doesn't really work well with serverless deployments, or the other approach is you setup your own websocket server and you send events to your client's browser as the sections finish but this requires either managing your own websocket server or using a third party service (like pusher) to make it easier to push events to the browser.
out of all those options, I decided I rather just use a service that already have websockets built in and a very easy interface for having my UI refresh when stuff finishes.
Man doing anything to avoid server sent events for like 2 videos straight now
I miss the old thumnails, they seemed more authentic.
I get that but this space on TH-cam is very competitive. Creators need to stand out and draw new viewers in. If they don’t, their channel will stagnate likely die
Id rather see Cody continue making videos and the content behind the thumbnail is still great!
@@beakerbkrI get what you're saying. But just check where you have more views
@@dejangegic when Linus tech tips stopped using clickbait titles and wacky thumbnails their viewership declined steeply. They quickly went back to their click bait video titles and thumbnails
I'm here for the videos, but not gonna lie this thumbnail bring more of my attention
@@beakerbkrexactly!
Doesn't supabase have real-time
I really don’t understand why a db switch is needed for this. The issue you’re solving you’re solving in the wrong architecture layer.
Now you’re tied to a proprietary DB SAAS.
Can I work for you for free as your apprentice/assistant?
Just use supabase realtime and stick with Postgres
But Postgres also can be real time
with what, pg_eventserv ? I'd rather not have to think about reading through setup guides when I can just use a BaaS in like 5 minutes.
You JS guys are funny. Really funny. We have a defintion of real time in software development. What are your
boundadiers?
Realtime in internet with unpreidcable latentencies. Common. Say how do you do it.
The main part is JS, TS ans so are misplaced languages for backend dev.
JS developer can create an entire start up company in a week 🤷♂️
...go off though
@@WebDevCody others not?
Crud is simple. Very simple.
The single reason why the folks doing js at the backend is the madness to dont want learn an appropriate language for the backend.
Prototyping is easy. What about maintainability, security, code protection, dependency managent.
Dude you need to download the whole node repo for a fucking hello World.
And the worst thing you are propagating is a db that only available as a cloud service?
You don't got it that is a horror story of an architecture. One company is closing in your business is gone.
What about data governance?
I should go off?
Think about your education.
@WebDevCody you don't get my points dude.
That's not about elitism or evangelism of an approach.
Then persuade me with your wisdom.
1. Convex is a cloud only solution right? Why I should trust you my data?
2. What is if your little company gone. Then my business is gone as well right?
3. That is your defintion of real-time in your doc.
What makes Convex realtime?
Convex’s realtime database tracks all dependencies for every query function. Whenever any dependency changes, including any database rows, Convex reruns the query function and triggers an update to any active subscription on the client
You are kidding me. Don't you?
Real time gives you gutanties about processing the whole stuff within defined boudaries. And you must do chek of your code if you can hold this boundaries. You will do it with js. I will know how.
You can say that your db is fast. But realtime is somewhat other.
4. And again. Js is pain in the ass for backend. Why the stuff with webassemly is comming more and more popular? The future of js even for frontend is something like a byte code for the browser. Backend needs more the fast clicki clacki stuff.
@@suikast420 I'm sorry your feel me making a video about javascript and using a BaaS is leading people down a wrong path. Like I mentioned, I'm doing what works for my side project. If someone else watches this video and thinks postgres is bad or using a BaaS is the proper solution every single time, that's on them. My project has specific needs, I has limited time to work on these side projects, and my current skill set is mainly node and javascript. I've shipped code to production for many years that uses nodejs, so I think your ridicule of javascript and node is a bit harsh. Every language can accomplish the job if you know how to use it.
@@WebDevCodyyou were too generous in your response to this guy who criticizes with 0 substance
whats so special about postgres ? why people use it ? i hate that role and template model and it does not provide any major benefits over mysql in terms of performance
I've heard that PostgreSQL excels in handling complex queries, thanks to features like materialized views.
Moreover, it boasts various specialized data types (e.g., BSON, XML, geospatial), stored efficiently as binary data.
Extensions empower users to add custom features, including data types, authentication logic, and stored procedures in languages like Python.
Additionally, some even add supports for REST APIs and real-time pub/sub on top of Postgresql.
As a free and open-source enterprise database akin to Oracle or MSSQL Server,
it provides a cost-effective alternative for those seeking to move away from pricier enterprise databases,
even though not every user may find its myriad features essential.
Because you don't need to care about your data?
Oh God now he’s putting weird facial reaction thumbnails in all his videos. The previous one looks like he’s holding in a fart.
I was holding a fart; how else could I make a realistic facial expression?
@@WebDevCody Well then please promise a more realistic farting expression in the next one.
@@rawper23 I’ll buy some more beans and coliflower, ill keep you updated
I don't understand how you can't do this with postgres and webhooks....
OMG... this is TSQL all over again... triggers in the database...
You can, I decided not to, it’s that simple
Legend 🫡